r/databricks • u/tony-dang • Jun 03 '26
General Saw a super cool Databricks explainer video on Instagram
Original post: https://www.instagram.com/reel/DZFnttWijxS
r/databricks • u/tony-dang • Jun 03 '26
Original post: https://www.instagram.com/reel/DZFnttWijxS
r/databricks • u/[deleted] • Feb 16 '26
Hi everyone,
I cleared the databricks data engineer associate yesterday (2026-02-15) and just wanted to share my experience as I too was looking for the same before the exam.
It took me around 1.5 months to prepare for the exam and I had no prior Databricks experience.
The difficulty level of the exam was medium. This is the exact level I was expecting in the exam if not less after reading lots of reviews from multiple places.
>The questions were lengthier and required you to thoroughly read all the options given.
>If you look the options closely, there would be questions you can answer simply by elimination if you have some idea (like a streaming job would use readStream)
>Found many questions on syntax. You would need to practise a lot to remember the syntax.
>I surprisingly found a lot of questions in autoloader and privilege in unity catalog. Some questions made me think a lot (and even now I am not sure if those were correct lol)
>There were some questions on Kafka, Stdout, Stderr, notebook size and other topics which are not usually covered in courses. I got to know about them from a review of courses on Udemy. I would suggest you to go through the most recent reviews of practice test udemy courses to understand if the test is as per the questions being asked in the exam.
>There were some questions which were extremely easy like the syntax to create a table, group by operations, direct questions on data assets bundle, delta sharing and lakehouse federation (knowing what they do at the very high level was enough to answer the question)
How did I prepare?
I used Udemy courses, Databricks Documentation, Chatgpt extensively.
>Udemy course from Ramesh Ratnasamy is a gem. It is a lengthier course but the hands on practise and the detailed lectures helped me learn the syntax and cover the nuances. However, the level of his practise tests course is on the lower end.
>Practise tests from Derar on Udemy are comparatively good but again not at par with the actual questions being asked in the exam.
>I would suggest not to use dumps. I feel that the questions are outdated. I downloaded some free questions to practise and they mostly were using old syntax. Maybe in premium they might have latest questions but never know. This can cause you more harm if you have prepared to some extent.
>I used chatgpt to practise questions. Ask it to quote documentation with each answer as answers were not as per the latest syllabus. I practised the syntax a lot here.
I hope this answers all your questions. All the very best.
r/databricks • u/Kira-1996 • Aug 15 '25
I just passed the Databricks Certified Data Engineer Associate exam and wanted to share a quick brain-dump to help others prepare.
My Experience & Study Tips: The exam is 90 mins / 45 questions, mostly scenario-based, not pure theory. Time management is key. I prepared using the Databricks Academy learning path, did lots of hands-on labs, and read up on DLT, Auto Loader, Unity Catalog in the docs. Hands-on practice is essential.
Key Exam Concepts & Scenarios to Expect
DataFrame & Spark SQL API
Aggregations using groupBy(), sum(), avg(). Interpreting Spark UI metrics. Handling OutOfMemoryError (filtering, driver sizing).
2. Data Ingestion & DLT
Error handling in pipelines (drop/quarantine/fail). cloudFiles syntax in Auto Loader. Schema evolution modes (failOnNewColumns, addNewColumns). @dlt.table vs @dlt.view
3. Delta Lake & Medallion Architecture
Bronze/Silver/Gold layering. Behavior of OPTIMIZE.
4. Compute & Cluster Management
Choosing correct compute (Serverless SQL, All-Purpose, Job Clusters, spot instances). Job output size limits.
5. Governance & Sharing
Delta Sharing for external partners. Lakehouse Federation to query external DBs in place. Unity Catalog privilege model (e.g., Schema Owner).
6. Development & Tooling
Databricks Connect for local IDE development. Databricks Asset Bundles (DAB) in YAML.
Focus on picking the right tool for the scenario and understanding how Databricks features work in practice. Good luck! Drop your questions or share your own experience in the comments.
r/databricks • u/JackCactusLaFlame • Feb 25 '25
r/databricks • u/databricks • Jun 13 '26
We're excited to announce ๐ข๐บ๐ป๐ถ๐ด๐ฒ๐ป๐, a meta-harness that combines, controls, and shares your agents.
The best teams already mix models and harnesses and design loops that drive teams of agents. No single harness can keep up with that alone. So we built the layer above โ we call it a ๐บ๐ฒ๐๐ฎ-๐ต๐ฎ๐ฟ๐ป๐ฒ๐๐.

Omnigent sits above the tools you already use, Claude Code, Codex, Pi, and your own agents, and gives them one shared layer:
๐๐ผ๐บ๐ฝ๐ผ๐๐ถ๐๐ถ๐ผ๐ป: combine models, harnesses, and techniques without rewriting code, and switch between them with one-line changes
๐๐ผ๐ป๐๐ฟ๐ผ๐น: stateful, data-centric policies and cost budgets enforced at the meta-harness layer, not via prompts โ let agents run without watching them
๐๐ผ๐น๐น๐ฎ๐ฏ๐ผ๐ฟ๐ฎ๐๐ถ๐ผ๐ป: share a live agent session via URL with full history, so teammates can review, comment, and steer in real time
Learn more at:
๐ omnigent.ai
โ๏ธ discord.gg/omnigent
r/databricks • u/Significant-Guest-14 • Mar 20 '26
What do you think?
r/databricks • u/sai-nageshwaran • Aug 13 '26
Databricks just closed a $5B round at a $190B valuation โ up from the $188B term sheet from July, after crossing $7B in revenue run-rate and growing 80%+ YoY.
r/databricks • u/Remote-Grapefruit214 • Oct 05 '26
Hey Guys, Data Engineer with 3.2 YOE here,
I have been working on and off on databricks for multiple projects across domains like Retail, logistics and Now Pharma.
In my current role, I focus on creating end to end pipeline with different data sources which often get refreshed on a bi-weekly basis for pharma related data
The client is extremely demanding and has no technical expertise to gauge how much time it really takes to build and maintain something so complex and therefore expect the team to use Claude teams license and also Genie
The problem with Genie is that, I can not trust it, It will not listen to my instructions even after having a separate instructions.md which I update on a daily basis, in fact I update this after each session is complete, and no, I do not use AI to update this, every change request that the client and the client's team has goes through my own words of instruction updates, I have an entire markdown file which has contexts for multiple islands of projects, workspace folders and notebooks that I have to maintain and transition to devops team.
I also have created 4 different skill family for Genie, in relation to
migration,
devops,
maintainence and
review.
And it is so disgustingly bad at handling all this context, that it fails me across all 4 areas.
I have seen it lie to my face multiple times,
Casting columns as null,
altering data types without asking consent,
never following my work stream and process in a sequential format
Having set the wrong notebook paths in a task even though i explicitly tell it exactly what to do.
It often confuses streams of works and ends up mixing so many things that remove this knot and fuck up itself costs me so much time!!!
It is very poor in adhering to the domain specific instructions that we provide
More than 3 requests in a chat, and it is practically useless
And branching of chats is the most useless feature that they have introduced.
I have created multiple diagnosis queries to check the work of this agent, and it will straight up lie to you, so much, so convincingly, that you know it knows all the biases you have and it even goes a step ahead and just narrates a story that you can provide to your team in the stand-up.
My only concern is, after all this bullshit, my team, of nearly 6 (some of them have never worked in databricks or delta table environments like this) was charged $990 dollars in the month of august, for this shit output??? Have some shame Databricks, fix your worthless product or remove that feature or stop charging so much if you are beta testing in actual prod.
Today, I am writing this post because of something that I caught live, that pushed me to the edge.
Something that is critical, production level issue, which If I was not paying attention while merging could have been a huge disaster, and mind you the pipelines I make are client facing, what is even more distrubing is the fact that it will just make so many unnecessary changes to a simple query or a pyspark function just enough so that the tests are passed.
(So basically it does not want to get caught, and makes a mistake so that we can prompt it again to fix this mistake and then Databricks can charge us more, this looks like a dark pattern to me)
In your preview (prima-facia), everything is good, the tests are passing, the job is running, but holy!!!,
it was casting 3 columns which are essential for downstream processes as null.
It boldly suggests that we remove those 3 columns, and or cast them as null. How is this a solution Databricks? The AI is supposed to have more context than me because it is a agent made specifically for Databricks Environment Correct?? That is how it is sold?? And upon spending just 2 mins, I realized that the fix that it is suggesting will blow up critical information that the client team should see in the app because it is not even there in the first place, and all of this is because it can not read the correct notebooks, even if you tag it.
It had set a retired notebook's path in the task and very boldly claiming that everything is good.
My concern is, let us say that there is a junior who is good at SQL, but has never maintained projects like this in this scale, he/she would be overwhelmed and won't even be able to find the mistakes that the agent is stating, because unlike GPT 6 or Opus 5.5 Genie does not openly claim that it does not know something,
it does not ask for more context,
it does not state that something is missing,
it just patches with some assumption that we don't even provide in the first place.
My advice to any Juniors out there, Stop using Genie, Completely.
The product is not ready, it is not good.
r/databricks • u/Significant-Guest-14 • Sep 21 '26
Databricks give us a lot of cost data, but usually not where we actually need it. When I open a job run, I can see duration, compute, tasks and logs, but not what that run actually costs. When I create a cluster or SQL warehouse, I choose the size first and DBU, and discover the real cost later.
Thatโs the problem weโre trying to solve, bringing cost directly into the Databricks UI. We created a free browser extension, with no backend or telemetry https://finopsway.com/
r/databricks • u/Miraclefanboy2 • May 19 '26
I've been highly critical of genie or databrick assistant for quite some time now. I even have a post here criticizing it, but kudos to the DTB team! It is sooooo much better now that i dont even bother connecting claude code via dev ai kit anymore.
r/databricks • u/[deleted] • Sep 20 '25
Coming straight to the point who wants to clear the certification what are the key topics you need to know :
Be very clear with the advantages of lakehouse over data lake and datawarehouse
Pyspark aggregation
Unity Catalog ( I would say it's the hottest topic currently ) : read about the privileges and advantages
Autoloader (pls study this very carefully , several questions came from it)
When to use which type of cluster (
Delta sharing
I got 100% in 2 of the sections and above 90 in rest
r/databricks
•
u/4DataMK
•
Mar 27 '26
r/databricks • u/hubert-dudek • Jun 25 '26
Fast reads and fast writes, thanks to Lakebase, including writes to open formats. It creates a unified platform both for analytics and operational use cases. #databricks #DataAISummit
https://databrickster.medium.com/my-favorite-announcements-from-the-data-ai-summit-2026-317fc68d4e75
https://www.sunnydata.ai/blog/data-ai-summit-2026-announcements
r/databricks • u/EffectiveSignal4763 • Aug 17 '25
I was a bit scared with the recent syllabus updates but I made it through this morning.
I studied from Databricks partner academy (16-18 hours course videos), used ChatGPT for mock tests, and finally did 4-5 mock tests on Udemy in the last 3 days.
Happy to answer any questions or help anyone.
r/databricks • u/Significant-Guest-14 • Sep 13 '26
"Just ask Genie." You hear this phrase more and more often when questions about Databricks arise. It frequently appears in Academy materials, too.
AI reduces routine work itโs convenient, and we get used to it. But do we stop thinking in the process? The problem arises when a person no longer understands why a given answer is correct.
Does AI really free us from the need to know the details, or does it actually make fundamental knowledge even more important?
r/databricks • u/Fearless_Jeweler1415 • Sep 15 '25
Hi all,
I'll be sharing the resources I followed to pass this exam.
Here are my results.

Follow the below steps in the order
Refer to the recommended material by Databricks for the professional course
Databricks Streaming and Delta Live Tables
Databricks Data Privacy
Databricks Performance Optimization
Automated Deployment with Databricks Asset Bundle
Now do exam mock questions from skillcertpro.
Do the first three very attentively since the exam will follow very similar questions
While doing this make you refer to the relevant area in the documentation. Eg: if one question tests on Z-Ordering, make sure you read everything on that area in the Databricks documentation. https://docs.databricks.com/aws/en/delta/data-skipping
Some of skillcertpro answers are wrong or may not make sense in the present. So you must refer to the documentation and come up with the correct answer.
Do the next two mocks as well. Some questions might be useful
You might realize you have doubts in some areas while taking the mocks, so please create your own notes referencing the documentation. I used notion to take down notes.
Now watch these youtube videos. Every time you are not sure of the answers please refer to the Databricks documentation and figure out the answer.
Watch this video and the comment section. He has attached some important stuff in the comment section (I got this from another reddit post). https://youtu.be/yDWPtSGXDhM?si=HckOvVSe13zazAsu
This youtube video showcases questions and answers. Some answers are wrong so please use your judgment to figure out the correct answer. https://youtu.be/0Qp9j6c2RlQ?si=9xyDMJb5f2nBzsKQ
This youtube video is the part two of the above video. https://youtu.be/LQ-58qJLDjw?si=z1G5j04DnQdL5dEs
Repeat step 1 at a higher playback speed. Now by doing this you would further clear out the doubts. Trust me you would feel really good about yourself when the doubts get cleared, especially in structured streaming.
Now do the first three mocks of skillcert pro again at a very fast pace.
Take the exam!
Done, That's it! This is what I did do pass the exam with the above score.
FYI,
I directly did professional certificate skipping associate certificate
I have around 8 months of Databricks work experience. I guess it helped me a bit with the workflows part.
I got 60 questions. So please makes sure you practice well, It took me the entire two hours.
You need 80% to pass the exam. I guess you can only get 12 wrong. I believe they have 5 non-credit questions which will not count to the score.
If you get stuck in a question you can flag that question and get back to it once you finish answering rest of the questions.
Good luck and all the best!
r/databricks • u/KnownConcept2077 • Jun 11 '25
Did not have republican political bullshit on my dais bingo card. Super disappointed in both DB and Ali.
r/databricks • u/Significant-Guest-14 • Jul 20 '26
I recently contributed to REbricked, a community project that tracks Databricks product renames.
It includes:
Product rename history
Links to official documentation
A short quiz to test yourself
We're still improving it, so feedback is very welcome. If we missed a rename or you have ideas, let us know.
r/databricks
•
u/Significant-Guest-14
•
Aug 24 '26
Recently, a new post about icons by Eduardo Rabelo appeared in our Databricks Community Articles on Medium.
I don't know why Databricks itself didn't create this set, but I think it's something many people were missing. Now we'll have to keep up with maintaining it, as Databricks releases quite a lot of new features frequently.
https://oieduardorabelo.github.io/databricks-architecture-icons/
r/databricks • u/IanWaring • Sep 20 '24
r/databricks • u/Nomyfir • Apr 30 '26
Review Materials:
Derar Course (Both Prepation and Practice Exams)
Introduction to Databricks : A beginner's guide by Medium
3.Get Started with Databricks for Data Engineering by databricks
4. Utilized Chatgpt to understand concepts in scenario
r/databricks
•
u/oneaffidavit1
•
23d ago
Below are the things covered in the video:
Overview: drag the month slider and every figure on the page moves with it, including a cash flow diagram from sales through to profit
P&L: the statement drawn as a ladder from revenue down to profit, plus a bridge showing why we beat or missed budget
Cafes: ranking, a map of the Front Range, a margin heatmap by month, and how fast each new cafe paid back its fit-out
Menu: which items earn their place and which are just popular
Cash: a 13-week forecast, overdue invoices by age, and the supplier payment calendar
Close: the month-end checklist as a board, with how long each close took historically
What-if: move price, volume, wage and ingredient levers and watch profit change
A one-page monthly report that prints to A4
Under the hood: seven Delta tables in Unity Catalog, read through a SQL warehouse, with the app running as a Databricks App in Python. All the finance logic (consolidation, budget variances, the close tracker, the cash forecast) sits in one file that a finance analyst could read.
r/databricks • u/Significant-Guest-14 • Jun 24 '26
Got to visit the San Francisco HQ after the Data + AI Summit last week. Not open to the public โ Databricks MVP status got me in. Figured some of you might be curious what it's actually like inside.
Quick rundown:
The office feels like the product. No unnecessary flash, everything has a reason.
Food ordering system โ employees order individually through an app, each gets their own delivery. No cafeteria.
Dedicated bike room โ not a hook on a wall, an actual room. SF culture is fully absorbed.
Free merch stand for visitors โ great idea in theory. After DAIS, it was completely wiped out. Showed up too late.
Ice cream machine โ didn't get to try it. Still thinking about it.
Tried to recreate the famous balcony photo. The view is legitimately incredible. Slightly terrifying height, though.
From the same balcony, you can see Meta, Salesforce, Anthropic, and LinkedIn offices across the street. Would love to visit those too โ unfortunately, I don't know anyone there. Yet.
r/databricks • u/szymon_dybczak • May 22 '26
With Genie Code, you can now add a Tableau or Power BI file and have it build an AI/BI dashboard that replicates your existing visualizations - while connecting them to metric views that mirror the underlying business logic.
Import BI files using Genie Code - Azure Databricks | Microsoft Learn
Many organizations have years of BI logic embedded inside workbooks, reports, templates, and semantic layers. Rebuilding that logic manually in a new platform can be slow, error-prone, and difficult to govern.
This new workflow helps accelerate that migration path:
Upload a Tableau or Power BI file directly into Genie Code Supported formats include .twb, .twbx, .tds, .tdsx, and .pbit.
Use the /importBI command in Agent mode Genie Code imports the BI asset and generates an AI/BI dashboard.
Review the generated dashboard and metric views Measures and dimensions from the original file are transformed into metric views.
Promote metric views to Unity Catalog This makes them reusable across dashboards, Genie Spaces, and notebooks, while adding governance, lineage, access controls, and discoverability.
Currently, there is also a 100 MB limit for direct file uploads. For larger files, the recommended path is to store the file in a Unity Catalog volume and reference it directly, for example:
/importBI @/Volumes/my_catalog/my_schema/my_volume/sales_workbook.twb