r/dataengineersindia • • May 19 '26

General WTF is this

Post image
269 Upvotes

I know there is some subtle discrimination in job market may be based on region, religion, caste etc..,

But this is just blatant. Is this normal?


r/dataengineersindia • • Oct 03 '25

General Google Data Engineer Interview Experience

255 Upvotes

Hi, I am the guy got into Google as a Data Engineer, this post is a common response for the most asked question of my previous post - link, "pls give interview experience", I personally don't think knowing my interview experience is that helpful since I am not going to go deep but I wrote this experience in a very monologue and critique-type style. This is not a strategy guide, its just experience of a random DE who managed to attend all rounds of Google, you will find 100's of these online (which would probably be more informative than this), so nothing special. Here goes nothing. Hope this helps, it took me 1.5 hours to type.

Disclaimer: This is a stream-of-consciousness account of my thoughts.

Note: To respect the confidentiality of the hiring process, I will not be sharing specifics on the questions asked. I will only discuss the high-level experience here.

My intention is not to brag, but I consider myself a decently above-average Data Engineer in terms of performance and career experience, but not a brilliant one, not even close to one. This is mostly because I don't particularly enjoy coding. While I'm reasonably good at it, it's not something I'm passionate about. I didn't even know how to code before starting my job at a WITCH company, and I wasn't hired as a Data Engineer. The project I was assigned to needed one, and I fell into the role. It just so happened that I was quite comfortable with Data Engineering, as it was a mix of some coding and being an SQL junkie (I've loved SQL since college).

I believe my experience and skill level is relatable for the average Data Engineer. If I can inspire people to bridge the gap between 'average' and 'above-average,' I'll consider this write-up a success.

Considering all of the above, I should also preface that I am, to a degree, obsessed with optimizing my professional profile for visibility. I have probably spent more hours trying to perfect my LinkedIn profile, my Naukri profile, and my resume than most. Basically, I do anything that can give an above-average data engineer like me a fighting chance against the brilliant ones.

Just to show the severity of this obsession, here is a screenshot of my Naukri profile performance from today: https://imgbox.com/YJWzbGx2

Profile

  • Education: B.Tech. from a Tier-3 Engineering college.

  • WITCH Company: 2.5 years (1 promotion to Senior DE)

  • Big 4: 2.5 years (No promotions)

  • Total Work Experience: 5 Years

Recruiter Screening

I received an InMail from a Google recruiter asking if I would be interested in exploring an opportunity for a Data Engineer position at Google. My first reaction was to ignore it, assuming there was no chance of me getting in anyway. After a few hours, I thought, "Why not give it a shot for the heck of it?"

The reason for my hesitation is simple: I'm not a great coder and don't enjoy code-heavy jobs. On the contrary, I LOVE data modeling, warehousing, architecting, and system design. I was already on a path to transition into an architect role, so I treated this screening as just an experiment.

The recruiter scheduled a one-hour meeting (I did no prep). The recruiter explained the role and its responsibilities, and I was immediately all ears. It was a very architect-heavy role. After the explanation, the recruiter asked me two SQL coding questions, one Python and one Spark coding question, and around 8-10 theoretical questions, plus the basic HR-type questions about why I would be a good fit.

  • Self-critique: I struggled with one Python question, but the rest went decently.

  • Result: Hire signal from the recruiter, approved by the Hiring Manager. Moved to the RRK (Role-Related Knowledge) round.

I asked for three weeks to prepare, as I needed to study DSA. My sole focus for those three weeks was creating and executing a DSA study strategy. I did not practice any SQL, Big Data, or Cloud concepts.

RRK (Role-Related Knowledge)

The RRK round for this role is a discussion where the interviewer tests your understanding of Big Data and the Cloud. Consider it 80% theory and 20% coding, but this can shift based on the interview; there's no hard-and-fast rule.

I was asked a ton of technical questions on Big Data technologies, warehousing, GCP services, and hypothetical questions on arriving at solutions. 

  • Self-critique: This round was my time to shine. As an aspiring Data Architect, discussing these theoretical topics is my strong suit, and I felt I made a very strong impression.

  • Result: Strong Hire signal. Moved to the GCA (General Cognitive Ability) round.

Note: From the recruiter's reaction, I understood that a "Strong Hire" signal in any round at Google is a big deal. If you get this rating, you're pretty much cemented as a top candidate compared to your competition interviewing in parallel (and trust me, there is competition).

GCA (General Cognitive Ability)

The GCA for this role was a coding round, split into two sections: Data Modeling and DSA.

First, I was asked to create a data model for a real-life, practical system. Then, I was asked 3-4 SQL questions that I had to solve based on the data model I provided. This is a tricky scenario, if you mess up your data model, you won't be able to solve the subsequent questions. I was also asked a few theoretical "what-if" questions.

Next, we moved to DSA. I was asked a unique question that involved a concept similar in pattern to a LeetCode Medium problem. (I won't go into detail, but trust me: when you only have 30 minutes to discuss, solve, optimize, and code a problem. I solved it with a few hints.

Overall, this round confirmed that the level of DSA required for a Data Engineer position, even at FAANG-level companies, is not excessively high.

  • Self-critique: Surprisingly, I performed below average in data modeling for my standards. I was overconfident in my data modeling and SQL abilities and should have done some prep here. I did zero prep, focusing only on coding since that's my weak point. I would give myself a Lean Hire or No Hire based on my expectation of the round as an interviewer.

  • Result: Hire. Moved to the Googleyness round.

Googleyness

The recruiter had warned me that a lot of people mess up this round, so I prepped for it like crazy for four days. I was asked two hypothetical and two behavioral questions, and the round took about 40 minutes.

Result: Hire.

After this came the offer negotiation and the offer letter rollout.

Total time from first contact to offer rollout: ~2 months.

Ratings

Interviewers: 10/10

Format: 10/10

Difficulty: 10/10

Stress Testing: 11/10

Closing thoughts: Google interviews are unique and atypical of standard interviews at other companies. If you go in without understanding what Google is testing for in each specific round, you will likely be unsuccessful. This applies to all rounds, INCLUDING Googleyness.

Over these two months, I also managed to bag two other offers: one from Amazon and another from a service-based company that I really liked (if I had messed up the Google interview, I would have joined them over Amazon).

Companies I Interviewed For During This Timeframe:

  1. Capgemini (Offer)

  2. Barclays (Withdrew mid-process)

  3. Wipro (Rejected)

  4. EY (Rejected)

  5. Razorpay (Rejected)

  6. DoorDash (Rejected)

  7. Snowflake (Rejected)

  8. Amazon (Offer)

  9. Acoustic (Could not attend due to scheduling conflicts; Rejected)

  10. Meta (Rejected)

And that's a short "word vomit" of my experience and how I got into Google.

Side Note: Depending on the interest this post receives, I might create a series on preparation strategies for product and service-based companies. I could also cover topics like understanding different roles at various companies and curating your profile to your strengths as a Data Engineer. I have done extensive research on optimizing LinkedIn, Naukri, and resumes to maximize interview calls. I usually get 2-3 InMails or 3-4 Naukri calls per week from recruiters when my profile is set to "Open to Work." Otherwise, I get about 2 InMails and 2 calls per month (excluding TCS recruiter spam).


r/dataengineersindia • • Sep 24 '26 Bravo

General A Data engineer story

253 Upvotes

A year ago, around this same time, I was struggling in a service-based company earning around 4 LPA, even after having close to 3.8 years of experience.

I had been trying to switch for almost 1.5 years.

Almost no interview calls.

A few interviews came, but I got rejected.

At one point, I genuinely started wondering whether I was doing something wrong or whether I would ever be able to move into a better role.

Then in December, I finally received an offer from a product-based company.

The package wasn't exactly what I wanted, but I accepted it because I felt I needed that first breakthrough. I resigned and started serving my notice period.

And somehow, after that first offer, everything started changing.

By February, I had received 12+ offers/interview conversions, and some of the offers were around 6x my previous salary.

I finally joined a new company in March.

Then, just a few months later, another opportunity came my way.

I left that company in September and joined one of the world's biggest pharmaceutical companies.

So within roughly 8 months, I made two switches, and compared with where I started, my compensation became roughly 8x.

But salary is not actually the biggest thing I learned from this journey.

What actually changed for me?

I realized that although my title was related to Data Engineering, I wasn't really working on core Data Engineering work.

A lot of my experience was more on the support/backend side.

So instead of waiting for my company to give me the perfect project, I started creating my own projects.

I watched YouTube videos, studied GitHub repositories, understood existing architectures, and then tried building things myself.

That made a huge difference.

Once you understand how an end-to-end data engineering architecture works, interviews become much easier because you're no longer just memorizing answers.

You understand why things are being done.

For anyone trying to become a Data Engineer, especially someone coming from a support role, I would suggest focusing on:

SQL – Make this very strong.

PySpark/Spark – Understand transformations, joins, partitions, shuffling, optimization, etc.

Databricks / Microsoft Fabric / Snowflake – You don't need to master all three initially. Pick one and understand it properly.

dbt – Don't just watch tutorials. Build a small project.

Airflow – Again, create a project and understand how orchestration actually works.

Don't keep jumping between 20 different YouTube playlists.

Pick one good resource.

Learn the basics.

Practice.

Build something.

Then slowly move toward advanced topics.

Most importantly, master SQL and Spark. Once these two become strong, many Data Engineering concepts start becoming much easier.

My suggestion would be:

Learn SQL + PySpark + one cloud data platform + dbt + Airflow.

Build at least one proper end-to-end project using them.

Then start giving interviews.

Don't wait until you feel like you know everything. You never will.

Go for interviews, get rejected, understand where you went wrong, learn that topic, and try again.

Coming from a support background, I used to think that people working in core Data Engineering or product companies were far ahead of me.

They weren't.

They simply had knowledge and exposure that I hadn't built yet.

That can be learned.

So if you're currently stuck in a support role, earning less than you expected, getting rejected, or not receiving calls keep working on your skills and keep applying.

One good opportunity can completely change the direction of your career.

If I could make this transition from a support/backend role, you can do it too.

Good luck. Keep learning and keep going💓


r/dataengineersindia • • Oct 03 '25

General Got into Google! Never even dreamt of this! 5 YoE as a Data Engineer from Tier-3 to WITCH to Big-4 to now Google, I think I have seen it all. AMA!

Post image
214 Upvotes

r/dataengineersindia • • Dec 27 '24

General Interview Experience at Delhivery

214 Upvotes

Randomly applied through LinkedIn for DE-1 role.

Round 1 : 2 DSA + 1 SQL + Spark questions

I solved DSA questions using python (1hr round) but got extended for 15more mins

Q1 : Merge intervals

Q2 : Longest increasing Sub sequence

Sql : Friend Requests II: Who Has the Most Friends from leetcode

Spark related questions : Spark Architecture, join strategies, serializers and it's type, deployment modes in spark

I answered all these Spark questions in 2-3 lines each, as I spent an entire hour solving DSA and SQL question.

Interviewer was really helpful and was giving hints whenever I was stuck somewhere.

Round 2 : Project Architecture + Spark coding +Spark discussion + types open table formats in detail (delta format) + 1 SQL Question

Spark Coding : Reading files, using functions like when, otherwise etc.

SQL : select 3 consecutive records with same value Explained logic using LAG but wasn't able to implement it due to time constraints

Round 3 : TechnoManagerial (System/ Data pipeline design) Asked about my work experience.

Design an alert system for a Ola/uber. Example if a woman is traveling alone after 11 PM and the cab stops on a remote road for 10–15 minutes, trigger an alert. Also, integrate a 5-star safety feature for immediate contact.

YOE - 1.5 years

TechStack - Azure (Data factory, Databricks, Datalake), AWS (S3, EMR), SQL

Result - Selected

Edit - Current CTC : 8LPA (all base) CTC offered : 14.5 LPA (all base)

Resources I used :

Dsa - for practice Neetcode (Array, String, Stack, Queues, recursion), Love babbar/ Striver to understand the basics concepts

Spark: Yt channel Manish Data Engineer, Ease with Data

Sql : Leetcode Easy, medium level questions

Data Pipeline Design : Chatgpt (How to design pipeline for different scenarios)


r/dataengineersindia • • Aug 06 '25

General Learning Series: Post 1: Things needed to be Data Engineer

209 Upvotes

Hi All,

Thanks for such a great response on my previous post. The response provided me a lot of motivation to be consistent and help the community as much as possible. Keep Supporting me like this, Your encouragement keeps me going.

Let's get back to the work.

In this Post, I will be sharing what you all need at fresher and mid-senior level to be in Data Engineering field.

1. SQL

This is major skill needed to be a data engineer.

Where it is required: Both Interviews and Daily work

Level Needed: Medium to Hard

Where to learn/Practice: Here are the few Sites you can refer(These sites I have tried and tested).

* Stratascratch: This site is for beginners. It can be used by mid level as well. You can go to analytics questions. Choose Free Questions. Sort the questions from Easy to Hard Question. Go in sequence to get used to questions at each level. It has around 100 Free question which are enough to get hold of SQL.

* LeetCode: Once you are comfortable with all the questions provided in stratascratch, you can start with leetcode. Leetcode problem set is bit lengthy and complex. So, Once who are comfortable with SQL, you will be able to leetcode questions.

* DataLemur: You can do company specific question here.

Experience: Needed for all level from beginner to senior level.

2. Coding

You will need DSA for interview and coding for your daily work. While you don't need hardcore competitive coding, you should know Arrays, Strings, HashMaps, Queues.

Where it is required: Both Interviews and day to day work

Level Needed: Medium, However few companies like Google and Uber ask Hard leetcode questions to data engineer as well but that's a exception I haven't seen it in other Major companies(in which i have interviewed or where I have been)

Where to learn/practice: For Learning the code, Use any of youtube playlist to get started with basic. Then, start doing questions for that topics on Neetcode and Leetcode. Always Start with Easy questions with high acceptance rate then move forward, else you will lose your confidence. Also be consistent with your Practice.

Mostly company ask DSA in Python only for Data Engineer, however few prefer JAVA. This vary company to company and interviewer to interviewer. for e.g. In one of interview, interviewer asked to solve question using python but my friend was more comfortable in JAVA interviewer was ok for it.

In Most of companies, I experienced that interviewer is ok with any of language. Mostly people prefer python in data engineering. Some exception like Walmart only prefer scala or java.

Experience: For all levels

3. Data Modelling + ETL/System Design

In System Design interviews for Data Engineers, Companies ask to create a flow of Data(with services being used for the purpose) from source to destination with different scenarios like Real time data flow, batch data processing etc and how end user will be consuming the data. With this ETL/System Design, they ask us to create data model as well.

For eg. Create a Amazon's order analytics platform. you will have to mention what will the fact tables and what will be the dimension table. how would you extract the data , transform it and load it. which service would you use to provide the data to end user. You would to explain this with flow diagrams(you can use draw.io to create diagrams)

Where it is required: Interviews and Time to Time in work

Where to learn:

* The DataWarehouse toolkit by Ralph Kimball.

* Designing Data-Intensive Application by martin kleppmann

Experience: Mid level

4. Big Data Technologies

You should be familiar with the modern big data stack like Spark, Kafka, Flink etc.

For beginners, Spark is enough. For mid level, Kafka, Flink and other other big data technologies are also needed which are required for batch and real time processing. May be you haven't worked on all but you should know the purpose. for eg: presto is used to query on big data.

Also, There could be cases in which companies ask to write pyspark code for processing a file.

Where it is required: Both Interview and Real life

Where to learn: For spark, Spark: The definitive Guide and Learning Spark (both are written by Spark creators)

Experience: Beginner to Senior Level

5. Cloud Technologies

Pick any one and get good at it.

  1. AWS: AWS Provides free $200 for 6 months. you can learn AWS via AWS Blogs and there are youtube videos for that.

  2. Azure : Azure provides a full catalog of free services upto free amount and additional $200 for a month.

  3. GCP : GCP also provides $300 in addition to 20+ free tier services.

I don't have much experience with GCP and find it difficult to use, may be due to inexperience. AWS being easiest to use.

Where it is required: Mostly in day to day work but can be asked in interviews

Where to learn: Youtube has a lot of videos for this, you can start with any cloud basic certification videos. In those videos, they start with basic services and their usage. After that you can level up.

Experience: All levels.

if you have made it this far, thanks for reading.

Let me know in case you find anything missing or need more information.

Please upvote and share this as much as possible so we are able to help as many as we can.

Thanks all, Signing off, will meet you next post with other information you guyz asked.


r/dataengineersindia • • Apr 30 '26

General AI replaced 70% of our Data Engineering team overnight

198 Upvotes

I never thought I’d be writing something like this, but here we are.

I was working as a Data Engineer in a mid-sized team of 10 people. Over the past few months, leadership started pushing heavily into “AI-driven automation” — at first it sounded like productivity tools, code assistants, pipeline optimizations, etc.

But it didn’t stop there.

Gradually:

* Repetitive ETL jobs were replaced with AI agents

* Monitoring and debugging pipelines got automated

* Even data validation and transformation logic started being generated dynamically

Last week, I got laid off.

Out of our team of 10, only 3 people remain who now “oversee” the AI systems instead of building things from scratch.

The scariest part? This wasn’t a cost-cutting decision alone. This is a clear strategic shift:

> “Replace Data Engineering work with AI wherever possible.”

They openly said this is just phase 1.

I’m attaching some screenshots (internal discussions / planning docs) for reference.

I’m not posting this for sympathy — I’m posting because I think a lot of people are underestimating how fast this is moving.

If you’re in data, backend, or even software engineering:

* This is not “future risk” anymore

* This is already happening

* And it’s happening quietly inside teams

Now I’m trying to figure out what’s next — whether to adapt, switch roles, or move out of this space entirely.

Would really like to hear:

* Is this happening in your company too?

* Are people seeing similar team reductions?

* What skills are actually safe (if any)?

This feels like a turning point.

TL;DR:My Data Engineering team got cut from 10 to 3 because AI agents took over most of the work. Company plans to replace DE wherever possible. This is real and already happening.


r/dataengineersindia • • Feb 01 '26

General My last 6 months Senior DE interview experience

196 Upvotes
Warning long post no TL DR

I recently joined Crowdstrike as Data analytics engineer and was applying for Senior data engineer / Data engineer 2 (L4/L5) roles for last 6 months (interviewed at Uber, Amazon, Doordash got rejected at all places) been lurking in this sub for that time and wanted to share my experience , last company was salesforce (7 YOE)

Crowdstrike interview

HR reached out via Linkedin after applying

HR screening

General role and skillset understanding


Round 1 - Hiring manager round

Previous roles and responsibilities along with major projects executed most of the call was around projects, A data pipeline around pulling data from a location and he kept asking questions on tooling and added some scenarios like archiving, backfilling, data quality, schema change etc.


Round 2 - Technical round

3 people panel was there and call lasted for nearly 2 hours

Python - for loop based name lookup (easy)

SQL - Question started as Pandas question which I didnt knew so said will do it in sql and can use AI to translate it to pandas if required at end , were fine for SQL basic case/ coalese / group by question with null and error (division by zero) handling , some data related questions that what happens to code if this type of row comes were asked so had to work on data quality tests for it

Stakeholder management and scenario based questions


Round 3 - Skip Level Manager

Various STAR based scenario and discussion about roles and responsibilities for crowdstrike


Overall one of the easiest rounds I gave(NO DSA) and the process took about 5-6 weeks, but the friendliest bunch of folks so that really drawed me in to accept the offer along with being remote for now

Previous TC - ~55 , Crowdstrike TC - ~77

Other than crowdstrike gave interviews at Uber, Amazon, Disney, Doordash, Adyen, Netskope, Ebay , albertsons

Here is my application Sankey


Some pointers to share from my interview experience

Application and resume

  • Fortunately as I was already at a good product company calls were there but most didnt had budget so applying from Naukri didnt help I used career portal (most imp), linkedin , 6figr, uplers, instahyre to apply

  • Customised resume to job requirements helped as generic resume was mostly rejected need to have keywords aligning to job requirements and the tool coverage if possible

  • Referrals didnt help me as nowhere I applied through referral called me back but at least it helps in getting noticed as recruiter for sure looks at your profile even if its glancery

Technical

  • DSA is most important as I was rejected from Uber , Disney Amazon, Ebay due to it not just the solution but optimal solution , only easy mediums no hards

  • Topics I remeber were hashmap, tree, graph, recursion, stack, heap, two pointers, sorting, binary search

  • SQL will be hards didnt face recursion but sessionisation, gap islands, running sum, group by having and row number were there think about data too while wrinting query and ask questions if possible so as to not miss any flags in where clause

  • No AI tooling was allowed in interviews and even though I had some projects around it no questions were asked just DE fundamentals were expected

Data modelling

  • Practice with AI for the basic 5-6 businesses (eCommerce, cab , food delivery, social media , banking/finance) and for big tech prepare data model for their business and understand the north star metrics a business would have like eCommerce has retention and social media has engagment rather than focussing on making things strictly star or snowflake focus on metric you want to track/analyse and refine after discussion with interviewer

Pipeline/System design

  • I have only worked for batch so don't have steaming experience nor faced it in interviews so I just mention we can use pub sub to land this data in S3 that's it after that I have one standard pipeline I use everywhere based on there stack I am comfortable with as the interviewer is not looking at you to nail the tooling they are looking that do you understand tradeoffs so prepare that (for eg I prepared snowflake vs databricks but was not ready when someone asked why not redshift as rest of stack is aws so we would get discount for using it ) overall just be ready to explain your why before every decision

Hiring manager

  • ⭐️⭐️⭐️ use Star and focus on your action and result a lot don't be afraid to inflate your results but don't go overboard and be clear what part of project you want to ownership of as mostly it's given that you are not owning end to end , try to have some questions

  • If possible don't go blank when you a chance to ask something I mostly go with generic what is the role about and try to align my current exp saying oh this thing I have worked in past in this way , this works on my experience but if you are junior just try to show your enthusiasm and don't be just silent listener when they speak and most importantly be clear and refine your communication for this round as soft skill has most weightage in this round


Resources used

I am a reading heavy person

Books - DDIA , Data warehouse toolkit , Data engineering design patterns, deciphering data architectures None of them cover to cover just bits and pieces

Blogs - Uber, slack , doordash, aws etc engineering blogs

Youtube - Manish Kumar,Afaque ahmad, Love Babbar

DSA - Leetcode (Neetcode 150, Blind 75), Neetcode, Algo monster

SQL - Datalemur, leetcode , stratascratch


There might be some minor mistakes above as I didnt use AI to format


r/dataengineersindia • • May 01 '25

General Interview Experience - Best Buy | Walmart | Amex | Astronomer | 7-Eleven | McAfee

193 Upvotes

Hi,

My Info -

CCTC - 17LPA

YOE - 4 YOE

This is in order of interviews given.

  1. Best Buy - Selected

Offer - 31.5LPA (28.6Base Rest Variable)

  • Recruiter Reached Out.

1 Round -

(Fitment and Behavioral ) (Before Christmas)

With US manager, extremely Nice fellow, explained about himself, Role and asked for my introduction. Asked Behavioral questions about solving a time when I solved a hard problem, Helped teammates/colleagues out. Some simple technical questions on ETL/ELT.

2nd Round

(Technical F2F in their Office in BLR) (after 3 weeks)

2 Managers were there - Started with a DSA problem, you were given a laptop and you've to code it there itself and interviewees can see you type it was on Hacker rank platform. Never saw that question before.

Pretty simple Hashmap (dictionary question) don't remember it. Solved it and it passed all 15/15 test cases in single run.

Then given a SQL question to find the user with most amount of transaction from their sign-up to a decade from sign-up.

Interviewer asked me to just explain it as they had only a limited time for coding. They seemed very happy and told me I'm the one only solving both questions today.

Then they started with lot of questions around DE, Data Quality, Data Security, BigQuery and Google Cloud (had mentioned in resume), Data Modelling.

All were open ended questions and invited discussions with the managers. I loved it.

Main questions were like - Batch vs Streaming for some use case.

How would you design a Data Pipelines for dashboard.

Questions around BigQuery Architecture, internals and optimisations.

How will you secure PII data.

Round was for 1 hour went for 1.5 Hour. I asked them for feedback as it was my first F2F interview. They were happy.

HR came and told me I'm selected.

3 Round - (Same day as F2F) - Discussion about role, and numbers. Got offer after a week.

2. Astronomer - Reject

CTC discussed - Ballpark 33LPA Fixed + ESOPS

Mainly interviews were around Airflow and Python

R1 - Technical round (Easy)

Asked to Solve some random question for SQL/Python/ and an airflow DAG.

R2 - Hiring Manager ( Easy - Medium)

Asked questions on frequent switches, explained the role, asked tricky questions on airflow around backfilling, Scheduled time, etc. discussed on my compensation.

R3 - Technical ( Medium)

Revolved entirely around airflow, architecture, use cases.

My current project and using airflow, how does airflow work, it's components.

Lots of questions on Scheduler, parsing of DAGs, Executors (which one to use in which use case), Workers, Operators, Hooks, Deferred Operators, Dataset Triggered DAGs.

Little bit on Spark - How to manage overheadheapmemory error. RDDs and their implementation.

R3 - Technical (Easy - Medium)

Interviewer was a lovely person.

Questions around Airflow implementation and how will I achieve a specific use case like Parallelism in Airflow, How to manage concurrency of DAG, Handling Issues in Airflow, Notifications when issues happened, CI/CD with airflow.

Lovely interview felt like a discussion.

R4 - Technical (Hard) - Reject

Interviewer was nice introduced me about role, himself etc.

Asked me to implement a custom operator. I implemented one Custom operator class inherying the airflow base operator class but I felt my approach or my explanation wasn't at par to their expectations.

I wasn't able to answer few of his questions around DAG mechanics at low level and their implementations.

My gut feeling near the end of interview was a reject.

3. Walmart - Reject -

Apparantly they do drive Interviews on Zoom will assign you to a breakout room randomly. All interviews happened the same day

R1 - (Difficulty - Easy)

Questions on Project Spark Optimisation Techniques with lots of discussion on Spark Shuffle Partitions

2-3 Easy SQL questions on Deleting Duplicates, Window Functions

Python Coding questions - 2 Sum modification

R2 - (Difficulty - Easy)

Questions on Spark Joining two large tables and Aggregation (group by) scenarios and how to optimise it.

Discussion on Salting/Skewness

2-3 Easy SQL questions and asked me to code in Pyspark as well.

HM - (Difficulty - Easy)

Questions on Projects.

Asked me about Why am I switching so frequently?

Asked me Current Compensation and Expected Compensation?

Got stuck with Frequent switches and why am I looking for switched if I already have such "good" offer.

Didn't hear back after HM round, tried calling HR once. HR didn't pick up phone.

4. 7Eleven - Reject (Ghosted after collecting Documents)

R1 - (Difficulty - Easy)

Technical

Interviewer seemed like Junior DE.

Was asking all random questions, Wasn't sure on what to ask? Seemed lost.

2-3 Easy SQL questions

2 Python Questions (On finding Duplicates in List, Valid Parenthesis)

Rapid questions ranging from SCDs, Data Modelling, Normalisation, Spark Transformations, Optimisation Techniques, Spark Join Techniques.

R2 - (Difficulty - Easy)

Technical

Interviewer seemed Calm and composed unlike last interviewer.

Lots of Easy theoretical questions similar to last round.

Spark Scenario Question on Handling data which changed for past dates.

Implemented a SQL scenario using Merge/Insert. Seemed satisfied then wanted a Spark Solution.

2-3 SQL easy questions

2 Python Question ( Flattening a Nested Dictionary and returning Keys of Dictionary in list)

R3 - (Difficulty - Medium)

Managerial Round

1 Easy SQL question, didn't code he was happy with my approach.

How to debug a Spark Job that suddenly is taking way more time?

How will you go about code or logic fixing an urgent issue if you suddenly have to take an emergency leave.

Behavioral question on one difficult problem solved.

R4 F2F - HR/Fitment round in their Bengaluru Office.

Round was with HRBP -

Questions on why 7-11?

My current CTC and Last working date.

Expected CTC - Didn't seem too pleased after listening my number and my current offer. Was interested in knowing about the firm I hold offer from.

Got an email asking for documents. Didn't hear back. I didn't follow up.

P.S. - Got a call after 2 weeks, They'd like to move forward with 30LPA max, I rejected the same. Said, my CTC was high and they filled up the initial positions with people with less CTCband recently new ones opened up. Hence, contacted me for the newer ones.

5. Amex - Reject

Hiring was in a Drive both rounds happend on the same day. Recruiter reached out.

R1 - (Difficulty - Easy) Technical

Lots of questions on My Resume.

Easy SQL question on finding consecutive occuring numbers.

Easy questions on Pandas around Data Quality checks, finding Outliers.

Questions of Optimising Hive queries.

R2 - (Difficulty - Easy)

Technical Managerial

Easy questions on SQL and Python. Decorators

Finding Duplicates in the order they appear.

Interviewers seemed lost on what to ask.

Started asking about my frequent switches.

Current CTC and Expected CTC, didn't seem to pleased after listening my expectations and my current offer.

Didn't hear back. Didn't follow up.

6. McAfee - Data Platform Engineer - Selected

100% remote

Recruiter reached out.

CoderPad Assesment (Easy) -

Needed it to do it in 3 days

Almost 1 h 50 min were given to attempt. I did it in 1h 15m.

Got around 90% score. (You'll get results after couple of hours of giving the Assesment)

It had everything from Linux, Docker, Kubernetes, Python, SQL, Pandas, PySpark but it was easy.

R1 - HM round (Easy)

HM was nice, explained the role, asked about me and asked about the work I've done.

They've their infra on AWS so seem interested in AWS.

General Questions on Spark, Pipeline Management, Deployment, Errors and issues.

R2 - Panel Interview (Easy)

3 panelists were there.

Each asked questions one by one.

Questions were around Python, Python OOPs concepts, Inheritance, Constructor, Sets and Dictionaries implementation and how to order them, JSON library and parsing, Pandas simple questions, PySpark Optimisations.

Python Coding questions on Sets, Implemeting functions for separating Alphabets and Numbers, Sorting Dictionary by Keys and Values.

Questions on AWS services.

R3 - Python/Pandas/PySpark Hands-on (Easy-Medium)

To see your hands-on on the above technology.

They'll give you a dataset and ask you to code a lot of things to answer business questions like too 10 by years etc.

You've to do the entire thing in 45 mins. Time is really important.

Verdict - Got selected but I rejected the HR call citing I won't be joining to save both our times.

Calls from companies I got but rejected due to their Budget. If it helps anyone with negotiation.

Verizon - 22LPA

McKinsey - 25LPA

Paytm - 25LPA

EY - 22LPA

Axis Bank - 22LPA

UST Global - 27LPA

NTT Data (Hiring for Kotak Mahindra) - asked 35LPA and I dropped them after one round after understanding it's not directly for Kotak Mahindra Bank. They were ready to go even higher after I dropped them.

Arctic Wolf - 29LPA (their work was intresting)

Key Takeaways -

  1. If you know answers don't straight answer them take time, act like you're solving it for the first time. This will eat up interview time and save you from interviewer going blank awkward on what to ask, questions on Frequent Switches, CTC etc.

  2. Stay prepared, keep grinding, keep reading, good firms ask stuff which you can't prepare in a day or two or week .

  3. DSA will set you apart.

  4. Data Engineers are a second thought compared to SDEs, we're not paid on par with SDEs, also our interview bar is way lower than SDEs.


r/dataengineersindia • • Aug 18 '25

General 10-week data engineering interview plan (Google Calendar + CSV)—Blind 75 + SQL + Spark/Flink/AWS (IST timings)

180 Upvotes

Hey folks! I built a practical, day-by-day prep plan for my prep for Senior/Staff/Lead Data Engineering interviews and figured I’d share it in case it helps anyone preparing as well. It’s designed for full-time workers: realistic hours, steady progress, and DE-focused (not just DSA).
"Targeting": 90+ LPA Total Compensation by Jan 1st, 2026

Daily mix (balanced for DE interviews)

  • DSA: exactly 2 Blind-75 problems/day (NeetCode/Blind order; second pass from Sep 20).

  • SQL: one specific interview problem per day (e.g., Second Highest Salary, Gaps & Islands, 7-day rolling average).

  • Data Engineering Tools & Ecosystem (practice-first): Spark/Flink transformations (joins, maps, windows), Airflow DAGs, Polars, Kafka, S3/Glue/Athena/EMR, DynamoDB, Kinesis, Redshift, Hive/HDFS, NiFi, Cassandra/HBase, Kubernetes, Docker, Grafana, Prometheus, Jenkins, Lambda, plus dbt & Iceberg/Delta/Hudi.

  • System Design (concrete scenarios): Ride-sharing dispatch (Uber), Ticket booking, Parking lot, URL shortener, Chat system, Video streaming, Recommender pipeline, Data lakehouse, CI/CD pipeline, etc.

  • Rust hobby: 30–40 min daily (kept as a sanity/fun slot).


r/dataengineersindia • • Aug 18 '26

Wholesome Moment Closing the loop — I got the job! 🎉

179 Upvotes

Two months ago, I posted here after receiving my 12th interview rejection(https://www.reddit.com/r/dataengineersindia/comments/1u6nkca/need_motivation/)

At that point, I was honestly struggling. I had spent months studying, building projects, learning new technologies, preparing for interviews, and trying to understand where I was falling short. The hardest part was continuing to believe in the process when the results weren't showing.

I asked this community what kept you going during a long job search.

The responses, encouragement, advice, and personal experiences genuinely helped me. More than anything, they reminded me that rejection doesn't necessarily mean you're not improving — sometimes you just need to keep going until the right opportunity comes along.

Today, I'm very happy to come back and say:

I got a job! 🙌

The journey wasn't easy, and there were plenty of moments when I doubted myself. But I'm grateful I didn't give up.

A big thank you to everyone who commented, encouraged me, shared their experiences, or simply took the time to read my post. Your words meant more than you probably realized at the time.

For anyone currently going through rejection after rejection: keep learning, keep improving, keep applying, and don't let the number of rejections define your ability.

Sometimes one opportunity can look very different from all the applied/rejectedoppurtuinties.

Grateful for this community. ❤️

And finally, I'm happy to close the jobless chapter with a much better update.

Thank you, r/dataengineersindia**! 🚀**

Thank you


r/dataengineersindia • • Apr 19 '24

Meme! We gotta help others realise joining FAANG is the only success metric 🤡

Post image
170 Upvotes

r/dataengineersindia • • Sep 05 '26 Bravo

General Amazon Data Engineer L5 Interview Experience (Applied Directly, Full Loop) — Sharing for anyone prepping

166 Upvotes

Round 1: Online Assessment

Got the OA link by email shortly after applying. It had:

  • 2 easy SQL questions (joins, group by)

  • A set of SQL MCQs

  • A set of behavioral/situational MCQs

Passed this and got a call from the recruiter a few days later saying I'd move to a technical screen.

Round 2: Technical Phone Screen

No fixed agenda going in. The interviewer started by asking about my most recent project (dashboard-building work) and went genuinely deep — asked about OLAP vs OLTP and which is better suited for serving a frontend dashboard, and why.

Then a scenario question: how would I handle a data pipeline failure — steps to resolve it, communication during the incident, documentation afterward, basically the full incident-handling flow.

SQL portion started with a trivial-but-tricky one (NULL = NULL type gotchas), then three more questions — one involved a join plus window functions for ranking, plus an aggregation question. Also gave me a data processing problem on a sample table, which I solved in PySpark.

Closed with a couple of leadership-principle-based behavioral questions.

Recruiter called 2 days later to confirm I was moving to the onsite loop (5 rounds).

One note on timing: getting the loop actually scheduled took about 1.5 months due to some back-and-forth on both ends. Once it was locked in, the recruiter did a 30-minute prep call and gave a clear breakdown of each round's focus and which Leadership Principle it maps to, which was genuinely useful going in.

Round 3 (Loop Round 1): Performance Measurement

Focused on indexing, partitioning, and tuning. I walked through a relevant project, and the interviewer went deep with follow-ups — Spark internals, partitioning strategy, bucketing, how executors relate to data volume, join strategy selection, and data skew. Almost every answer led to another follow-up, so precision mattered a lot here — vague answers got picked apart pretty quickly. Closed with LP-based behavioral questions tied to this round's LP.

Round 4 (Loop Round 2): System Design, Data Modeling, and Trade-offs

Explained a project involving SCD Type 2 logic and surrogate key generation — went deep into different methods for generating surrogate keys and the trade-offs between them.

Then a live design exercise: design a data model for an e-commerce platform (Amazon-style) — had to reason through grain, facts vs. dimensions, and table structure, followed by 2 SQL questions on top of that model.

Then it extended into a full system design for the same platform. I proposed a Kappa architecture, explained the trade-offs versus Lambda, and built the design mostly around Spark and Databricks — Structured Streaming with foreachBatch for the real-time path, Redis for serving.

Honest note: the interviewer wasn't fully convinced by my design since I leaned on a fairly narrow tool set. Looking back, showing awareness of other real options (even while still picking Spark/Databricks) would've made the answer stronger — naming trade-offs against alternatives rather than just defending one stack.

Closed with LP-based questions specific to this round.

Round 5 (Loop Round 3): Coding, Problem Solving, DE Concepts, ETL/Architecture Design

Started with about 10 minutes on my project. Then a data modeling exercise: design a data model for clickstream data tracking promotions — how to store users and events.

I initially designed a normalized star schema. The interviewer was actually expecting a denormalized table for this specific use case, so I adjusted once I picked up on that. Followed by SQL queries on top of the model.

Closed with 3-4 SQL questions — window functions, CTEs, and a rolling average question.

Round 6 (Loop Round 4): Bar Raiser

This round is a big deal in Amazon's process and worth specifically prepping for. Entirely behavioral — "tell me about a time you made a mistake in a data pipeline," "tell me about a data quality issue and how you resolved it," that style of question, all mapped back to the round's specific LP.

One thing worth knowing going in: Amazon generally brings in a Bar Raiser from outside the team you're actually interviewing for — mine was an SDE, not a data engineer. So I had to explain data engineering concepts in a way that would land clearly with someone outside the DE world, not assume shared context. From what I understand, being able to simplify your explanation while still weaving in the LP clearly is actually one of the core things Amazon is evaluating in this round.

Round 7 (Loop Round 5): Hiring Manager

Walked through my project again with follow-up questions, plus behavioral questions similar in style to the Bar Raiser round (failure handling, mistakes made) — also tied to LPs.

One interesting one: asked what metrics I'd present to senior leadership if I were looking at a platform like Netflix — a more business-acumen-flavored question mixed into the usual technical/behavioral format.

Overall takeaways for anyone prepping:

  • Every technical answer can and will sprout follow-up questions. Don't give an answer you can't go two levels deeper on.

  • Know your own projects well enough to defend every design decision, including ones you'd make differently now.

  • For system design rounds, show awareness of multiple tools and approaches even if you ultimately pick one. Defending a narrow toolset without acknowledging alternatives can come across as limited breadth rather than conviction.

  • For Bar Raiser specifically, practice explaining your work simply enough for someone outside your discipline to follow, while still tying it back to the LP clearly. That combination seems to be exactly what they're testing for.

  • LP stories need to map cleanly to whichever LP each round is testing. Know which ones you're using where, and try not to reuse the same story across rounds if you can help it.


r/dataengineersindia • • Oct 11 '26 Bravo

General I've mentored a lot of people for DE interviews in India. Here's a free list of what keeps getting asked

161 Upvotes

Some background so you know where this comes from. I'm Durgesh Yadav a Senior Data Engineer in Bangalore Ex- Target, I've given 20+ DA/DE interviews myself, and I've been mentoring people for these interviews for about four years.

When mentees come back from their loops, the same questions keep showing up. So here they are, with what the interviewer is really checking. No link, no course pitch.

SQL round

Top 3 salaries per department. They want to see if you know ROW_NUMBER vs RANK vs DENSE_RANK and what happens with ties.

Users who logged in 3 days in a row. Gaps and islands: date minus ROW_NUMBER stays the same across a streak, so you group by it.

Keep only the latest record per customer. ROW_NUMBER partitioned by customer_id, ordered by updated_at desc, keep 1.

Why does this NOT IN return zero rows? There's a NULL in the subquery. Use NOT EXISTS.

PySpark and Spark

repartition vs coalesce. repartition does a full shuffle and can increase or decrease partitions. coalesce only reduces them and avoids the full shuffle.

Which transformations cause a shuffle? groupBy, join, distinct, orderBy. The real question is narrow vs wide.

One join key holds most of the data and the job is stuck at 199 of 200 tasks. Broadcast the small side if you can, otherwise salt the key, and mention AQE skew handling.

The job took 20 minutes yesterday and 2 hours today. Walk through the Spark UI: input size, skewed tasks, spill, a stage that suddenly shuffles more.

Data modeling

Star vs snowflake, fact vs dimension. State the grain first, every time.

SCD Type 2 with MERGE in Delta. Expire the old row, insert the new one with start date, end date and an is_current flag.

Design the tables for a food delivery app. Orders, order items, restaurants, riders, customers, and what the grain of each fact table is.

Pipelines and Databricks

A daily load failed halfway. How do you rerun it without duplicates? Idempotency: overwrite the partition or MERGE on a key.

Yesterday's data arrives 2 days late. How does your pipeline handle it?

Incremental loads: watermark column vs CDC, and what breaks when the source updates old rows.

Bronze, silver, gold: what actually goes into each layer in your project. Then OPTIMIZE and VACUUM, and what VACUUM does to time travel.

Project round

"Explain your project end to end." Answer in numbers and decisions: how much data, how often it runs, what failed in production and what you changed. A list of tools is not an answer.

My one tip: before you write a single line, say the grain, the data volume and the edge cases out loud.

If you've interviewed recently, drop the question that surprised you and I'll add it to the list. I also have notes on most of these with solved answers, happy to share them free if it helps.


r/dataengineersindia • • Mar 04 '26 Bravo

General Amazon Data Engineer II (L5) Interview Experience

159 Upvotes

Hi everyone, I recently cleared the loop for Amazon DE 2 role

My exp - 5yrs

Here's my interview experience

OA Round - Check my other post on this sub. The recruiter reached out to me after a month.

Each round was 1hr each, you can ask around 2 qs each round in the end.

Round 1 - Data Modelling

Retail data model for yearly/monthly sales per product, vendor & location. Then SQL queries on top of the data model you created.

Round 2 - ETL

Discussion about project. Streaming v/s Batch use cases, Optimizations on 100GB daily load & 1 Billion rows table.

Round 3 - SQL and Scripting

1 DSA medium question

Given list1 = [(1,"a"), (2,"b"), (4,"c")] and list2 = [(2,"e"), (4,"g"), (7,"h")]*, find all common keys and pair their values. Expected output:* [(2, ("b","e")), (4, ("c","g"))]

, 2 Medium-Hard SQL questions based on joins, window functions, ranking etc

Round 4 - Performance Optimizations

Deep dive on spark optimizations, data skew, checkpoints, partitioning and indexing for OLTP writes and analytics queries. 1 sql query optimization qs

Round 5 - Bar Raiser

Questions based on Amazon leadership principles.

Each of the other rounds also had 2 leadership principles questions. So prepare the stories well on these. Follow STAR method to answer the questions. Expect them to dive deep into the stories - timelines, learnings, what would you have done differently etc..

Hope it helps!

[Update] - TC range

Base - 35 - 45 LPA

Joining Bonus - 1st year 10L - 20L , 2nd year similar but will be less than first

Stocks - 30L - 40L vested over 4 yrs [ 5%, 15%, 40%, 40% ]


r/dataengineersindia • • Feb 18 '26

General Interview at JPMC

148 Upvotes

I was recently interviewed at JPMC for Data Engineer II role. I have 3+ years of experience. These are the questions asked -

  1. How do you add a docker image to EMR cluster?

  2. We have to process 50 CSV files and jobs may get failed after (let's 20 files), don't want to run from already processed files, how do handle this?

  3. Airflow Operators used in my project.

  4. An airflow task runs every 15 mins and some times it may take longer than 15 mins to complete, how do you handle this?

  5. String reversal, valid email id, missing number in a list.

  6. Difference between and copy and deep copy

  7. How do you decide whether to run a job on EC2 or Serverless? explain in detail

  8. Spark Architecture

  9. What is DAG? Difference between DAG and Lineage

  10. Explain how spark is fault tolerant?

  11. Given a user_df and events_df, find the active users who have logged in in last 30 days for 3 consecutive days.

  12. Various scenarios of joins, when to use what

  13. Production issues encountered in my current job and how did I debug and resolved.

Edit: I didn't apply for this anywhere, I was lucky enough that an HR from JPMC reached out to me over LinkedIn as their JD matched my profile.

Update: Not selected


r/dataengineersindia • • Aug 05 '25

General Giving back to the community

146 Upvotes

Hi All,

I am Data Engineer , currently working one of the MAANG companies, totalling experience of 6+ years. Previously worked in Amazon and other PBCs where i build tools and data warehouse from scratch.

Recently, I have seen many people started taking interest in Data. I have seen a lot of questions regarding career. I have helped few in DMs but it can't be scaled to a point that I can help the whole community.

So, in short, I will be start writing about interview experiences, career guidance, work culture, About work in PBCs and other things coming my way.

Please throw your questions in comments, I will pick most asked question and will try to post atleast twice or thrice a week.

Share the post as much as possible so it can be echoed to whole community

P.S - I have seen a lot of AI post. So wanted to mention that I won't be creating any via AI as it lose the sense of personal experience.


r/dataengineersindia • • Jun 03 '26

Wholesome Moment Finally joined Publicis Sapient

142 Upvotes

Hey all,

As you may know from previous posts. I was serving notice and looking for offers.

My current CTC: 12LPA Infosys

Tech stack: Python, SQL, Snowflake, Azure, GCP, Dragster, Dbt Yoe 4.6 years Notice served: 90 days

First offer: Deloitte Consultant: 17.8+10%+ 2L joining Bonus 2 consecutive years + perks like relocation assistance and well-being subsidy. The location was hyderabad.

Dropped paper after getting Deloitte offer.

Second Offer: I used the Deloitte offer to get 19+10% from PWC.

Third offer: IBM lowballed with 16LPA, rejected.

Fourth Offer: I used the Deloitte offer to get 20 from TCS.

Fourth Offer: I used PWC to get 23.8 fixed from Publicis Sapient.

Other companies Interviewed in: Acuity Analytics, GSPANN, EXL, Impetus, Tiger Analytics, IBM

Feel free to ask any questions if you may have.

Interview questions:

https://www.reddit.com/r/dataengineersindia/s/Ouw19FDOlz https://www.reddit.com/r/dataengineersindia/s/P081nhSRGV https://www.reddit.com/r/dataengineersindia/s/zuJbHVT4wC https://www.reddit.com/r/dataengineersindia/s/Fei0FhpPlK


r/dataengineersindia • • Aug 27 '26

Wholesome Moment Finally got the offer. Here’s what actually helped me prepare.

141 Upvotes

After months of learning, applying, interviewing, getting rejected, and starting again, I finally got the offer I was hoping for, Senior Data Engineer at Good Product Based Company.

I had around 5 years of experience, but my journey into data engineering wasn't straightforward. I worked across different technologies and roles before realizing data engineering was where I genuinely wanted to build my career.

I was laid off in July when a client engagement ended. Instead of rushing into the next opportunity, I decided to prepare properly.

Here’s what worked for me:

1. Prepare scenarios, not just questions.

I grouped my preparation into ingestion, transformation, storage, orchestration, data quality, performance, scalability, and production issues. This helped me handle questions I hadn't seen before.

2. Understand the "why".

Whenever I was weak in a concept, I focused on understanding why it exists, its trade-offs, and how I would use it in a real system. I also wrote code instead of only reading.

3. Follow the job description.

I compared what the role actually required with what I knew and focused my learning around those gaps.

4. Use your own experience.

For behavioral and scenario questions, I used real situations from my projects instead of memorizing generic answers.

5. Learn from every interview.

After each interview, I wrote down what was asked, where I struggled, and what I needed to improve. Each interview became preparation for the next one.

6. Use AI as a learning partner.

I used AI to explain concepts, challenge my answers, simulate interviews, and help me think through scenarios. I also relied on YouTube, documentation, Google, and Reddit.

One small change that helped me more than I expected: I uninstalled Instagram on May 10th. It gave me back a surprising amount of time and mental space. It may not be necessary for everyone, but for me, it was a real game changer.

The biggest lesson for me?

Don't prepare just to clear an interview. Prepare to become someone who can actually do the job.

There were plenty of days when nothing seemed to be happening. But I kept learning, kept preparing, and kept showing up.

I'm grateful to everyone who supported me along the way, directly or indirectly.

If you're going through a difficult phase in your own journey, stay hopeful, keep a positive mindset, and keep moving forward. Sometimes the work is happening long before the result becomes visible.

Keep going.


r/dataengineersindia • • May 17 '26 Bravo

General KPMG Data Engineer Interview Experience (Face-to-Face | Azure + Databricks + PySpark)

141 Upvotes

Recently attended a face-to-face interview for a Data Engineering role at KPMG. Sharing the questions asked:

  1. Short introduction

  2. From a Python list with repeating items, get only the first occurrence.

  3. Extract even numbers from a nested JSON.

  4. How do we read Excel files in Databricks?

  5. If CSV data has extra commas inside values, how do we handle it?

  6. What is the most challenging task you have done?

  7. Write SCD Type 2 logic in Spark.

  8. Explain your day-to-day work.

  9. How do you write tests in CI/CD pipelines?

  10. What is flake8? I mentioned we use black formatter, then they asked why black is used.

  11. Explain CI/CD pipeline architecture. Why are pipelines written in YAML?

  12. What is git stash?

  13. Given an email column, extract the first name and create a new column.

  14. Difference between Tables and Volumes in Databricks.

  15. Difference between Serverless and Dedicated SQL Pools.

  16. What are the libraries you have used most?

  17. Difference between External and Managed tables. Where is the data stored in both?

Interview was mostly practical and project-oriented with focus on PySpark, Databricks, CI/CD, and real-world scenarios.


r/dataengineersindia • • Feb 20 '26

Rant! Rejected After 4 Rounds(Warner Bros) - Interview Experience

139 Upvotes

Hey everyone, sharing my recent interview experience for a Data Engineering role.

Round 1 (CoderPad):

Coding on Binary Search and Heap. Mostly standard problems but focus was on edge cases and time complexity.

Round 2 (Senior Manager):

Project deep dive + real-time streaming pipeline design. Lots of tradeoff discussions, architecture thinking, plus some behavioral questions.

Round 3 (Senior VP):

Quite intense — GenAI basics, AWS stack, concurrency, system design, and unexpectedly a deep dive into Kafka/Kinesis even though it wasn’t on my resume.

Round 4 (Senior Director):

Detailed discussion on projects, architecture decisions, and technologies I’ve actually implemented.

Verdict: Rejected.

My honest takeaway: I haven’t worked much on real-time streaming pipelines, and they pushed hard on Kafka/Kinesis internals despite it not being on my resume. Conceptual answers weren’t enough.

Key learnings:

- DSA still matters for DE roles.

- Be ready for adjacent tech even if not on resume.

- Real hands-on experience beats theoretical knowledge.

- Senior rounds go very deep into architecture.

Now planning to build some streaming projects and strengthen system design.

Any good resources or advice would be appreciated 🙏


r/dataengineersindia • • May 14 '25

General Finally got the offer

Post image
138 Upvotes

Finally got the offer after almost 4 weeks. Just wanted to say thanks to everyone who provided info. Had to reject one offer I was already holding, that HR was angry and threatened to not consider me in whichever organisation he works even in future. I feel a little guilty as it was my first time switching companies but I had to what was best for my career. I am told it's something that is not very uncommon just wanted to see what other people say.


r/dataengineersindia • • Jul 09 '26

General For folks who think AI is going to replace Data engineers !!

Post image
136 Upvotes

This post is from someone in openai ...welcome to uno reverse 😃


r/dataengineersindia • • Apr 06 '26 Heartwarming

General Amazon Data Engineer II interview experience

134 Upvotes

Hey everyone,

I recently went through the Amazon Data Engineer interview process (Hyderabad) and wanted to share my experience. I didn’t find many detailed India-specific posts while preparing, so hopefully this helps someone.

---

Round 1 – Coding

The first round was a coding round with basic problems. Think LeetCode easy to medium level. Mostly arrays and hashmap-based questions (similar to 2-sum type problems). It wasn’t heavy DSA—no trees or graphs in my case. The intent seemed to be to check whether you can code comfortably.

---

Rounds 2 & 3 – Architecture + Past Experience

These rounds focused heavily on my previous work. There was a deep dive into the pipelines and systems I had built. Questions were around:

End-to-end design of data pipelines

Challenges faced and how I handled them

Trade-offs in design decisions

A lot of follow-ups came from whatever I mentioned, so being clear about your own work is important. My Mastercard experience carried most of these discussions.

---

Round 4 – Deep Data Engineering Concepts

This was the most intense technical round. The focus was on core data engineering concepts like:

Query optimization

Partitioning strategies

Index vs query rewrite vs data model changes

They were not just looking for answers but for reasoning. For almost every answer, there were follow-up questions asking why I chose that approach over others.

---

Round 5 – Data Modeling

This round was entirely focused on data modeling. I was asked to design a vendor payment system from scratch.

We discussed:

Fact vs dimension tables

Business events and how they map to tables

Granularity decisions

Payment batch vs order-level tracking

Schema design (star vs snowflake)

How to store payment-related information (normalization and security considerations)

SCD types were also asked, so that’s something you should definitely prepare.

---

Round 6 – Bar Raiser (Behavioral)

This was a pure behavioral round based on Amazon Leadership Principles. The questions were detailed and required structured answers with clear examples. This round felt quite important in terms of final decision-making.

---

Overall Takeaways

The process is not focused on heavy DSA. Coding is required, but at a basic level. The main focus areas are:

SQL and data handling

Data modeling

System design for data problems

Real-world experience and decision-making

---

Preparation Advice

Be very strong in SQL (joins, aggregations, window functions)

Know your projects inside out—they will go deep

Be prepared to explain trade-offs, not just solutions

Practice behavioral answers aligned with Amazon leadership principles

---

Overall, the process felt practical and aligned with real data engineering work rather than just theoretical questions.

Happy to answer questions if anyone is preparing.