How do candidates stand out in a data science case study interviews? From my experience giving well over a hundred mock interviews, mostly for senior-level FAANG roles, the best candidates can calmly justify their choices and don't get flustered under questioning.

Some red flag examples that happen **all the time** in mocks, again even among senior candidates with experience and advanced degrees where more justification could help tremendously: (i) memorizing that marketplace experiments have interference, but then being unable to walk through whether a geo-based XP or switchback is more appropriate, (ii) defaulting to an alpha of 0.05 without recognizing it's a choice parameter in your XP design or recognizing context where you can be more or less conservative, and (iii) using 'everyday' guardrail metrics such as app latency or crashes but missing unintended consequences more directly tied to the treatment. IMHO, pretty much any product feature launch could have latency as a guardrail, but that's more of an engineering issue, and expected guardrails in interviews are usually tied more closely to the treatment going wrong, causing winners and losers, or short-term revenue gains vs long-term losses from quality degradation.

Enjoy them, learn from them...but mainly be frustrated by how fast you need to answer 🙂

You can take on the challenge here

Around this time last year, I did an AMA and thought it was a pretty good experience. I wanted to do another one. If folks have any questions, maybe we can have some discussion together : )

I recently ran a benchmark comparing tabular and time-series foundation models on German day-ahead electricity price forecasting.

The models I evaluated were:

• Tabular foundation models: Kumo and Nori

• Time-series foundation models: Chronos-2 and TimesFM-3

• Traditional ML baselines: Tuned LightGBM and CatBoost

The foundation models were used without training on the electricity market data. I also made sure the models had access to comparable information during evaluation.

The article covers the experimental setup, how to use tabular foundation models for forecasting without data leakage, the comparison against tuned tree-based models, and the impact of additional training data on the traditional ML baselines.

Full discussion and tutorial about the experiment here

(I’m Hadley Wickham, Chief Scientist at Posit and I’m posting this for my colleague Charlie Gao (u/shikokuchuo) who doesn't have enough karma to post here 🤣).

Resampling (bootstrap, cross-validation, ...) is a common part of modelling that takes a long time and in principle should be trivially parallelisable. Unfortunately, in reality the speedup is usually well short of the allocated cores. The reason is how the process pool has to distribute work: each task is serialized, copied through a socket then deserialized on the other side, and each result has to come back the same way. This has two problems: when tasks are small the overhead can exceed the computation, and when the data is large every worker holding a copy exhausts memory.

Charlie has come up with a new solution using shared memory. pymizu (Python) and mizu (R) use shared memory to parallelise work. The data for each task is written once into memory that the scheduler and worker share, so the worker reads it without copying. Compared to using sockets (the current fastest approach) a round trip drops from about 100 µs to under 1 µs (measured on an Apple M4 Pro). And this makes a difference for data science workflows: four typical tidymodels workloads  chosen to stress data movement relative to compute, 12 mizu workers gave 6.7–8.1x speedups versus 1.4–6.4x for mirai, already the most performant parallel framework in R. 

In Python, pymizu is a drop-in replace for theconcurrent.futuresexecutor, and NumPy arrays and polars/pyarrow/pandas frames cross between processes without any copies. pymizu and mizu are designed to work together for cross-language workflows. Each can launch workers in either language, so a data frame in an R session can cross to a Python worker as an Arrow-backed frame, and the polars result comes back as an R data frame.

More details, benchmarks, and examples are available at: https://opensource.posit.co/blog/2026-11-05_mizu-0-1-0/ or get started with
pip install pymizu / install.packages("mizu").

I'm a senior data scientist with about a decade working in the industry. Lately I had been helping college grads with job search but found myself lacking knowledge of the entry-level hiring practices, specifically for data analyst roles.

Are ATS checker a must now before submitting resume? Is it true that if one doesn't apply immediately after a job is posted, say within 2 hours, their resume will no longer be reviewed?

If it's true that these AI-curated resumes are flooding the market, how does one correctly signal to the company that he or she has the appropriate background for the role?

Lastly, would you consider the following a worthwhile project to pursue, if not, what would be an example of project that gets attention?

Building a dashboard to answer a business question (tracking price changes of BTS tickets, for example), with a sqlite database as backend. In addition, a process to periodically scrap/API call, transform data and refreshes the database, and a process to send an email should certain criteria be met (price is below $x, for example).

Thanks in advance.

I keep running into data scientists who are trying to understand conversational analytics, like talk-to-your-data Slackbots, because they either want to build one for their company or understand how this could change the field over the next few years. What's usually missing is a clear picture of how the whole thing fits together, beyond text-to-SQL or semantic layers.

I've spent the last 2 years building these systems in production. I built Nextory's first talk-to-your-data Slackbot, and I ran into a lot of these challenges along the way.

And even after going through books like Chip Huyen's "AI Engineering" or "LLM Engineer's Handbook", I found that reading about these systems only gets you so far. You get the intuition from seeing how the components actually connect and behave.

So I built two free interactive tools: one walks you through how an agentic analytics system works from the inside, and the other focuses on AI evals.

Here they are: https://futureproofds.com/tools#agentic-analytics

In each tool you can:

1. Walk through the entire system and dig into how every component works

2. Study the key concepts with quick summaries

3. Take a free quiz to test your understanding

Would love to hear what you think (I’m still doing a lot of testing with some close DS colleagues)

If you want to save them for later, all of my free tools are in this GitHub repo: https://github.com/Future-Proof-DS/interactive-ai-ml-tools

Over the past month, I’ve interviewed with two companies that didn’t have any traditional coding rounds. Instead, their technical interviews were entirely case study or scenario based. Another company I interviewed with a couple of months ago had only a fairly basic Pandas round.
These were all senior-level roles at hedge funds or investment management firms.
I’m wondering whether this is specific to the industry or if companies are starting to recognize that Leetcode style coding rounds aren’t necessarily the best way to evaluate a candidate’s abilities, given heavy dependence on AI for coding at work.

Welcome to this week's entering & transitioning thread! This thread is for any questions about getting started, studying, or transitioning into the data science field. Topics include:

• Learning resources (e.g. books, tutorials, videos)

• Traditional education (e.g. schools, degrees, electives)

• Alternative education (e.g. online courses, bootcamps)

• Job search questions (e.g. resumes, applying, career prospects)

• Elementary questions (e.g. where to start, what next)

While you wait for answers from the community, check out the FAQ and Resources pages on our wiki. You can also search for answers in past weekly threads.

There's a lot of "conceptual conflation" going around lately, especially regarding the term 'Harness Engineering'.

This article formalizes harness engineering with a clear conceptual model. This definition has also been echoed by other top data scientists and AI engineers, thus we can start to collectively agree on a definition.

Secondly, I introduce 3 primitives that underpin harness engineering. I wanted to clearly separate this from a basic agent tutorial and focused more on the underlying concepts that make great harnesses.

There's a lot of (not so good) harnesses out there. And, you probably shouldn't be staking your entire future on top of claude code.

My strong opinion: if you're not ultra-focused on building harnesses today as a practicing data scientist, you're a target for layoffs. You need to build your own harnesses.

The article: https://towardsdatascience.com/what-is-harness-engineering-a-hands-on-introduction/

What do you think?

I have been giving interviews over the last few months and I have categorized most of the hits and misses into various buckets I need to work on.

One of them is scenario based "why and why not" style questions? Why would a model work, why not so and so method etc. My problem is I am not really good at this - I can explain the decisions I made in my projects, but if the interviewer tweaks something I will likely not be able to answer this.

I have also been thinking in gaming terms (I am an avid gamer) and a passionate DoTA and R6 Siege player. I have, with a little shame, put ~3K hours each in both. But that grants me a vast knowledge (not an expert) where I can play with scenarios and theorycraft stuff and matchmaking keeps pitching me new challenges.

Akin to this, how do I get this level? Working on projects isn't helping a lot, as I am sticking to stuff I know. Since I don't know what I don't know, it's really hard to grow.

I’m a junior data scientist and I have 1 yoe at a medium sized company. Recently my job has made me so anxious I cannot even sleep. Basically, I deployed a model for the first time, and I cited great improvements, but due to holes in my QA, the QA person did not find that the model actually improved that much in production.

Im already anxious enough, but my manager made me even more anxious. He got pretty pissed that this happened.. so he proceeded to publicly shamed me in standup, telling my team of 6 that they should learn from this lesson. In our 1-1, he told me that if our director sees this, it’ll be bad for my reputation, especially if our clients complain about this.

So I asked him if I could get a second pair of eyes on my work. This is because barely anyone helped me with this project. Only one other person helps me with deployment, but they don’t know anything about how this model works. My manager agreed, but he also said that even if the second person looks over my work, they wouldn’t really know what’s happening either. So it’s all up to me to catch stuff like this. And we ended the meeting with no concrete solution.

I get that it’s important to not make mistakes in production, but is it normal to be this tough on a junior? I literally feel so shit right now. I just feel so incapable and I can’t believe I didn’t catch it the first time. I’m literally considering quitting this field if I have to deal with this for another 40 years….

Edit: he also said "i thought you would have a good handle of this by yourself already, but I guess not" 💀

Edit2: thank you all for your reassurance. I have set up a meeting with the QA person to go over stuff before I deploy again. Hopefully I can learn something from them. Wish me luck for the next deployment 🤞

Over the past several months, our research team pre-trained and empirically audited a 1,189,135,740-parameter Financial Conformer Foundation Model across 14 calendar years (2010–2023; 15,030 steps) of 1-minute US equity bars across 15 liquid assets (AAPL, NVDA, SPY, QQQ, etc.).

Instead of cherry-picking positive backtests, we subjected the model to 35 hypothesis tests, 1-bar execution delay, and state-dependent turnover:

1. The Scale Saturation Limit: Scaling from a 63-parameter linear Ridge regression to 1.189B parameters yields no consistent out-of-sample advantage on 1m OHLCV bars (mixed sign across horizons, reversing direction across years). The bottleneck in intraday price prediction is information content (low SNR), not parameter capacity.

2. Adam Optimizer Diffusion: Decomposing walk-forward online fine-tuning shows tokenizer boundary updates sit at 97.9% of the theoretical Adam Brownian noise ceiling (matching shuffled-label controls), while deep backbone weights are heavily damped by second-moment normalization, producing zero statistically significant forward-transfer gains (p >= 0.50).

3. Execution Friction Reality: Apparent long-horizon directional predictability is predominantly prompt 1-minute order-flow continuation. Introducing a 1-bar execution delay cuts 2024 net Sharpe from +2.04 to +0.48, and at 1.5 bp execution fee, out-of-sample returns collapse to -0.99 in 2025 and -1.67 in 2026 (DSR <= 0.028).

All model weights, audit logs, and papers are open-sourced:

• Model Hub (weights & checkpoints): https://huggingface.co/Ultron28/ohlcv-transformer

• Code & Telemetry: https://github.com/AirBorneAI/ohlcv-transformer

• CERN Zenodo DOI: https://doi.org/10.5281/zenodo.23466965

Happy to discuss architecture specifications, pre-training dynamics, or empirical backtest methodology!

I'm experimenting with agents for data analysis, but I find myself struggling to work with more than one at a time. I'm interested in how other data scientists handle the human review part.

Thinking of a recent analysis, what did you let the agent do, and what did you check yourself before trusting the result? A concrete example of a check that caught a problem, or convinced you the output was usable, would be helpful.

If you've found a repeatable workflow, I'd also like to hear how you split up the work and whether the review effort got smaller over time.

Welcome to this week's entering & transitioning thread! This thread is for any questions about getting started, studying, or transitioning into the data science field. Topics include:

• Learning resources (e.g. books, tutorials, videos)

• Traditional education (e.g. schools, degrees, electives)

• Alternative education (e.g. online courses, bootcamps)

• Job search questions (e.g. resumes, applying, career prospects)

• Elementary questions (e.g. where to start, what next)

While you wait for answers from the community, check out the FAQ and Resources pages on our wiki. You can also search for answers in past weekly threads.

In marketplace experiments, the most actionable metrics are often the ones with more variance, making them harder to power. This tradeoff is particularly important when using geo-based experiments where you can't necessarily or cheaply add sample.

Suppose Airbnb wants to test highlighting cheaper listings; this should increase bookings while likely reducing AOV. The net effect of a higher conversion rate and reduced AOV depends on elasticities and profit margins. Revenue or contribution margin are the metrics that matter the most (should be the most actionable), but these have far more variance than simple proportions like a conversion rate.

Running the geoX longer helps less than you might expect if you haven't done these before: weekly bookings within the same region are almost certainly correlated so you'd need to cluster the SEs, which can brings the effective sample size "roughly" down to the number of unique geos (if the correlation within geos is high).

So you have a few options: (1) focus on the actionable metric but accept more uncertainty through a higher alpha, ideally combined with (2) variance reduction techniques like CUPED, or (3) power the test on a lower-variance metric like conversion rate and try to bound the AOV loss using product sense and reasonable assumptions.

This comes up frequently on the job and also in senior and staff data science interviews. Real-world business problems can be complex, and there's not always a single correct answer. Trying to memorize answers and metrics can help in junior interviews, but at senior you typically will get probed on teh tradeoffs. .

Is there an official definition of an "AI predictive model"?

Can any model, including a linear regression, be considered artificial intelligence if it's using historical data to make predictions?

Or is AI restricted to neural networks that simulate human speech or generate images, music, or scrape the Internet for information? Like ChatGPT or Claude.

My company is setting up guardrails for AI use and there isn't broad agreement how to define "AI".