r/datascience • u/avourakis • 16d ago
Tools Interactive tools for learning Agentic Analytics and evals for free
I keep running into data scientists who are trying to understand conversational analytics, like talk-to-your-data Slackbots, because they either want to build one for their company or understand how this could change the field over the next few years. What's usually missing is a clear picture of how the whole thing fits together, beyond text-to-SQL or semantic layers.
I've spent the last 2 years building these systems in production. I built Nextory's first talk-to-your-data Slackbot, and I ran into a lot of these challenges along the way.
And even after going through books like Chip Huyen's "AI Engineering" or "LLM Engineer's Handbook", I found that reading about these systems only gets you so far. You get the intuition from seeing how the components actually connect and behave.
So I built two free interactive tools: one walks you through how an agentic analytics system works from the inside, and the other focuses on AI evals.
Here they are: https://futureproofds.com/tools#agentic-analytics
In each tool you can:
Walk through the entire system and dig into how every component works
Study the key concepts with quick summaries
Take a free quiz to test your understanding
Would love to hear what you think (I’m still doing a lot of testing with some close DS colleagues)
If you want to save them for later, all of my free tools are in this GitHub repo: https://github.com/Future-Proof-DS/interactive-ai-ml-tools
1
u/shikokuchuo 14d ago
Have you thought about showing how to apply the theory to actual tools - evals frameworks like inspect-ai for instance?
1
u/avourakis 14d ago
Not really, how do you envision that to look like?
1
u/shikokuchuo 12d ago
Like, now you know how this works, this is what you'd actually do as an engineer / data scientist and you'd show each practical step corresponding to the theoretical step. I think you could even just use one popular framework without seeming to be biased, as the concepts are transferrable. May not be where you want to go with this, just an observation.
2
u/Charlotte1309 13d ago
I really liked it.
I'm not sure there is enough theory walkthrough though before taking the test (I did good because I'm in the field, but for a beginner it would have been difficult I think)