r/ETL • u/an_tonova • Jun 30 '26
I've used 6 million claude code tokens in 3 months
I burned through 6 million tokens on Claude Code last quarter. It started as a productivity boost, I was shipping faster than ever, crushing dbt models and ingestion scripts in hours instead of days.
Then Claude went down for about half an hour during a pretty routine ETL fix. I just sat there staring at the terminal. Couldn't remember how to structure a simple window function without it suggesting the whole thing. That's when it hit me: I've outsourced my working memory to an autocomplete tool.
Coding assistants are great for finishing your sentences, but they don't manage the full pipeline. They don't understand your schema drift, your monitoring, or your deployment steps. You still have to hold all that context yourself and stitch the pieces together. That's the real bottleneck, not writing the code but knowing what to write and where it fits.
I've been looking at platforms that try to automate more of the end-to-end data engineering workflow, Databricks, AI workflows, Genesis Data Agents, and Snowflake Cortex AI, but they all introduce their own failure modes. Debugging a multi-step agent decision is even harder than debugging a bad SQL query.
The tool doesn't fix the fundamental need to understand your data and your architecture. I'm now forcing myself to do one raw coding session per week without any AI help, just to keep the muscle memory alive. Feels like studying for a test I already passed, but the alternative is being useless when the API goes down.
