Every week I see the same discussion:
I increasingly think this is wrong.
Most long-horizon agent failures I’ve seen are not:
IQ failures,
reasoning failures,
or benchmark failures.
They are:
text
execution dynamics failures
And we keep trying to solve them with:
better prompts,
larger context windows,
reflection loops,
constitutional layers,
self-critique,
more reasoning tokens.
But the underlying issue is that modern agents are effectively:
text
opaque stochastic distributed systems
with almost no runtime observability.
The hidden problem
A coding agent runs for 6 hours.
At the beginning:
text
read → validate → patch → test
6 hours later:
text
rewrite → retry → rewrite → rollback → retry → patch → retry
Final output still sometimes works.
But the trajectory has already degraded.
This is the scary part:
most agent failures are not catastrophic.
They are:
gradual,
sparse,
silent,
accumulative.
Exactly like entropy growth in distributed systems.
Current agents are architecturally weird
Right now we ask the LLM to simultaneously be:
planner,
memory,
scheduler,
filesystem manager,
execution engine,
validator,
recovery layer.
That’s insane if you think about it.
We essentially turned a probabilistic next-token predictor into:
text
kernel + RAM + orchestrator + process manager
with almost no formal execution semantics.
The industry keeps focusing on "reasoning"
But I think the real bottleneck is:
Stability(T0→Tn)Stability(T_0 \rightarrow T_n)Stability(T0→Tn)
not:
Correctness(output)Correctness(output)Correctness(output)
where:
Modern evals mostly measure:
text
single-shot correctness
Real production systems fail because of:
What if we treated agents like observable stochastic systems?
Not deterministic systems.
Not explainable cognition.
Observable stochastic systems.
This changes everything.
Instead of asking:
text
"why did the model think this?"
(which is probably impossible)
we ask:
text
"how is the execution behavior changing over time?"
Runtime metrics become more important than prompts
Imagine monitoring agents like distributed infrastructure.
Metrics like:
Transition Entropy
H(At∣St)H(A_t \mid S_t)H(At∣St)
How chaotic action selection becomes over time.
Rollback Density
R=#rollback#stepsR = \frac{\#rollback}{\#steps}R=#steps#rollback
A surprisingly strong early-warning signal.
Path Variance
How much execution trajectories diverge from healthy baselines.
Invariant Violation Rate
V=#violations#transitionsV = \frac{\#violations}{\#transitions}V=#transitions#violations
Filesystem corruption.
Invalid transitions.
Unexpected mutations.
Tool Churn Rate
Repeated useless tool invocations:
text
edit → rewrite → retry → rewrite
Often the first sign the agent is "melting".
This is NOT about understanding latent reasoning
That’s the key distinction.
I am NOT claiming:
text
we can explain transformer cognition
We probably can’t.
I’m saying:
text
we can observe execution dynamics
Huge difference.
The uncomfortable analogy
Modern agents increasingly resemble:
NOT chatbots.
And distributed systems engineering learned this lesson decades ago:
You do not eliminate uncertainty.
You:
contain it,
observe it,
replay it,
bound the blast radius.
The really hard problems
This is where things get ugly.
1. What is "healthy" behavior?
A successful execution can still be degraded.
Example:
task succeeded,
but:
14 retries,
3 rollbacks,
exploding token usage,
unstable tool loops.
Success metrics alone completely miss this.
So now you need:
trajectory families,
probabilistic baselines,
task archetypes.
This becomes:
text
runtime science
not prompt engineering.
2. Snapshotting state is expensive
For coding agents:
state ≈ entire filesystem.
Naive observability will kill performance.
You probably need:
Basically:
text
Git/Nix semantics for agents
3. Adapter layers are hell
LangChain.
Claude Code.
OpenHands.
MCP.
Streaming tools.
Nested tools.
Async execution.
Normalizing execution traces across frameworks is probably a research project itself.
4. Thresholds are dangerous
Simple:
python
if drift_score > threshold:
will absolutely fail.
Healthy exploration can look unstable.
Hard tasks naturally produce entropy spikes.
You likely need:
Bayesian change point detection,
probabilistic regime shifts,
adaptive thresholds.
But despite all this…
…I increasingly think this direction is inevitable.
Because the alternative is:
text
trust increasingly autonomous opaque systems
with no runtime observability.
And I don’t think that scales.
The core idea
The future may not belong to:
text
smarter prompts
but to:
text
observable stochastic execution systems
Systems that:
Not AGI gods.
More like:
text
Kubernetes for stochastic actors
And honestly?
We spent decades learning that distributed systems become production-safe only after observability, replayability, and bounded failure semantics.
Why are we assuming stochastic autonomous systems will be different?
Maybe the next major leap in agent engineering is not better reasoning.
Maybe it’s finally admitting that reasoning is not enough without runtime observability.
