Turn 9 of a 20-turn agent session. The model starts repeating itself — asking for context it already has, re-deriving conclusions it reached three turns ago. Your agent has effectively restarted mid-session.
This isn't a model capability failure. The model hasn't "forgotten" in the way humans forget. What happened is a cache collapse: the token prefix that was cached at the provider's inference layer was invalidated, and the model received a materially different context than it expected. The session restart is the symptom. The cache collapse is the cause.
This week, DeepSeek Reasonix landed on Hacker News with 640 points for solving this problem at the agent-framework level. Their core claim: most agent loops achieve less than 20% cache hit rate because they inadvertently break prefix immutability — by reordering messages, injecting timestamps, or modifying tool call outputs between turns. Their fix is a "cache-first loop" that keeps the prefix frozen. At DeepSeek's pricing (10× cheaper on cache hits), this translates to roughly a 50% reduction in effective per-turn cost.
We solved the same problem at the proxy layer. Here's what it looks like from the infrastructure side, and why it matters regardless of which agent framework you're running.
Prefix caching works by storing the KV (key-value) computation for the first N tokens of a context. When the next request starts with those exact same tokens — same bytes, same order — the provider reuses the cached computation instead of recomputing from scratch. For DeepSeek V4, that's a 10× cost difference. For Anthropic, it's roughly 10× on cache hits vs. misses on input tokens.
The problem is prefix immutability. Prefix caching only works if the prefix is genuinely frozen between turns. Any modification — even a minor one — invalidates the cache and forces a full recomputation.
Common ways agent loops break prefix immutability without knowing it:
In production, the failure mode is subtle. Token cost goes up (cache misses are expensive). Session quality degrades (the model loses accumulated context). The agent loop appears to "forget" — but what actually happened is that the proxy infrastructure was silently rewriting the context that should have been frozen.
Earlier this week, one of Trimio's proxy-test users — call him Tony — ran a 20-turn session on a Fireworks/Qwen3 backend. Fireworks uses server-side prefix caching with pricing similar to DeepSeek: cache hits cost roughly 10× less than fresh reads.
On turn 9, Tony's agent entered a loop. The model was asking "Let me check..." followed by the same lookup it had completed four turns earlier. Seven near-identical lookup messages in 90 seconds. Session quality tanked.
The request logs told the story:
Root cause: Trimio's compression layer (specifically the deduplication features — ToolDedup, SubstringDedup, ContextDedup, SemanticDedup) was rewriting messages at indices 2, 5, 6, 7, 8 inside the cached prefix. These rewrites were correct from a compression standpoint — they reduced token count. But they changed bytes inside the prefix, collapsing the cache on turn 9.
The underlying bug: the session anchor machinery (which tells the compression layer "don't touch messages inside this prefix") was gated on IsAnthropicRequest(). For Fireworks, Gemini, and OpenAI — all of which use implicit server-side prefix caching — the anchor gate never fired. Every message was eligible for compression, including messages inside a live cached prefix.
The fix required three PRs:
isImplicitCachedProvider() helper — returns true for Fireworks, Gemini, OpenAI (all providers with automatic server-side prefix caching). The session anchor machinery now calls SetImplicitAnchor() after each turn for all four provider families, not just Anthropic.After these PRs deployed, Tony's session type — a long multi-turn Fireworks session with heavy compression — would maintain its cached prefix for the full session duration. Cache hit rate for equivalent sessions jumped from ~40% (unreliable, anchor missing) to ~90%+ (stable, anchor protecting the prefix). Per-turn cost dropped accordingly.
DeepSeek Reasonix is a well-designed solution to a real problem. Their cache-first loop design is elegant. But it requires you to build your agent with Reasonix's architecture, and it targets DeepSeek backends specifically.
The proxy approach is different:
The Reasonix approach and the proxy approach are complementary, not competing. Reasonix optimizes your agent loop for cache stability. The proxy handles the compression and routing layer that sits below the agent loop. You can run both.
For a session with 70k context tokens and 10k new tokens per turn:
For a 20-turn agent session on Fireworks/Qwen at these context depths:
At 1,000 sessions per day, that's $2,400/day in recoverable cost that was invisible until someone noticed Tony's agent loop.
If your proxy or provider dashboard exposes per-turn token breakdowns (fresh vs. cached read vs. cached write), look for this pattern:
cache_read_tokens (this is good — the prefix is being hit)cache_read_tokens, spike in fresh tokens (cache collapsed)cache_read_tokens stays low (no recovery — the session is effectively restarted)The drop can be gradual or sudden. In Tony's case it was sudden (turn 9, full collapse). In cases where compression is less aggressive, it can be a slow erosion over many turns.
The other signal: agent quality degradation mid-session that doesn't correlate with any change your application made. If your agent starts asking for context it already received, or repeating conclusions it reached earlier, check the cache token trace before assuming it's a model capability problem.
The session anchor work matters beyond Fireworks/Qwen. With Gemini response translation now complete (PRs #620/#621), LCR routes to Google Gemini backends return responses in the client's native format (Anthropic or OpenAI). Combined with the session anchor extension, a route that fails over from Anthropic to Gemini — or from Fireworks to OpenAI — now preserves cache stability across the provider switch.
Without the session anchor: a failover event collapses the cache (the new provider doesn't have the cached prefix). With it: the anchor is provider-aware, and the compression layer doesn't touch the frozen prefix regardless of which provider receives the request.
Multi-provider routing is only useful if the session quality holds across providers. Cache stability is a prerequisite for that.
Agent memory failure is an infrastructure problem, not a model problem. The model behaves consistently when its context is stable. The proxy and compression layers that sit between your application and the model can silently destabilize that context — unless they're designed with cache prefix protection as a first-class constraint.
DeepSeek Reasonix approached this from the agent-framework direction and got 640 HN points for it. The problem is real and the community knows it. The proxy-layer approach solves it transparently, for any agent, any framework, any provider — without requiring you to rebuild your agent loop.
Trimio is the LLM API gateway with built-in session anchor protection, multi-provider cache stability, and per-request token cost visibility. One URL change. No agent modifications required. See how it works.