Most AI gateway compression dashboards show you a "tokens saved" number. Most of them are computing it wrong — and not by a rounding error. We found a 5× overstatement in Trimio's own analytics aggregates. We're publishing the bug, the math, and the corrected formula, because if we got this wrong, others have too.
original_tokens_in - prompt_tokens — breaks the moment prompt_tokens semantics change. If prompt_tokens = fresh-only (post-caching), you're attributing cached reads as compression savings. That's not savings. That's billing math you didn't write.The naive formula for compression tokens saved is:
sum(original_tokens_in - prompt_tokens) AS compression_tokens_saved
Where original_tokens_in is the token count before compression, and prompt_tokens is what was actually sent to the provider. On the surface, this makes sense: you sent fewer tokens than you had, the delta is the savings.
The problem: prompt_tokens is an ambiguous field. Depending on the provider and schema version, it can mean:
These two interpretations produce wildly different numbers. And they're both called prompt_tokens in different implementations.
In Trimio's analytics pipeline, PR #615 introduced a schema normalization: prompt_tokens was standardized to mean fresh-only input tokens. This was the correct semantic — it makes the caching and compression savings columns cleanly separable. But it had a side effect on the CAGG (continuous aggregate) formula that hadn't been updated.
Before PR #615, the formula worked (approximately):
-- Old semantics: prompt_tokens = all input tokens sent on the wire
original_tokens_in - prompt_tokens = compression savings only
After PR #615:
-- New semantics: prompt_tokens = fresh-only tokens
original_tokens_in - prompt_tokens = compression savings + cached_read tokens
The formula was now attributing cached read tokens as compression savings. Those aren't the same thing at all — one is a reduction in what you sent to the provider, the other is tokens the provider served from its own cache that you never sent in the first place.
Take a typical heavy agentic turn with an active cache session:
With the buggy formula:
compression_tokens_saved = original_tokens_in - prompt_tokens
= 100k - 10k (prompt_tokens = fresh-only)
= 90k ← WRONG (includes 70k cached reads)
With the correct formula:
compression_tokens_saved = original_tokens_in - compressed_tokens_in
= 100k - 80k (compressed_tokens_in = what was actually sent)
= 20k ← CORRECT
The overstatement ratio in this example is 4.5×. On sessions with higher cache utilization (90k+ cached reads against smaller compression wins), the ratio climbs above 5×. The specific 5× figure comes from our median ratio across the 1,754 affected rows in the historical audit.
The audit surfaced something else: 1,754 Anthropic rows were showing negative compression savings. That's not a rounding error. That's the formula producing a result that says "this turn cost more tokens after compression than before."
How? Anthropic's billing reports prompt_tokens inclusive of cache_write tokens in some response shapes. Cache writes are more expensive than normal input tokens — Anthropic charges ~25% of input cost as an upfront write cost, amortized across cache hits. When the old formula subtracted those write-inclusive counts from original_tokens_in, it could produce a negative delta on turns where cache was being established (write-heavy) but compression savings were modest.
These weren't real negative savings. They were a measurement artifact. All 1,754 rows were corrected to positive values after the formula fix.
compressed_tokens_in (wire-as-sent, independent of caching) not prompt_tokens (which folds in cache semantics). The correct formula: sum(original_tokens_in - COALESCE(compressed_tokens_in, prompt_tokens)). If your gateway doesn't track these as separate columns, it can't produce an accurate compression savings number.The fix introduced in PR #616:
sum(original_tokens_in - COALESCE(compressed_tokens_in, prompt_tokens))
AS compression_tokens_saved
compressed_tokens_in is the token count after compression, before any caching is applied — it's the literal count of tokens in the request body sent to the provider. It doesn't fold in caching semantics, which means:
prompt_tokens is definedThe COALESCE handles rows that predate the compressed_tokens_in column — falling back to prompt_tokens for pre-compression-era records where the distinction didn't exist.
Historical audit results across the compressed request table:
The 250 overstatement rows are what drove the headline number. These are the turns that matter for FinOps — they're the heavy sessions, the ones your CFO sees in the "top savings contributors" table. Getting them wrong by 5× means your compression ROI story is built on inflated numbers.
If your gateway tracks compression savings, ask your engineering team three questions:
prompt_tokens in your schema? Is it fresh-only, or all-input? Did that definition change at any schema migration point?compressed_tokens_in column (or equivalent) that captures what was actually sent to the provider, independent of caching?If you can't answer question 1 precisely, you probably have the bug. It's a common one because schema migrations happen incrementally and the analytics CAGG formulas are rarely updated in lockstep.
compressed_tokens_in independent of caching semantics. The numbers in your dashboard are real. No 5× overstatement. No negative savings artifacts.