Two kinds of cache. Two different kinds of saving.
“Cache hit” means two completely different things in AI coding. A provider context cache makes repeated prompt context cheaper. The PatchMesh solution cache removes repeated work from the request entirely. PatchMesh reports each separately and never blurs them together.
What is provider context caching?
When you send a prompt to a model provider, it processes your input tokens. If a later request begins with the same prefix — the same system instructions, the same repository context — the provider can reuse the computation it already did for that prefix and bill those input tokens at a cheaper cached-input rate (sometimes free, depending on plan). The tokens are still part of the request; they're just discounted.
Reusable prefixes become hits
A stable prefix the provider has seen before — identical system prompt and context — can be served from its cache.
New content is a miss
The genuinely new part of your prompt — today's task, the latest conversation — is processed fresh at the normal input rate.
Output is still generated
Caching discounts input. The model still has to generate the output tokens, which still cost money unless a provider prices them otherwise.
Pricing differs by provider
Cached-input rates, minimum cacheable sizes and cache-write fees vary by provider and model. Don't assume one mechanism fits all.
Sometimes free — on some plans
A few providers make cache hits free under specific plans or token packs. That's plan-specific, not universal.
Retention can expire
Provider caches are typically short-lived. A prefix cached minutes ago may be a miss now — you can't rely on it persisting.
Order matters
Cache eligibility usually depends on an exact, stable prefix. Reordering or editing earlier content can destroy the hit.
It isn't semantic retrieval
Provider caching matches identical text it has already processed. It does not understand that two differently-worded tasks are the same engineering problem.
It's billing/performance metadata
A provider cache hit tells you about price and speed. It says nothing about whether the code is correct — never treat it as a quality signal.
Meituan's open-source LongCat-2.0 (MIT-licensed, ~$0.20/M input) is a live case of free cache hits: “only cache-miss inputs and final token generations consume the package quota.” Cheap tokens and free cached input are great — but notice what still costs: every output token, and every time the agent re-derives work that already exists. A free input cache discounts the prompt; it doesn't stop the model regenerating the same implementation. That gap is exactly what PatchMesh removes.
How PatchMesh is different
PatchMesh doesn't make your repeated context cheaper — it removes the need to send and generate so much of it. When a task is already solved, PatchMesh returns a verified solution capsule (code, tests, provenance, licence). The agent adapts a proven answer instead of re-reading the repo and regenerating an implementation from scratch. Those tokens are avoided, not discounted — they were never sent or generated. And because it matches on the engineering task, it works even when the wording, the repo or the developer is different — across people and organisations, persisting independently of any provider.
Tokens discounted. Still sent. Cheaper input rate. Provider-controlled, expiring, prefix-exact, single-provider, no quality signal.
Tokens avoided. Never sent or generated. Semantic task match. Cross-team, persistent, with tests, licence and provenance.
A worked example
Illustrative figures, to show how the two effects stack — not measured production values.
First request
Nothing cached yet.
2nd request — provider cache only
Repeated prefix is discounted.
2nd request — with PatchMesh
Reuse removes work; cache discounts the rest.
Provider caching took the repeated 90k prefix and made it cheaper — but all 100k tokens were still part of the request, and the model still generated 5,000 output tokens. With PatchMesh, 80,000 of those input tokens (and most of the output) are avoided entirely; the provider cache then discounts the small remainder. Both effects, working together.
How PatchMesh reports each saving
Every PatchMesh receipt splits the two effects so you always know which is which:
- Tokens avoided — input and output PatchMesh removed from the request by reusing a verified solution.
- Tokens discounted — input still sent, billed at the provider's cached rate, with the cache-hit ratio shown.
- Honest about unknowns — if the provider didn't report cache usage, we say so rather than infer a hit. A provider cache hit never raises a solution's trust level.
- Never flattened — value is classified as direct cash avoided, allowance preserved, or equivalent compute value depending on how you actually pay, and the categories are never summed into one fake number.
Because PatchMesh isn't a proxy, it doesn't see your provider's response — so by default the split is an honest estimate. Report the real usage and it becomes measured. Two ways: the MCP tool patchmesh_record_provider_cache_usage, or the CLI, which can parse a raw Anthropic/OpenAI response:
# parse a saved provider response and report the real cache split npx @patchmesh/agent usage report --request <id> --file response.json PatchMesh request savings Tokens avoided (PatchMesh) 67,500 Provider-cached input 15,000 (75% of input sent — discounted) Uncached input (full rate) 5,000 Output generated 1,500 Confidence measured
We only accept provider-reported numbers — a cache hit is never inferred from repeated context, and never raises a solution's trust level.