CacheVerifier vs Krites

Krites (arXiv 2602.13165, February 2026, Apple Machine Learning Research) targets tiered caches: a static tier of offline-vetted answers and a dynamic tier filled online, sharing one similarity threshold. On the request path it behaves exactly like a static-threshold policy, so latency doesn’t change. When a prompt’s nearest static neighbour falls just below the threshold, it asks an LLM judge, asynchronously, whether the curated answer fits the new prompt; if it does, the answer is promoted into the dynamic tier under the new prompt, so repeats and paraphrases reuse it. In trace-driven simulations on conversational and search-style traffic it serves up to 3.9 times more requests with curated answers than tuned baselines. It’s a research paper; we found no released code.

CacheVerifier is a hosted verification call plus the loop around it. On the gray-zone hits your cache routes to /v1/verify, a cross-encoder fine-tuned on your own feedback approves or rejects serving the cached answer, in tens of milliseconds. Randomized audits measure how often approved answers are wrong, drift triggers recalibration, and entries your feedback marks stale are listed for refresh.

Both put a model that reads the answer, not just the embeddings, on the band of similarity scores around the threshold, which is where a single threshold both misses safe reuse and serves wrong answers. Our own paper studies the synchronous version of that idea; Krites is a careful asynchronous one.

When it decides. Krites decides after the response, so it adds no latency, but the first request in the gray zone is still a miss and only later ones benefit. CacheVerifier decides before serving, at the cost of a verify call on that request.

Which direction. Krites only adds hits: it recovers curated answers that fell below the threshold. A wrong answer that scores above the threshold is served as before. CacheVerifier checks whatever band you route to it, including hits above your cutoff, so it can also reject.

What it needs. Krites assumes a curated static tier. CacheVerifier works with any cache, including a purely dynamic one.

The judge. Krites uses a general LLM judge, a model call per gray-zone request, and the paper doesn’t report the judge’s own error rate. CacheVerifier’s verifier is a small model trained on your labels, and audits measure its error rate in production.

The two aren’t exclusive. The promotion step Krites describes needs a judge, and a verify call made off the request path can be one: check a near-miss after responding, and write the answer under the new prompt if it’s approved. We haven’t benchmarked that setup, so treat it as a design option, not a result.