Frequently asked questions

Every gray-zone sample you submit is scoped to your tenant — it's only ever used to train and evaluate your own verifier, never shared across tenants. Passwords are bcrypt-hashed, API keys are stored as a one-way hash, and all traffic runs over HTTPS. See the full Privacy Policy for details.

Yes — CacheVerifier doesn't replace your cache or do similarity search itself. You call /v1/verify on candidates your own cache lookup (GPTCache, LangChain, a self-built Redis lookup, a RAG pipeline's retrieval cache) already produced, specifically the gray-zone ones where similarity alone isn't a confident enough signal.

No — different layer, different failure mode. Prompt caching (Anthropic's, OpenAI's) is a provider-side optimization that skips redundant token processing when a request reuses the same prompt prefix; it only fires on a byte-for-byte (or near-identical) match, so there's nothing ambiguous to verify. Semantic caching matches on meaning instead of exact text — a differently-worded question can still hit a cached answer — and that's exactly what creates the gray zone /v1/verify exists to catch. The two aren't competitors; a system can use both at once, and verification only applies to the semantic cache side. Full breakdown: Prompt caching vs semantic caching.

No — a KV cache lives inside a single model's own forward pass, caching the attention key/value tensors for tokens already processed so a later token in the same (or a resumed) context doesn't recompute them. It's an inference-optimization layer, not a response cache, and there's no cross-request ambiguity for it to get wrong. A semantic cache reuses a previous response across separate requests based on how similar two different queries are — that's a different layer with a different failure mode (a near-miss served as if it were an exact match), which is what /v1/verify exists to catch. The two can run in the same stack without conflicting.

No — those evaluate retrieval/generation quality offline, against a labeled or LLM-judged eval set, usually before or between deploys. CacheVerifier runs synchronously on live traffic, specifically on cache hits your own semantic cache (or RAG pipeline's retrieval-and-reuse layer) already flagged as similar-but-uncertain, deciding per-request whether to actually serve that cached answer. Complementary, not competing: an eval framework tells you your system is accurate on average; CacheVerifier catches the individual gray-zone reuses that would have been wrong anyway, at serve time, on real traffic an offline eval set won't have seen.

The similarity range where a cache hit might be right or might be wrong — high enough that a plain threshold would call it a match, not high enough to be confident. That's the range where a verification call actually changes the outcome; outside it, a plain similarity score is usually fine on its own. See Why Similarity Fails for the concrete failure modes that live in this range — false-positive taxonomy, hallucination memory, and more — and Semantic cache similarity thresholds for how the τ_low / τ_high cutoffs that bound it are set.

Any of them — /v1/verify is a plain REST call that takes a (query, candidate_answer) pair, so it's backend-agnostic by design. The cacheverifier package's cacheverifier.integrations.gptcache.CacheVerifierEvaluation is a ready-made drop-in for GPTCache's own SimilarityEvaluation interface, and cacheverifier.integrations.langchain.VerifiedCache wraps any LangChain cache, such as RedisSemanticCache. LlamaIndex, a vector database like Pinecone or Weaviate, or a self-built Redis cache call the REST API — directly or via the Python client.

There's an official Python client: pip install cacheverifier gives you a typed CacheVerifier class over httpx — cv.verify(...), cv.feedback(...), cv.finetune() — plus a GPTCache adapter (the [gptcache] extra) and an offline cacheverifier healthcheck CLI (the [healthcheck] extra). Source and examples: cacheverifier-python. Since 0.3 its cv.verify(...) is built for the request path: a 1 s timeout and fail-closed (approved: false) on a timeout or 5xx, with fail_open=True to flip that. No Node or other-language client yet — everything else calls the REST /v1/ endpoints directly.

Measured on our origin with no network in the path (loopback, 300 warm calls, 2026-09-04): p50 33 ms, p95 44 ms, p99 47 ms for a single /v1/verify call, end to end on our side, of which the model's forward pass is about 17 ms. The latency_ms field in every response reports just the forward pass; a batch call amortizes it across items. Your own end-to-end number is that plus the network round trip from wherever you call us — which depends entirely on your location, so measure it against your own traffic rather than taking a figure from us. One real data point: 150 calls from a US-East client to this same production endpoint measured a full round trip of p50 ~33 ms, p95 ~39 ms (2026-09-09) — close enough to the server-only numbers above that network overhead was negligible on that one path. That's one geography, one run, not a guarantee for yours; a client on another continent or a different network path will see a different number. Either way, verification only runs on the gray-zone candidates your cache flagged as uncertain, not every request, so it's off the hot path for confident hits and misses.

No — it only runs on the gray-zone candidates your own cache already flagged as uncertain, a small fraction of hits, and even then it's a single cross-encoder pass, far cheaper and faster than a full LLM call. Semantic caching exists to cut LLM API costs and latency in the first place; a wrong cache hit — the wrong refund policy, the wrong cancellation step — costs more than the call it avoided. CacheVerifier's job is keeping the savings caching was supposed to give you, without paying for them in wrong answers.

Your dashboard's Overview page has a "This period's savings" card: LLM calls avoided and wrong cache hits avoided, both computed from this period's real /v1/verify volume and your active verifier's own held-out performance — an estimate extrapolated from your own feedback data onto real traffic, not a per-request count. It converts to a $ figure only once you tell it what one of your own LLM calls costs (PUT /v1/settings/llm-cost-assumption) — we don't call your LLM ourselves, so we have no way to know that number, and we'd rather show call counts than a made-up dollar amount.

Yes — auth endpoints (login/register) are limited to 10 requests/minute per IP, classic brute-force-target treatment. Verify calls (/v1/verify, /v1/verify/batch, /v1/verify/decisions) allow about 20 requests/second per API key, since they sit on your request path; everything else under /v1/ is 120 requests/minute per key. Beyond that, batch your verify calls or score locally on a Managed plan, which has no request limit. See the API Reference for exact numbers.

It depends on your data. The default cold-start mode is auto: run a Health Check and it measures whether the shared, un-tuned stock model is actually accurate on your traffic. If it clears the accuracy bar, measured on a test split large enough to trust, /v1/verify serves the stock model during cold start automatically; if it's below that, the Health Check was too small to tell, or you haven't run one yet, every gray-zone call returns approved: false instead, because internal testing found the stock model can be net harmful on a mismatched domain, not just unhelpful (a false approval serves a wrong cached answer; a false rejection only costs one regeneration). You can also force it either way — always run the stock model, or never — from cold-start settings.

The API enforces a hard floor of 20 rows. Our own testing across four independent datasets found fine-tuning can be a net negative below roughly 1,000 rows on the hardest dataset we tested — see the Research page for the actual numbers. That's exactly what the Health Check Report and the built-in guidance on that page are for: telling you whether your current sample is ready before it becomes your tenant's live verifier.

Most teams don't — a real was_correct means knowing the right answer, which is exactly what a cache is trying to avoid recomputing. Send an implicit_signals object on /v1/feedback instead: the proxies you can observe — the user re-asked, the chat was escalated to a human, the message got a thumbs-down, the session was abandoned. The service derives was_correct plus a confidence from them; strong proxies are used like a real label, weak ones are recorded but held out of training. You don't need every row to have a signal, and you can mix implicit and explicit feedback freely.

Yes, with too little data or too-noisy labels — this is a real, measured failure mode, not a hypothetical (see Research). Three defenses against it: the Health Check Report flags small samples and near-ceiling baselines before you commit to anything; a real fine-tune job itself won't auto-activate a result that looks wrong (an operating-point jump, a calibration-segment rate mismatch, or too-noisy a training signal all come back model_status: "held_for_review" instead of replacing your live verifier — see the API Reference); and drift monitoring watches for a verifier that's since drifted out of step with current traffic.

Optionally, yes. Pass target_risk to POST /v1/finetune/jobs (e.g. 0.01 for "keep false reuses under 1%") to certify a threshold via Conformal Risk Control — a conformal-prediction technique that gives a distribution-free, finite-sample guarantee, not a best-effort estimate, that accepting only candidates scored above certified_threshold keeps false-reuse risk at or below your target, as long as future traffic looks statistically similar to the feedback it was calibrated on. It's a separate number from the day-to-day threshold /v1/verify actually uses — certification is informational, never auto-applied — and it needs enough held-out feedback to certify anything non-trivial, so tighter targets need more data. See the API Reference for exact field names and what each certification_status value means. One caveat, stated plainly: that calibration pool is whatever feedback you happened to submit, not automatically a random sample of the verdicts /v1/verify actually approves — whether you report on an approved call is likely correlated with whether it was right. Turn on randomized auditing (PUT /v1/settings/audit-rate) and GET /v1/monitor/audited-crc-threshold gives you the same guarantee calibrated only on a known-probability sample instead, once you've collected enough of it.

Yes. By default threshold is picked via Youden's J, which weighs a wrong approval and a missed cache equally — most businesses don't actually value those two mistakes the same amount. Pass cost_ratio to POST /v1/finetune/jobs (how many times more expensive a wrong-answer reuse is than one extra LLM call — e.g. 5.0) to instead pick the live threshold by minimizing expected cost at that ratio. Every job also reports threshold_by_cost_ratio, a reference table showing what threshold you'd get at a few common ratios on your own data, before you commit to a value — the dashboard renders this as a "which row matches your situation" table on both a real fine-tune job and the Health Check Report. See the API Reference and the Research page for the methodology.

Scoring, yes, on a Managed plan. The Python SDK's LocalVerifier downloads the model we fine-tuned for you and runs it in your own stack: no network hop on the request path, your query and answer texts stay with you, and local calls aren't metered or rate-limited. It decides exactly as the hosted API would (every download is checked against reference verdicts before it's used), and if our service becomes unreachable, a running client keeps deciding with the model it has. Fine-tuning, recalibration, drift monitoring and audits stay hosted. See Local scoring. Evaluation, yes, on any plan: cacheverifier healthcheck (from pip install "cacheverifier[healthcheck]") runs the same stock-vs-fine-tuned evaluation as the hosted Health Check Report offline, with nothing leaving your machine — see Local Health Check. There's no packaged self-host distribution of the whole service.

Not on the hosted verifier yet. It's an English model: it can't read other languages, so its verdicts on that traffic are close to all-approve or all-reject, depending on the model. Please don't send non-English traffic to it for now. Multilingual base models fine-tune well on Chinese gray-zone data in our offline evaluation (see the Docs), but they aren't deployed. You can run that evaluation on your own data with the local Health Check.

Keep randomized audits on (they're on by default for new accounts) and report each audited call's outcome with its verification_id — that's what gives you a measured error rate. When you fine-tune, set cost_ratio (how many times worse a wrong reuse is than an extra LLM call) instead of relying on the default threshold. Once audits are flowing, turn on ACI so the live threshold follows what the audits find. Send cache_entry_id with feedback so Entries to refresh hands back your own cache keys. And at volume, score locally.

Almost certainly — CacheVerifier is available worldwide, except where US sanctions law prohibits it. We block access from Cuba, Iran and North Korea, and the service may not be used from Crimea or the so-called Donetsk or Luhansk People's Republics. Accounts found to be used from those regions are suspended. See section 6 of the Terms of Service.

None yet — the service doesn't have a formal SLA yet. We'll publish one as the service matures; see the Terms of Service for the current status.

Verify and fine-tuning on the Free plan stay free forever, so there's nothing to lose access to on that front regardless of payment status. If a Managed subscription lapses, your account and data aren't deleted — only an explicit account deletion (from the dashboard's Danger Zone) removes them.

Extra hosted verify calls are billed from your prepaid topup balance at $0.05 per 1,000 (minimum topup $10) rather than blocked outright — you only actually get a 429 once both the included quota for the period and the balance are exhausted. Top up anytime from the Billing page, with or without an active Managed subscription. Local scoring (Team and Scale) isn't metered, so it never uses quota.

You can cancel anytime from the Billing page — cancellation stops future renewals, and you keep access through the end of the period you already paid for. We don't refund the current billing period, and usage top-ups are non-refundable once credited to your account. See the Terms of Service for the full policy.

Your subscription moves to a past_due state — access isn't cut off immediately, and Antom automatically retries the charge. You'll see a warning on the Billing page while this is happening. Contact support if a retry doesn't resolve it.