Frequently asked questions

Every gray-zone sample you submit is scoped to your tenant — it's only ever used to train and evaluate your own verifier, never shared across tenants. Passwords are bcrypt-hashed, API keys are stored as a one-way hash, and all traffic runs over HTTPS. See the full Privacy Policy for details.

Yes — CacheVerifier doesn't replace your cache or do similarity search itself. You call /v1/verify on candidates your own cache lookup (GPTCache, LangChain, a self-built Redis lookup, a RAG pipeline's retrieval cache) already produced, specifically the gray-zone ones where similarity alone isn't a confident enough signal.

No — different layer, different failure mode. Prompt caching (Anthropic's, OpenAI's) is a provider-side optimization that skips redundant token processing when a request reuses the same prompt prefix; it only fires on a byte-for-byte (or near-identical) match, so there's nothing ambiguous to verify. Semantic caching matches on meaning instead of exact text — a differently-worded question can still hit a cached answer — and that's exactly what creates the gray zone /v1/verify exists to catch. The two aren't competitors; a system can use both at once, and verification only applies to the semantic cache side. Full breakdown: Prompt caching vs semantic caching.

No — a KV cache lives inside a single model's own forward pass, caching the attention key/value tensors for tokens already processed so a later token in the same (or a resumed) context doesn't recompute them. It's an inference-optimization layer, not a response cache, and there's no cross-request ambiguity for it to get wrong. A semantic cache reuses a previous response across separate requests based on how similar two different queries are — that's a different layer with a different failure mode (a near-miss served as if it were an exact match), which is what /v1/verify exists to catch. The two can run in the same stack without conflicting.

No — those evaluate retrieval/generation quality offline, against a labeled or LLM-judged eval set, usually before or between deploys. CacheVerifier runs synchronously on live traffic, specifically on cache hits your own semantic cache (or RAG pipeline's retrieval-and-reuse layer) already flagged as similar-but-uncertain, deciding per-request whether to actually serve that cached answer. Complementary, not competing: an eval framework tells you your system is accurate on average; CacheVerifier catches the individual gray-zone reuses that would have been wrong anyway, at serve time, on real traffic an offline eval set won't have seen.

The similarity range where a cache hit might be right or might be wrong — high enough that a plain threshold would call it a match, not high enough to be confident. That's the range where a verification call actually changes the outcome; outside it, a plain similarity score is usually fine on its own. See Why Similarity Fails for the concrete failure modes that live in this range — false-positive taxonomy, hallucination memory, and more — and Semantic cache similarity thresholds for how the τ_low / τ_high cutoffs that bound it are set.

Any of them — /v1/verify is a plain REST call that takes a (query, candidate_answer) pair, so it's backend-agnostic by design. The cacheverifier package's cacheverifier.integrations.gptcache.CacheVerifierEvaluation is a ready-made drop-in for GPTCache's own SimilarityEvaluation interface; LangChain, LlamaIndex, a vector database like Pinecone or Weaviate, or a self-built Redis cache call the REST API — directly or via the Python client.

There's an official Python client: pip install cacheverifier gives you a typed CacheVerifier class over httpx — cv.verify(...), cv.feedback(...), cv.finetune() — plus a GPTCache adapter (the [gptcache] extra) and an offline cacheverifier healthcheck CLI (the [healthcheck] extra). Source and examples: cacheverifier-python. Since 0.3 its cv.verify(...) is built for the request path: a 1 s timeout and fail-closed (approved: false) on a timeout or 5xx, with fail_open=True to flip that. No Node or other-language client yet — everything else calls the REST /v1/ endpoints directly.

Measured on our origin with no network in the path (loopback, 300 warm calls, 2026-09-04): p50 33 ms, p95 44 ms, p99 47 ms for a single /v1/verify call, end to end on our side — the cross-encoder forward pass (~17 ms), FastAPI auth / tenant lookup / rate-limit / serialization (~13 ms), and nginx TLS termination plus the proxy hop (~3 ms). The latency_ms field in every response reports just the forward pass; a batch call amortizes it across items. Your own end-to-end number is that plus the network round trip from wherever you call us — which depends entirely on your location, so measure it against your own traffic rather than taking a figure from us. One real data point: 150 calls from a US-East client to this same production endpoint measured a full round trip of p50 ~33 ms, p95 ~39 ms (2026-09-09) — close enough to the server-only numbers above that network overhead was negligible on that one path. That's one geography, one run, not a guarantee for yours; a client on another continent or a different network path will see a different number. Either way, verification only runs on the gray-zone candidates your cache flagged as uncertain, not every request, so it's off the hot path for confident hits and misses.

No — it only runs on the gray-zone candidates your own cache already flagged as uncertain, a small fraction of hits, and even then it's a single cross-encoder pass, far cheaper and faster than a full LLM call. Semantic caching exists to cut LLM API costs and latency in the first place; a wrong cache hit — the wrong refund policy, the wrong cancellation step — costs more than the call it avoided. CacheVerifier's job is keeping the savings caching was supposed to give you, without paying for them in wrong answers.

Your dashboard's Overview page has a "This period's savings" card: LLM calls avoided and wrong cache hits avoided, both computed from this period's real /v1/verify volume and your active verifier's own held-out performance — an estimate extrapolated from your own feedback data onto real traffic, not a per-request count. It converts to a $ figure only once you tell it what one of your own LLM calls costs (PUT /v1/settings/llm-cost-assumption) — we don't call your LLM ourselves, so we have no way to know that number, and we'd rather show call counts than a made-up dollar amount.

Yes — auth endpoints (login/register) are limited to 10 requests/minute per IP, classic brute-force-target treatment. Everything else under /v1/ (verify, feedback, fine-tune jobs, ...) is 120 requests/minute, keyed by API key where one is sent. See the API Reference for exact numbers per endpoint.

It depends on your data. The default cold-start mode is auto: run a Health Check and it measures whether the shared, un-tuned stock model is actually accurate on your traffic. If its held-out AUC clears 0.68, /v1/verify serves the stock model during cold start automatically; if it's below that — or you haven't run a Health Check yet — every gray-zone call returns approved: false instead, because internal testing found the stock model can be net harmful on a mismatched domain, not just unhelpful (a false approval serves a wrong cached answer; a false rejection only costs one regeneration). You can also force it either way — always run the stock model, or never — from cold-start settings.

The API enforces a hard floor of 20 rows. Our own testing across four independent datasets found fine-tuning can be a net negative below roughly 1,000 rows on the hardest dataset we tested — see the Research page for the actual numbers. That's exactly what the Health Check Report and the built-in guidance on that page are for: telling you whether your current sample is ready before it becomes your tenant's live verifier.

Most teams don't — a real was_correct means knowing the right answer, which is exactly what a cache is trying to avoid recomputing. Send an implicit_signals object on /v1/feedback instead: the proxies you can observe — the user re-asked, the chat was escalated to a human, the message got a thumbs-down, the session was abandoned. The service derives was_correct plus a confidence from them; strong proxies are used like a real label, weak ones are recorded but held out of training. You don't need every row to have a signal, and you can mix implicit and explicit feedback freely.

Yes, with too little data or too-noisy labels — this is a real, measured failure mode, not a hypothetical (see Research). Three defenses against it: the Health Check Report flags small samples and near-ceiling baselines before you commit to anything; a real fine-tune job itself won't auto-activate a result that looks wrong (an operating-point jump, a calibration-segment rate mismatch, or too-noisy a training signal all come back model_status: "held_for_review" instead of replacing your live verifier — see the API Reference); and drift monitoring watches for a verifier that's since drifted out of step with current traffic.

Optionally, yes. Pass target_risk to POST /v1/finetune/jobs (e.g. 0.01 for "keep false reuses under 1%") to certify a threshold via Conformal Risk Control — a conformal-prediction technique that gives a distribution-free, finite-sample guarantee, not a best-effort estimate, that accepting only candidates scored above certified_threshold keeps false-reuse risk at or below your target, as long as future traffic looks statistically similar to the feedback it was calibrated on. It's a separate number from the day-to-day threshold /v1/verify actually uses — certification is informational, never auto-applied — and it needs enough held-out feedback to certify anything non-trivial, so tighter targets need more data. See the API Reference for exact field names and what each certification_status value means.

Yes. By default threshold is picked via Youden's J, which weighs a wrong approval and a missed cache equally — most businesses don't actually value those two mistakes the same amount. Pass cost_ratio to POST /v1/finetune/jobs (how many times more expensive a wrong-answer reuse is than one extra LLM call — e.g. 5.0) to instead pick the live threshold by minimizing expected cost at that ratio. Every job also reports threshold_by_cost_ratio, a reference table showing what threshold you'd get at a few common ratios on your own data, before you commit to a value — the dashboard renders this as a "which row matches your situation" table on both a real fine-tune job and the Health Check Report. See the API Reference and the Research page for the methodology.

Not the verifier service — there's no packaged self-host distribution of /v1/verify today, and running your own instance isn't a supported product path right now (the underlying verifier and fine-tuning code is architected to be backend-agnostic, but that's not the same as a supported install). One piece does run locally: cacheverifier healthcheck — from pip install "cacheverifier[healthcheck]" — runs the same stock-vs-fine-tuned AUC evaluation as the hosted Health Check Report entirely offline, with no queries or answers leaving your machine. See the Local Health Check section of the docs.

None yet — the service doesn't have a formal SLA yet. We'll publish one as the service matures; see the Terms of Service for the current status.

Self-serve verify and fine-tuning stay free forever, so there's nothing to lose access to on that front regardless of payment status. If a Managed subscription lapses, your account and data aren't deleted — only an explicit account deletion (from the dashboard's Danger Zone) removes them.

Extra calls are billed from your prepaid topup balance at $1.50 per 1,000 (minimum topup $50) rather than blocked outright — you only actually get a 429 once both the included quota for the period and the balance are exhausted. Top up anytime from the Billing page, with or without an active Managed subscription.

You can cancel anytime from the Billing page — cancellation stops future renewals, and you keep access through the end of the period you already paid for. We don't refund the current billing period, and usage top-ups are non-refundable once credited to your account. See the Terms of Service for the full policy.

Your subscription moves to a past_due state — access isn't cut off immediately, and Antom automatically retries the charge. You'll see a warning on the Billing page while this is happening. Contact support if a retry doesn't resolve it.