Provider Credential Flag

A provider row names where its credential comes from in api_key_env, but a sealed key is never returned, so that field alone cannot tell “a key is stored” from “no credential at all”. Every provider the control plane returns therefore carries one more field:

The field is on GET /api/v1/orgs/{org_id}/providers, on the response to POST /api/v1/orgs/{org_id}/providers, and on the response to PUT /api/v1/providers/{id}.

{  "id": "6d0f1f9c-6f0e-4d1a-9d3f-2a5d8f3a1b22",  "name": "openai-primary",  "kind": "openai",  "api_base": "https://api.openai.com",  "api_key_env": null,  "has_stored_key": true}

Reading a row as a credential source:

has_stored_key reports only that a row exists. It never carries the key, its ciphertext or its nonce, and it does not say whether the key can still be unsealed; Test connection on the provider page does. An update that omits api_key leaves the stored key alone and the response still reports it, and sending api_key: "" deletes the key and flips the field to false.

Testing a provider and one of its models#

POST /api/v1/providers/{id}/test probes the provider’s free model-list endpoint and reports whether it answers: reachable, status, latency_ms, probed_url, credential, models_found and error. It sends the provider’s own credential, so it needs provider:update in the provider’s scope (the organization for an org-wide provider, the project for a project-scoped one) and goes through the egress policy. A credential variable that names one of the control plane’s own secrets is refused.

Add ?model=<name> to also look an upstream model up in the catalogue the probe returned. The response then carries model_listed:

null also covers an unreachable provider and a kind whose probe is a liveness endpoint rather than a model list (tei). The name is compared by its last path segment, the way the catalogue’s own ids are reduced, so publishers/google/models/gemini-2.5-pro and gemini-2.5-pro agree.

curl -X POST "$CONTROL/api/v1/providers/$PROVIDER_ID/test?model=gpt-4o" \  -H "Authorization: Bearer $TOKEN"

The dashboard’s model sheet uses it: the Check button beside a target’s upstream model reports listed, not listed, no catalogue to check, or why the probe failed.

Whether a provider reports prompt-cache metrics#

Cache-aware routing and the cache columns of analytics only work for a provider that returns its cache usage. POST /api/v1/providers/{id}/probe-cache?model=<name> asks the provider for one token of that upstream model and records whether the answer’s usage carries a prompt-cache field, under any of the spellings the gateway reads (see Cached prompts and thinking tokens). A field stated as 0 counts: the provider reports, it just had nothing cached. It needs provider:update, sends the provider’s credential through the egress policy, and is not run on its own, because it is a completion and costs a token.

The response carries recorded, verdict (provider_id, reports, spelling, model, probed_at), status, latency_ms, probed_url and error. An upstream that does not answer with a completion that has a usage object tells nothing about its cache reporting, so recorded is false and the earlier verdict is left alone. Bedrock, Vertex, Azure OpenAI, native Gemini and the embedding and decision servers cannot be probed this way and answer 400.

GET /api/v1/orgs/{org_id}/provider-cache-probes lists the stored verdicts. A provider with no entry was never probed, which is not the same as “does not report”. The dashboard shows the three apart on Providers, Models and Provider Groups, with the time the verdict was established. The verdict is what one response showed, so a provider that adds the field only on a hit reads as not reporting; probe again with another model, or read the time and model beside it.

Cache telemetry sources#

A provider can name the source the cache-aware strategies read, in two optional fields on the provider row, on POST /api/v1/orgs/{org_id}/providers and on the provider update. They hold the same settings as [providers.kv_events] and [providers.lmcache] in rolter.toml; see cache-aware routing.

Only endpoint is required; the other keys take the defaults the config file documents. Both endpoints go through the egress policy and a zero interval is refused with 400. Neither holds a credential.

On update, omitting a field leaves it unchanged, null clears it, and an object replaces the whole setting. A provider that came from rolter.toml is managed by the file, so its settings change there.