A provider is one upstream endpoint plus the credential Rolter uses to reach it. That credential is the provider key — the one the provider issued to Rolter, never the virtual key a client sends (which key do I need?). Routes and provider groups select targets, and every target names a provider.
Add them from Providers in the dashboard, or through the management API. Every column of the list sorts: click a heading for ascending, again for descending, and a third time to go back to the order the control plane sent.
The Credential column says where each provider’s key comes from: Sealed for a key stored in the control plane, the name of the environment variable when the provider names one instead, and None when it has neither. A stored key wins over a variable the provider also names. The column shows the source only, never the key; the same fact is the has_stored_key field in the provider API.
The name is fixed once the provider is created, and the form says so. The slug, which is the provider-slug/model address, is fixed too. Leave the slug blank and its placeholder shows the one the name will give.
The Kind picker lists each provider by display name with a line on what it is (OpenAI-compatible: vLLM, TGI, LM Studio or any server that speaks the OpenAI API); the stored id sits under it. Below API base the form previews the address requests go to, and that depends on the kind: /v1/messages for Anthropic, an embeddings endpoint for Hugging Face TEI and Infinity, /v1/systemone for a System One decision server, /v1/chat/completions for most of the rest. The hint under Provider key says which header the key is sent in.
Where the credential lives#
There are two places, and they resolve in this order:
- A stored key — typed into the dashboard, sealed with AES-256-GCM under
ROLTER_KEKbefore it reaches the database, and never returned by the API again. - An environment variable —
api_key_envnames a variable that is read from the environment of each process that needs the key. The value never enters the database at all. The control plane’s own secrets cannot be named:ROLTER_KEK,ROLTER_DATABASE_URLand the like, any other name starting withROLTER_, and the database and cache connection variables, are refused with a400(invalid_fieldonapi_key_env) when you save the provider and when you test it, because the connection test sends the value to the provider’s base URL.
A stored key wins when both are set.
An environment variable has to be set in both processes. The gateway reads it for every request it forwards, and the control plane reads it when you press Test connection, so a gateway that has the variable and a control plane that does not passes traffic and fails its test with credential: env (unset). The dashboard’s field hint says the same.
Storing a key requires ROLTER_KEK on the control plane. Gateways do not need it: the control plane unseals keys when it builds the snapshot, and gateways receive them over the token-guarded /internal/snapshot, so keep the KEK off them. With it unset, the control plane still starts and the provider row still saves — the key simply cannot be sealed or read back. Run rolter check before boot; it fails on exactly this.
Test connection#
Configuring a provider is otherwise a write with no feedback. The row saves whether or not the base URL is right, and the first sign of a problem is a failed request through the gateway, attributed to whichever route happened to select it.
Test connection on a saved provider closes that loop. In the dashboard it is one click from the create: Create provider leaves the sheet open on the new provider, says it is not tested yet, and puts focus on Test connection beside the save buttons. The test does not run by itself, since it is a call to the upstream. Done closes the sheet.
The same call is available to a script:
It issues a GET against the provider kind’s free model-list endpoint — the same one the gateway’s health sweep uses, so a green test means the sweep will agree. It is a catalogue call, not a completion, so it costs nothing to run.
The key is presented the way the provider’s own API reads it, exactly as when the gateway forwards a request: x-api-key for Anthropic, api-key for Azure OpenAI, x-goog-api-key for native Gemini and a bearer token for the rest. A 401 from the test therefore means the upstream rejected the key, not that rolter sent it in a form the API ignores.
It checks the provider, not a model. models_found is a count, so a green test does not say that the upstream model name a route sends is among them, and the route sheet has no connection check of its own until the probe can answer that (#2008).
The models a provider lists#
The ids behind that count have a read of their own, which is what the route sheet’s Upstream model field suggests from:
The list is sorted, has no duplicates and stops at 2000 ids. They are the ids the gateway lists under provider-slug/model. An upstream that is down, rejects the credential or answers with something other than a catalogue gives {"models": []} rather than an error, since the list only ever suggests; Test connection is the call that says why. It reads the same free model-list endpoint with the provider’s own credential and under the same egress policy as the test. It sends the provider’s credential to the upstream just as the test does, authenticated the same way, so it takes the same permission as the test, provider:update in the provider’s scope, and a role that cannot test a provider cannot list its models either. A provider the caller may not see is answered 404. The dashboard’s route sheet shows no suggestions on a refusal and still takes any model id you type.
Reading the result#
| Field | What it tells you |
|---|---|
probed_url | the URL actually tried. A doubled /v1 is the most common misconfiguration and is invisible without this |
credential | how the key was resolved — stored, env, or none. Separates “wrong key” from “no key configured” |
models_found | how many models the upstream listed. A provider can be reachable and still serve nothing |
status | absent when the request never completed at all (DNS, TLS, connection refused, timeout) |
A 401 names which credential source was used, so a rejected key points at the right place to fix it. When that source is env (unset), the message names the variable and the process that is missing it:
The test says where it ran. It cannot see the gateway’s environment, so a green test does not prove the gateway has the variable.
When something answers, but not with a model list#
reachable is not “the upstream returned 2xx”. For provider kinds whose probe is a model catalogue, it means the response was a catalogue. A catch-all route, an SPA index, a reverse proxy or a login portal will all answer 200 at an arbitrary URL, and reporting those as reachable moves you on to debugging the wrong layer.
That case is reported as its own thing — a 2xx status with reachable: false:
The usual cause is a duplicated path segment in api_base — see the /v1 rule — where the upstream’s own catch-all answers 200 at the doubled path.
The dashboard renders this as its own amber state — Answered, but not with a model list — rather than the red used for a refused connection or a rejected key. The host is up, and it is the URL or the service behind it that is wrong, which is a different thing to go and fix.
A route whose targets are all Hugging Face TEI or Infinity servers answers POST /v1/embeddings (and POST /v1/rerank when one of them is Infinity), and refuses chat completions, messages, responses, images and audio, and rerank on TEI alone, with 400 unsupported_endpoint without calling the server, which has no such endpoint.
Kinds whose probe is a liveness endpoint rather than a catalogue keep 2xx as the criterion. Hugging Face TEI is probed at /health, which has no catalogue to return, so a bare 200 there is the whole verdict and models_found stays null.
The probe reads the stored row, not what is currently typed in the form. With unsaved edits in the sheet, Test connection is off and says it tests the saved provider; save first, then test. The credential is sealed server-side and never leaves the control plane, so a draft key could not be tested without transmitting it somewhere first.
When the credential cannot be read#
If the stored key was sealed under a different ROLTER_KEK — or the KEK is unset — the test reports that directly and does not contact the upstream:
This is a deployment problem, not a provider problem, and the message says so rather than reporting the upstream as down.
What a green result does not prove#
The probe is a model-list call. A few self-hosted runtimes serve their catalogue without checking authorization, so on those a wrong key can still return reachable: true. The hosted providers all authenticate the endpoint, so the check is meaningful there. Treat reachable as “the provider’s API answered with its catalogue”, and 401 as authoritative when it appears.
Egress policy#
The base URL is checked against the egress policy when the provider is created, and again on every test. A stored row is not a standing permission to egress: if the policy has been tightened since, the test fails on the policy rather than reaching the host.
Egress proxy#
A provider’s egress proxy is optional. It sends that provider’s upstream calls, and no other provider’s, through a proxy. It takes an http://, https://, socks5:// or socks5h:// URL.
A URL that carries credentials (user:pass@host) is not accepted. To use an authenticated proxy, put the whole URL in an environment variable and enter a whole-value reference such as ${PROXY_URL}, so the credentials never reach a configuration file or a database row. The proxy address is checked against the egress policy like the base URL.
Deleting a provider#
Delete provider asks first and names the provider. The dialog also lists what still points at it: the routes that target it and the provider groups it belongs to. A route or group that would be left with nothing is marked only target or only member. While that list is loading, or if it could not be read, the dialog says so instead of claiming nothing uses the provider.
The control plane refuses to delete a provider that a route target or a group member still references: DELETE /api/v1/providers/{id} answers 409 naming each route and provider group that uses it, and the provider is left in place, so take it out of those first. A provider nothing references can still be addressed directly as provider-slug/model, and stops answering there once it is deleted.