> ## Documentation index > Fetch the complete documentation index at: https://rolter-ai-rolter.docs7.io/llms.txt > Ask it with GET https://rolter-ai-rolter.docs7.io/ask?q=. Short keywords rank best. > Use these to discover all available pages before exploring further. # Providers: Upstream Endpoints and Their Credentials > How Rolter reaches an upstream, where the credential lives, and how to check the pair actually works before a request depends on it. A provider is one upstream endpoint plus the credential Rolter uses to reach it. That credential is the _provider key_ — the one the provider issued to Rolter, never the virtual key a client sends ([which key do I need?](/security/which-key)). Routes and provider groups select _targets_, and every target names a provider. Add them from **Providers** in the dashboard, or through the management API. Every column of the list sorts: click a heading for ascending, again for descending, and a third time to go back to the order the control plane sent. The **Credential** column says where each provider's key comes from: **Sealed** for a key stored in the control plane, the name of the environment variable when the provider names one instead, and **None** when it has neither. A stored key wins over a variable the provider also names. The column shows the source only, never the key; the same fact is the `has_stored_key` field in the [provider API](/api/providers). The name is fixed once the provider is created, and the form says so. The slug, which is the `provider-slug/model` address, is fixed too. Leave the slug blank and its placeholder shows the one the name will give. The **Kind** picker lists each provider by display name with a line on what it is (_OpenAI-compatible: vLLM, TGI, LM Studio or any server that speaks the OpenAI API_); the stored id sits under it. Below **API base** the form previews the address requests go to, and that depends on the kind: `/v1/messages` for Anthropic, an embeddings endpoint for Hugging Face TEI and [Infinity](/configuration/infinity), `/v1/systemone` for a [System One decision server](/configuration/system-one), `/v1/chat/completions` for most of the rest. The hint under **Provider key** says which header the key is sent in. --- ## Where the credential lives There are two places, and they resolve in this order: 1. **A stored key** — typed into the dashboard, sealed with AES-256-GCM under `ROLTER_KEK` before it reaches the database, and never returned by the API again. 2. **An environment variable** — `api_key_env` names a variable that is read from the environment of each process that needs the key. The value never enters the database at all. The control plane's own secrets cannot be named: `ROLTER_KEK`, `ROLTER_DATABASE_URL` and the like, any other name starting with `ROLTER_`, and the database and cache connection variables, are refused with a `400` (`invalid_field` on `api_key_env`) when you save the provider and when you test it, because the connection test sends the value to the provider's base URL. A stored key wins when both are set. An environment variable has to be set in **both** processes. The gateway reads it for every request it forwards, and the control plane reads it when you press **Test connection**, so a gateway that has the variable and a control plane that does not passes traffic and fails its test with `credential: env (unset)`. The dashboard's field hint says the same. Storing a key requires `ROLTER_KEK` on the control plane. Gateways do not need it: the control plane unseals keys when it builds the snapshot, and gateways receive them over the token-guarded `/internal/snapshot`, so keep the KEK off them. With it unset, the control plane still starts and the provider row still saves — the key simply cannot be sealed or read back. Run `rolter check` before boot; it fails on exactly this. --- ## Test connection Configuring a provider is otherwise a write with no feedback. The row saves whether or not the base URL is right, and the first sign of a problem is a failed request through the gateway, attributed to whichever route happened to select it. **Test connection** on a saved provider closes that loop. In the dashboard it is one click from the create: **Create provider** leaves the sheet open on the new provider, says it is not tested yet, and puts focus on **Test connection** beside the save buttons. The test does not run by itself, since it is a call to the upstream. **Done** closes the sheet. The same call is available to a script: ```bash curl -X POST https://control.example.com/api/v1/providers/$ID/test ``` ```json { "reachable": true, "probed_url": "https://api.openai.com/v1/models", "status": 200, "latency_ms": 142, "credential": "stored", "models_found": 38, "error": null } ``` It issues a `GET` against the provider kind's free model-list endpoint — the same one the gateway's health sweep uses, so a green test means the sweep will agree. It is a catalogue call, not a completion, so it costs nothing to run. The key is presented the way the provider's own API reads it, exactly as when the gateway forwards a request: `x-api-key` for Anthropic, `api-key` for Azure OpenAI, `x-goog-api-key` for native Gemini and a bearer token for the rest. A `401` from the test therefore means the upstream rejected the key, not that rolter sent it in a form the API ignores. It checks the provider, not a model. `models_found` is a count, so a green test does not say that the upstream model name a route sends is among them, and the route sheet has no connection check of its own until the probe can answer that ([#2008](https://github.com/rolter-ai/rolter/issues/2008)). ### The models a provider lists The ids behind that count have a read of their own, which is what the route sheet's **Upstream model** field suggests from: ```bash curl https://control.example.com/api/v1/providers/$ID/models ``` ```json { "models": ["gpt-4o", "gpt-4o-mini"] } ``` The list is sorted, has no duplicates and stops at 2000 ids. They are the ids the gateway lists under `provider-slug/model`. An upstream that is down, rejects the credential or answers with something other than a catalogue gives `{"models": []}` rather than an error, since the list only ever suggests; **Test connection** is the call that says why. It reads the same free model-list endpoint with the provider's own credential and under the same [egress policy](/configuration/egress) as the test. It sends the provider's credential to the upstream just as the test does, authenticated the same way, so it takes the same permission as the test, `provider:update` in the provider's scope, and a role that cannot test a provider cannot list its models either. A provider the caller may not see is answered `404`. The dashboard's route sheet shows no suggestions on a refusal and still takes any model id you type. ### Reading the result | Field | What it tells you | | -------------- | --------------------------------------------------------------------------------------------------------- | | `probed_url` | the URL actually tried. A doubled `/v1` is the most common misconfiguration and is invisible without this | | `credential` | how the key was resolved — `stored`, `env`, or `none`. Separates "wrong key" from "no key configured" | | `models_found` | how many models the upstream listed. A provider can be reachable and still serve nothing | | `status` | absent when the request never completed at all (DNS, TLS, connection refused, timeout) | A `401` names which credential source was used, so a rejected key points at the right place to fix it. When that source is `env (unset)`, the message names the variable and the process that is missing it: ```json { "reachable": false, "credential": "env (unset)", "error": "401: the upstream rejected the request, and the key's environment variable OPENAI_API_KEY is not set in the control plane's environment (resolved from: env (unset)). This test runs in the control plane, so set OPENAI_API_KEY there; the gateway reads it from its own environment, so set it there too." } ``` The test says where it ran. It cannot see the gateway's environment, so a green test does not prove the gateway has the variable. ### When something answers, but not with a model list `reachable` is not "the upstream returned 2xx". For provider kinds whose probe is a model catalogue, it means the response **was** a catalogue. A catch-all route, an SPA index, a reverse proxy or a login portal will all answer `200` at an arbitrary URL, and reporting those as reachable moves you on to debugging the wrong layer. That case is reported as its own thing — a 2xx status with `reachable: false`: ```json { "reachable": false, "probed_url": "https://gpustack.example.com/v1/v1/models", "status": 200, "models_found": null, "error": "200: https://gpustack.example.com/v1/v1/models answered, but not with a model list. Something other than the provider's API is serving that URL — check api_base for a duplicated path segment, a catch-all route or a login portal." } ``` The usual cause is a duplicated path segment in `api_base` — see [the `/v1` rule](#where-the-credential-lives) — where the upstream's own catch-all answers `200` at the doubled path. The dashboard renders this as its own amber state — _Answered, but not with a model list_ — rather than the red used for a refused connection or a rejected key. The host is up, and it is the URL or the service behind it that is wrong, which is a different thing to go and fix. A route whose targets are all Hugging Face TEI or Infinity servers answers `POST /v1/embeddings` (and `POST /v1/rerank` when one of them is Infinity), and refuses chat completions, messages, responses, images and audio, and rerank on TEI alone, with `400 unsupported_endpoint` without calling the server, which has no such endpoint. Kinds whose probe is a liveness endpoint rather than a catalogue keep 2xx as the criterion. Hugging Face TEI is probed at `/health`, which has no catalogue to return, so a bare `200` there is the whole verdict and `models_found` stays `null`. The probe reads the **stored** row, not what is currently typed in the form. With unsaved edits in the sheet, **Test connection** is off and says it tests the saved provider; save first, then test. The credential is sealed server-side and never leaves the control plane, so a draft key could not be tested without transmitting it somewhere first. ### When the credential cannot be read If the stored key was sealed under a different `ROLTER_KEK` — or the KEK is unset — the test reports that directly and **does not contact the upstream**: ```json { "reachable": false, "credential": "stored (KEK unset)", "error": "the stored credential for 'openai-primary' could not be read: ROLTER_KEK is unset or does not match the key it was sealed with. The upstream was not contacted." } ``` This is a deployment problem, not a provider problem, and the message says so rather than reporting the upstream as down. ### What a green result does not prove The probe is a model-list call. A few self-hosted runtimes serve their catalogue without checking authorization, so on those a wrong key can still return `reachable: true`. The hosted providers all authenticate the endpoint, so the check is meaningful there. Treat `reachable` as "the provider's API answered with its catalogue", and `401` as authoritative when it appears. --- ## Egress policy The base URL is checked against the egress policy when the provider is created, and **again** on every test. A stored row is not a standing permission to egress: if the policy has been tightened since, the test fails on the policy rather than reaching the host. --- ## Egress proxy A provider's egress proxy is optional. It sends that provider's upstream calls, and no other provider's, through a proxy. It takes an `http://`, `https://`, `socks5://` or `socks5h://` URL. A URL that carries credentials (`user:pass@host`) is not accepted. To use an authenticated proxy, put the whole URL in an environment variable and enter a whole-value reference such as `${PROXY_URL}`, so the credentials never reach a configuration file or a database row. The proxy address is checked against the [egress policy](#egress-policy) like the base URL. --- ## Deleting a provider **Delete provider** asks first and names the provider. The dialog also lists what still points at it: the routes that target it and the provider groups it belongs to. A route or group that would be left with nothing is marked **only target** or **only member**. While that list is loading, or if it could not be read, the dialog says so instead of claiming nothing uses the provider. The control plane refuses to delete a provider that a route target or a group member still references: `DELETE /api/v1/providers/{id}` answers `409` naming each route and provider group that uses it, and the provider is left in place, so take it out of those first. A provider nothing references can still be addressed directly as `provider-slug/model`, and stops answering there once it is deleted.