> ## Documentation index
> Fetch the complete documentation index at: https://rolter-ai-rolter.docs7.io/llms.txt
> Ask it with GET https://rolter-ai-rolter.docs7.io/ask?q=<keywords or question>. Short keywords rank best.
> Use these to discover all available pages before exploring further.

# Rolter Config File Reference: rolter.toml Explained

> Complete reference for rolter.toml: every table, its keys and defaults, and which tables a database-backed control plane owns instead of the file.

Rolter reads its configuration from a TOML file named `rolter.toml` by default. The repository ships a fully annotated example file you can copy as a starting point. Once you have the file, open it in your editor and work through each section before you start the gateway.

```bash
cp rolter.example.toml rolter.toml
```

---

## Unknown keys are ignored — and reported

Rolter never rejects a `rolter.toml` over a key it does not recognise. That is what keeps the file compatible in both directions: a config written for a newer release still loads on an older binary, so rolling a release back is not an outage.

The trade-off is that a typo would otherwise be silent. `connect_sesc` under `[timeouts]` produces no error — the default simply applies, and a misspelled `api_key_env` leaves a provider with no credential until the first upstream `401`.

So rolter names the key instead, without changing what it accepts. **Every process that reads the file reports it**, as a warning in the startup log:

- `rolter gateway --config rolter.toml`
- `rolter control --config rolter.toml`
- `rolter easy-up`
- `rolter-seed --import rolter.toml`, before it writes anything to the database

```
WARN rolter_core::config_lint: unrecognised config key: `providers[0].api_key_evn` is not a
     key rolter reads. It is ignored rather than rejected, so the setting it looks like it
     configures is silently at its default. Did you mean `api_key_env`?
     config=rolter.toml key=providers[0].api_key_evn
```

`rolter check` reports the same keys without starting anything, which is the form to run before a deploy:

```bash
rolter check --config rolter.toml
```

```
warn   unrecognised config key providers[0].api_key_evn
       `providers[0].api_key_evn` is not a key rolter reads. It is ignored rather
       than rejected, so the setting it looks like it configures is silently at
       its default. Did you mean `api_key_env`?
```

Either way you get the full path to every unrecognised key, including keys nested inside tables and inside arrays of tables, plus the nearest real key when one is close enough to be a likely typo.

`rolter check --config` also runs the validation the gateway runs at startup, and reports each problem it finds as an error: a route on an unknown provider, a duplicate route model, a target with weight `0`, a budget limit that is not positive, a route whose every target is a [fallback target](/reliability/fallback-targets), and every other setting the gateway would refuse to serve. A file that passes `rolter check` is one the gateway accepts.

Unknown keys are always warnings. The file is valid, the gateway starts, and an import still runs — an unknown key never stops a process. The one exception is deliberate: `--strict` turns them into a non-zero exit, which is what you want in CI when a config change is under review.

```bash
rolter check --config rolter.toml --strict
```

<Note>
  A config served from the database (the control plane's snapshot, which is what `rolter easy-up --database-url ...` and any DB-backed deployment runs on) has no unknown keys to report: it is built from validated rows, not from a file you typed.
</Note>

---

### Which settings a control plane owns

`rolter.toml` is a gateway's whole configuration only when the gateway reads it itself (`rolter gateway --config rolter.toml`, with no `--snapshot-url`). A gateway that polls a control plane applies that control plane's snapshot instead, and a control plane with a database builds the snapshot from the database. For the tables below the database is then the source of truth: what an admin saves on the dashboard reaches every gateway, and the table in `rolter.toml` is not what the snapshot carries.

| Table in `rolter.toml`                                     | Edited in the dashboard under | Page                                                  |
| ---------------------------------------------------------- | ----------------------------- | ----------------------------------------------------- |
| `[compatibility]`                                          | Settings → Compatibility      | [Compatibility](/configuration/compatibility)         |
| `[client]`                                                 | Settings → Client Settings    | [Client settings](/configuration/client-settings)     |
| `[model_defaults]`                                         | Settings → Model Settings     | [Model settings](/configuration/model-settings)       |
| `[adaptive_routing]`                                       | Adaptive Routing → Settings   | [Adaptive routing](/concepts/adaptive-routing)        |
| `[logging]`: `sample_rate`, `payload_capture`, `ui_events` | Observability → Logs Settings | [Log settings](/observability/log-settings)           |
| `[retry]`, `[timeouts]`, `[queue]`                         | Settings → Performance Tuning | [Performance tuning](/reliability/performance-tuning) |
| `[feature_flags]`                                          | Settings → Feature Flags      | [Feature flags](/configuration/feature-flags)         |
| `[security]`                                               | the Security screen           |                                                       |

`rolter-seed --import` does not carry these tables into the database. The only parts of them it reads from the file are `[logging.payload_capture]` and `ui_events`, and only when the file writes them. So in a database-backed deployment a value written in `rolter.toml` changes nothing for these tables, and each of them takes effect from the file only for a gateway that reads it directly. The exception is a control plane that loads the file itself, described under the list. The sections for these tables repeat the caveat.

<Note>
  A control plane started with `--config` as well as a database (`rolter easy-up` runs it this way) starts its snapshot from the file and lets the database take over table by table. For every table in the list above except `[feature_flags]`, gateways get the file's table until something other than the defaults is saved on its screen, and the saved table from then on. Saving the defaults back over a file value that differs hands the file's value back, so clear a value like that by editing the file. `[feature_flags]` always comes from the database in that setup. In `[logging]`, `clickhouse_url`, `batch_max`, `flush_ms` and `queue_capacity` stay the file's either way, since the database holds no copy of them.
</Note>

Two tables work the other way round. `[currency]` and `[egress]` have no stored copy, so a control plane reads them from the file it was started with (`rolter control --config rolter.toml`), and it checks the URLs and prices written through its API against them.

---

### schema_version

A top-level key, written before the first table, that stamps which spelling of the config the file uses. You do not have to write it.

<ParamField path="schema_version" type="integer">
  The schema version the file was written for. The current version is `3`. A file without the key is read as the oldest version, which is right for every config written before the stamp existed. A file stamped with a version newer than the running build understands still loads unchanged, and the keys that build does not know are ignored. `rolter config export` writes the stamp, and `rolter check --migrations` reports a file that is behind. See [Upgrading](/deployment/upgrading#what-schema_version-is).
</ParamField>

```toml
schema_version = 3
```

---

### \[server\]

The `[server]` section controls how the gateway process binds and exposes its built-in endpoints.

<ParamField path="host" default="0.0.0.0" type="string">
  The address the gateway listens on. Use `127.0.0.1` to bind only to localhost.
</ParamField>

<ParamField path="port" default="4000" type="u16">
  The TCP port the gateway listens on.
</ParamField>

<ParamField path="metrics_path" default="/metrics" type="string">
  The path Rolter serves its Prometheus metrics on. Change this when another app or sidecar already owns `/metrics` behind the same reverse proxy. The value must start with `/` and must not collide with any built-in gateway route (`/healthz`, `/v1/chat/completions`, `/v1/completions`, `/v1/messages`, `/v1/models`).
</ParamField>

<ParamField path="key_pepper" type="string">
  A deployment-wide secret mixed into virtual-key digests. Keeping a pepper means a leaked digest is useless without it. Omit this field and set the `ROLTER_KEY_PEPPER` environment variable instead — that keeps the secret out of your config file entirely.
</ParamField>

<ParamField path="max_body_bytes" default="33554432" type="usize">
  Maximum accepted request body size in bytes (defaults to 32 MiB). Raises Axum's default 2 MiB limit so large prompts, base64 images, or audio uploads are not rejected. An oversized body gets an OpenAI-style HTTP 413 error.
</ParamField>

<ParamField path="require_auth" type="boolean">
  Enforces virtual-key authentication when the effective key set is empty. When `true`, requests are always denied if no keys exist; when `false`, keyless access is allowed; when omitted, managed gateways fail closed while static local configurations stay keyless for development.
</ParamField>

```toml
[server]
host = "0.0.0.0"
port = 4000
metrics_path = "/metrics"
max_body_bytes = 33554432
```

---

### \[tls\]

Outbound TLS trust for upstream providers. Rolter verifies HTTPS with the host's system trust store; this table adds private certificate authorities on top of it, which an air-gapped or self-hosted deployment with its own CA needs.

<ParamField path="ca_bundles" default="[]" type="list">
  Paths of PEM files whose certificates are added to the system trust roots for every provider's upstream TLS. Certificate-chain and hostname verification stay on. The `ROLTER_CA_BUNDLE` environment variable replaces this list with the one path it names, which suits a container that mounts a single bundle. A provider's own `ca_bundles` replaces the result for that provider, and `ca_bundles = []` on a provider trusts the system roots only. Every file is checked while the config loads, so a path that is missing, unreadable, empty or not valid PEM stops startup with an error that names it.
</ParamField>

```toml
[tls]
ca_bundles = ["/etc/rolter/ca/private-root.pem"]
```

---

### \[\[providers\]\]

Each `[[providers]]` block registers an upstream provider the gateway can forward to. You can define as many as you need; each one is identified by its unique `name`. The flat `[[providers]]` array is the deprecated spelling of the readonly tier; see "Config tiers: readonly and default" below for `[[providers.readonly]]` and `[[providers.default]]`.

<ParamField path="name" type="string" required>
  A unique identifier for this provider. Routes reference providers by this name.
</ParamField>

<ParamField path="slug" type="string">
  A stable, URL-safe identity for `provider-slug/model` addressing. When omitted, Rolter automatically derives a slug from `name`.
</ParamField>

<ParamField path="kind" type="string" required>
  The wire protocol this provider speaks. One of `openai`, `anthropic`, `openai_compatible`, `ollama`, `ollama_cloud`, `llama_cpp`, `openrouter`, `tei`, `azure_openai`, `bedrock`, `vertex`, `gemini`, `gemini_native`, `gemini_interactions`, `mistral`, `groq`, `xai`, `meta_llama_api`, `cohere`, `perplexity`, `together`, `fireworks`, `databricks`, `aleph_alpha`, `nebius`, `ovhcloud`, `scaleway`, `deepseek`, `qwen`, `zhipu`, `kimi`, `ernie`, `doubao`, `hunyuan`, `yi`, `minimax`, `baichuan`, `gigachat`, `yandex_gpt`, `cloud_ru`, `mts_ai`, `naver`, `upstage`, `rinna`, `rakuten`, `sarvam`, `krutrim`, `falcon`, `system_one`, or `infinity`. Use `openai_compatible` for vLLM, TGI, Ollama, or any other OpenAI-schema endpoint.
</ParamField>

<ParamField path="api_base" type="string" required>
  The provider's base URL with no trailing slash, e.g. `https://api.openai.com`.

**Whether to include `/v1` depends on the provider kind.** For openai-shaped kinds (`openai`, `openai_compatible`, `ollama`, `ollama_cloud`, `llama_cpp`, `anthropic`) Rolter appends the full `/v1/...` path itself, so the base must stop at the host: `https://api.openai.com`, not `https://api.openai.com/v1`. Including it produces `/v1/v1/chat/completions` and every request 404s.

For the remaining kinds — `mistral`, `groq`, `openrouter`, `azure_openai` and the other hosted OpenAI-compatible clouds — Rolter strips the `/v1` it was asked for, so the base has to supply one: `https://api.mistral.ai/v1`.

A base that would double the prefix is rejected on write rather than stored. The dashboard states the rule for the selected kind and previews the resolved upstream URL as you type, naming the endpoint that kind is called at: `/v1/messages` for `anthropic`, `/v1/embeddings` for `tei`, a `generateContent` path for `gemini_native`, `/v1/chat/completions` for the rest. `GET /api/v1/provider-kinds` returns the same rule, the endpoint and the header the key is sent in per kind if you are automating provider creation.

</ParamField>

<ParamField path="api_key_env" type="string">
  The name of an environment variable to read the API key from. Preferred over `api_key` because it keeps secrets out of your config file. The gateway reads it from its own environment for every request, and the control plane reads it from its own for **Test connection**, so set it in both when they run as separate processes. Written as the single `api_key_env` it is deprecated in favour of an `api_keys` entry with `env`; it keeps working in every file and Rolter logs a warning on load.
</ParamField>

<ParamField path="api_key" type="string">
  An inline API key. Use `api_key_env` instead wherever possible. Deprecated in favour of an `api_keys` entry with `key`; it keeps working in every file and Rolter logs a warning on load.
</ParamField>

<ParamField path="egress_proxy" type="string">
  An outbound proxy URL to route traffic through, e.g. `http://corp-proxy:3128`. Supports `http://`, `https://`, `socks5://`, and `socks5h://` schemes. Deprecated in favour of a one-element `egress_proxies`; it keeps working in every file and Rolter logs a warning on load.
</ParamField>

<ParamField path="egress_proxies" type="list">
  A pool of outbound proxy URLs that requests rotate through round-robin. When the list is empty, a legacy `egress_proxy` is used as a one-entry pool. Write an authenticated proxy as `${ENV_VAR}` so its credentials never appear in serialized configuration or database rows.
</ParamField>

<ParamField path="allow_custom_api_base" type="bool" default="false">
  Let this provider point at an `api_base` its kind would otherwise refuse. A hosted kind names one operator, so Rolter pins its base URL: an `openrouter` provider must be `https://openrouter.ai/api/v1`. That pin is what stops a config edit from sending an OpenRouter key to a host of the editor's choosing.

Set this when the endpoint really is somewhere else — a corporate egress proxy or internal gateway in front of the vendor, an endpoint that re-implements the vendor's API, or a local fake you use to exercise the dialect in tests. It relaxes the host pin and nothing else: the credential must still come from `api_key_env`, never an inline `api_key`.

`rolter check` reports every provider that sets it, and `rolter check --strict` fails on it, so the opt-out stays visible rather than becoming a copied default.

A provider created in the dashboard has the same opt-out: the provider sheet shows an **Allow a custom base URL** switch for a pinned kind, and the providers API takes `allow_custom_api_base`. A custom base without it is refused when the provider is saved. `rolter-seed --import` keeps the flag, and `rolter config export` writes it back out.

```toml
[[providers]]
name = "openrouter-via-proxy"
kind = "openrouter"
api_base = "https://llm-egress.corp.example/openrouter/v1"
api_key_env = "OPENROUTER_API_KEY"
allow_custom_api_base = true
```

</ParamField>

<ParamField path="project_scoped" type="bool" default="false">
  Keep this provider to one project. By default a provider is org-wide: every project of the organization can use it, which is how projects share fallbacks. With `project_scoped = true`, only virtual keys minted in the project can reach it, whether by a route or as `provider-slug/model`, and a key from another project gets `404 model_not_found`.

A file names no projects, so the flag means "the project this file is imported into": it takes effect through `rolter-seed --import` and `rolter config export` writes it back. A gateway that loads the file directly ignores it, because the keys in a file carry no project. It logs a startup warning naming each flagged provider or group, and `rolter check` reports the same as a warning; `rolter-seed --import` never warns. The same key exists on `[[provider_groups]]`. A project-scoped group may hold that project's providers and org-wide ones; an org-wide group may hold only org-wide providers, and the import stops with an error naming the group when it does not.

In the dashboard and API, the provider's `project_id` is the same scope. See [Project-scoped providers](/concepts/providers-and-routes#project-scoped-providers).

</ParamField>

<ParamField path="also_track_via_llm_call" type="bool" default="false">
  Make active health checks send a real `max_tokens = 1` completion to this provider instead of the free `/v1/models` liveness probe, so a healthy result proves end-to-end inference. **Costs a few tokens on every sweep** — leave it off unless you need inference-level health. Recorded as `source = llm_call` in `provider_health_events` and fed through the same failure/recovery thresholds as free probes. The request goes where a forwarded one would, in the provider kind's own format, and unlike the free probe it counts a `404`/`405` or a `401`/`403` answer as a failure. See [Health checks](/reliability/health-checks#what-counts-as-healthy-for-an-llm-call-check).
</ParamField>

<ParamField path="llm_probe_model" type="string">
  The upstream model id the `also_track_via_llm_call` completion targets (e.g. `gpt-4o-mini`). Required when the flag is on; otherwise the checker warns and falls back to the free probe.
</ParamField>

<ParamField path="status_page_url" type="string">
  An optional statuspage.io-style status JSON URL (e.g. `https://status.anthropic.com/api/v2/status.json`). When set, a background poll records the provider's public status as a secondary health signal.
</ParamField>

<ParamField path="ca_bundles" type="list">
  Provider-specific list of PEM CA bundle file paths. When set, overrides the global `[tls].ca_bundles` setting for tls verification against this provider.
</ParamField>

<ParamField path="role_profile" type="string">
  How this provider handles instruction roles: `openai` keeps `developer` and `system` messages distinct, `system_only` lowers `developer` to `system`, and `anthropic` sends top-level `system` blocks. When omitted, native OpenAI keeps both roles, Anthropic uses top-level system blocks, and OpenAI-compatible providers use `system_only`.
</ParamField>

<ParamField path="model_role_profiles" type="table">
  Per-upstream-model overrides of `role_profile`, keyed by the upstream model id, e.g. `model_role_profiles = { "my-model" = "openai" }`. Use it for a custom chat template that explicitly supports the `developer` role.
</ParamField>

#### \[\[providers.api_keys\]\]

Multiple weighted API keys for one provider. When present, this list takes precedence over the single `api_key`/`api_key_env` pair. Providers cap throughput per key, so rotating across keys multiplies your effective RPM/TPM and isolates a single revoked or throttled key from taking the whole provider down. On every request the gateway picks one key from the pool with probability proportional to its weight. A key that gets `429` or `401` upstream is parked on a cooldown and the request retries immediately on a sibling key; the target itself stays in rotation, and key cooldown trips are visible as `rolter_key_cooldowns_tripped_total` in `/metrics`.

<ParamField path="key" type="string">
  An inline API key. Use `env` instead wherever possible.
</ParamField>

<ParamField path="env" type="string">
  The name of an environment variable to read the API key from.
</ParamField>

<ParamField path="weight" default="1" type="integer">
  Relative selection share when balancing across keys.
</ParamField>

<Note>
  The Postgres-backed control plane stores one credential per provider. `rolter-seed --import` therefore keeps the **first** entry that names an `env` and logs a warning naming how many it dropped, and `rolter config export` writes that one back as `api_key_env`. Keep a multi-key pool in the file the gateway loads directly, not in a file you round-trip through the control plane.
</Note>

#### \[providers.kv_events\]

vLLM ZeroMQ KV-event subscription settings for the `precise_cache_aware` strategy. See [cache-aware routing](/concepts/cache-aware-routing).

<ParamField path="endpoint" type="string" required>
  Subscriber endpoint, e.g. `tcp://vllm-0:5557`.
</ParamField>

<ParamField path="topic" default="kv-events" type="string">
  ZeroMQ topic name.
</ParamField>

<ParamField path="max_blocks" default="1000000" type="integer">
  Maximum resident block identities kept.
</ParamField>

<ParamField path="stale_secs" default="30" type="integer">
  Telemetry older than this in seconds is ignored.
</ParamField>

#### \[providers.lmcache\]

LMCache controller signal settings for the `lmcache_aware` strategy. See [cache-aware routing](/concepts/cache-aware-routing).

<ParamField path="endpoint" type="string" required>
  HTTP URL returning `{ "occupancy": 0..1, "cache_available": bool }`.
</ParamField>

<ParamField path="refresh_secs" default="2" type="integer">
  Interval between occupancy refreshes in seconds.
</ParamField>

<ParamField path="stale_secs" default="10" type="integer">
  Telemetry older than this in seconds is ignored.
</ParamField>

```toml
[[providers]]
name = "openai"
kind = "openai"
api_base = "https://api.openai.com"
api_key_env = "OPENAI_API_KEY"

[[providers]]
name = "anthropic"
kind = "anthropic"
api_base = "https://api.anthropic.com"
api_key_env = "ANTHROPIC_API_KEY"

[[providers]]
name = "vllm-1"
kind = "openai_compatible"
api_base = "http://localhost:8001"
```

---

### \[\[provider_groups\]\]

A provider group lets a client send `group-slug/model` and have Rolter spread the request across several providers with a load-balancing strategy, without a `[[routes]]` entry per model. The slug shares the provider slug namespace, so one slug names at most one provider or group, and a provider wins a tie. A group with no members never resolves.

<ParamField path="name" type="string" required>
  The group's display name.
</ParamField>

<ParamField path="slug" type="string">
  A stable, URL-safe identity for `group-slug/model` addressing. When omitted, Rolter derives one from `name`.
</ParamField>

<ParamField path="strategy" default="round_robin" type="string">
  How requests are spread across the members. Accepts the same values as a route's `strategy`.
</ParamField>

<ParamField path="members" type="array of tables">
  The providers the group spreads requests across, written as `[[provider_groups.members]]` entries. See below.
</ParamField>

<ParamField path="project_scoped" type="boolean" default="false">
  Keep this group to one project, exactly as for a provider. A project-scoped group may hold that project's providers and org-wide ones; an org-wide group may hold only org-wide providers, and `rolter-seed --import` stops with an error naming the group when it does not. As on a provider, the flag means "the project this file is imported into", and a gateway that reads the file directly ignores it.
</ParamField>

#### \[\[provider_groups.members\]\]

<ParamField path="provider" type="string" required>
  The `name` of a provider defined in `[[providers]]`.
</ParamField>

<ParamField path="model" type="string">
  The upstream model id to send. When omitted, the model the client wrote after `group-slug/` is forwarded as-is.
</ParamField>

<ParamField path="weight" default="1" type="integer">
  The relative traffic weight for the `weighted` and `pipeline` strategies. Ignored by other strategies.
</ParamField>

```toml
[[provider_groups]]
name = "vLLM cluster"
slug = "vllm-cluster"
strategy = "round_robin"

[[provider_groups.members]]
provider = "vllm-1"

[[provider_groups.members]]
provider = "vllm-2"
model = "meta-llama/Llama-3.1-8B-Instruct"
weight = 2
```

A client then asks for the model `vllm-cluster/meta-llama/Llama-3.1-8B-Instruct`.

---

### Config tiers: readonly and default

`[[providers]]`, `[[provider_groups]]` and the model presets under `[models]` each come in two tiers, so one file can both pin a row and merely seed one (ADR-0022). The tiers are written `[[providers.readonly]]` and `[[providers.default]]`, `[[provider_groups.readonly]]` and `[[provider_groups.default]]`, and `[[models.readonly]]` and `[[models.default]]`.

| Tier       | Owned by                  | Behaviour                                                                                                                                                                                                                                                                                                                                         |
| ---------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `readonly` | the file                  | Served as written and immutable at runtime: the CRUD API and the dashboard refuse to edit or delete it. A readonly model joins `[[routes]]`, so its name must not repeat a route's.                                                                                                                                                               |
| `default`  | the database, once seeded | A control plane with a database inserts each entry into the default project of the `default` organization when it starts, unless the row exists or its name or slug is already in use, and never overwrites it afterwards. From then on it is edited like any row created on the dashboard, and one an admin deletes returns at the next restart. |

The bare `[[providers]]` and `[[provider_groups]]` arrays are the deprecated spelling of the readonly tier: every entry in them is readonly, and they cannot express a default. `rolter config export` writes the tiered spelling.

A `default` provider or group whose slug matches a readonly one, and a `default` model whose name matches a route, is a config error at load, since the readonly row is immutable and would shadow the seeded one.

Only a control plane that has a database and was started with this file (`rolter control --config`) acts on the default tier. It seeds into the `default` organization that `rolter-seed` creates, and without that organization it logs a warning and seeds nothing. A gateway that reads the file itself ignores the default tier.

---

### \[\[routes\]\]

Each `[[routes]]` block maps a public model name that clients request to one or more upstream targets and a load-balancing strategy.

<ParamField path="model" type="string" required>
  The public model name clients send in their request body, e.g. `gpt-4o`. Must be unique across all routes.
</ParamField>

<ParamField path="strategy" default="round_robin" type="string">
  The load-balancing strategy used to pick a target. One of:

| Value                 | Behaviour                                                                           |
| --------------------- | ----------------------------------------------------------------------------------- |
| `round_robin`         | Cycles through targets in order                                                     |
| `random`              | Picks a target uniformly at random                                                  |
| `power_of_two`        | Samples two targets and picks the one with fewer in-flight requests                 |
| `consistent_hash`     | Routes the same request consistently to the same target                             |
| `cache_aware`         | Steers toward the target most likely to have the request prefix cached              |
| `weighted`            | Smooth weighted round-robin proportional to each target's `weight`                  |
| `pipeline`            | Composable scorer pipeline: static weight, in-flight load and prefix-cache affinity |
| `cheapest`            | Lowest catalog price for the target's upstream model; load breaks ties              |
| `fastest`             | Lowest observed request latency; load breaks ties and carries cold routes           |
| `precise_cache_aware` | Exact resident-prefix score from vLLM KV-cache events; load fallback                |
| `lmcache_aware`       | Cache availability and free-capacity score from LMCache controller occupancy        |
| `adaptive`            | Weighted blend of latency, cost and load, governed by `[adaptive_routing]`          |
| `lora_aware`          | Prefers a target that already holds the requested LoRA adapter                      |
| `predicted_latency`   | Ranks by what this request is modelled to cost, from queue depth and prompt size    |

See [Load balancing](/concepts/load-balancing) for when to pick each.

</ParamField>

#### \[\[routes.targets\]\]

<ParamField path="provider" type="string" required>
  The `name` of a provider defined in `[[providers]]`.
</ParamField>

<ParamField path="model" type="string">
  The upstream model ID to send to the provider. When omitted, the requested model name is forwarded as-is.
</ParamField>

<ParamField path="weight" default="1" type="u32">
  The relative traffic weight for `weighted` and `pipeline` strategies. Ignored by other strategies.
</ParamField>

<ParamField path="takes_over" default="[]" type="string[]">
  The failures this target is a fallback for: any of `throttled`, `timeout`, `transport`, `server_error`, `refusal` and `refusal:<category>`. A target with classes is never dialled first and takes no ordinary traffic; it is tried only after another target failed with one of them. Empty, the default, makes it an ordinary member of the pool. A route whose every target sets it is rejected (`all_targets_restricted`). See [Fallback targets](/reliability/fallback-targets). Also valid on a variant's targets.
</ParamField>

#### \[routes.params\]

An optional table of admin-set default inference parameters injected into the request body. Any key accepted by the upstream provider is valid here (`temperature`, `max_tokens`, `stop`, etc.). A parameter you do not set here is passed through from the client untouched.

#### \[routes.param_policy\]

Controls whether clients may override the values set in `[routes.params]`.

<ParamField path="mode" default="allow" type="string">
  The baseline override policy. `allow` lets clients override any param that is not in `deny`. `deny` pins every param except those in `allow`. When an override is denied, the admin default silently wins.
</ParamField>

<ParamField path="allow" default="[]" type="list">
  Params clients may override when `mode = "deny"`.
</ParamField>

<ParamField path="deny" default="[]" type="list">
  Params clients may not override when `mode = "allow"`.
</ParamField>

#### \[\[routes.variants\]\]

Variants replace the top-level `targets`/`strategy` pool when you want A/B, canary, or traffic-split routing under a single model name. A request randomly samples one variant by weight (the primary) and falls back to the remaining variants in declared order if the primary fails. Within a variant, the route's `strategy` picks which target is tried first; the remaining targets follow in declared order.

<ParamField path="name" type="string" required>
  A stable identifier for this variant. Appears in request logs under the `variant` column for attribution and A/B analysis.
</ParamField>

<ParamField path="weight" default="1" type="u32">
  The relative traffic share for this variant in the primary draw.
</ParamField>

<ParamField path="targets" type="array" required>
  Ordered list of upstream targets tried within this variant. Same shape as `[[routes.targets]]`.
</ParamField>

<ParamField path="params" type="table">
  Variant-scoped default inference params layered over `[routes.params]`. The variant value wins on any key collision.
</ParamField>

```toml
# Simple round-robin across a single provider
[[routes]]
model = "gpt-4o"
strategy = "round_robin"
[[routes.targets]]
provider = "openai"
model = "gpt-4o"

# Cache-aware routing across a self-hosted vLLM pool
[[routes]]
model = "llama"
strategy = "cache_aware"
[[routes.targets]]
provider = "vllm-1"
model = "meta-llama/Llama-3.1-8B-Instruct"
[[routes.targets]]
provider = "vllm-2"
model = "meta-llama/Llama-3.1-8B-Instruct"

# Weighted routing: 3× traffic to vllm-1
[[routes]]
model = "llama-weighted"
strategy = "weighted"
[[routes.targets]]
provider = "vllm-1"
model = "meta-llama/Llama-3.1-8B-Instruct"
weight = 3
[[routes.targets]]
provider = "vllm-2"
model = "meta-llama/Llama-3.1-8B-Instruct"
weight = 1

# Admin-pinned params with a partial caller-override allowance
[[routes]]
model = "gpt-4o-strict"
strategy = "round_robin"
[routes.params]
temperature = 0.0
max_tokens = 1024
[routes.param_policy]
mode = "deny"
allow = ["max_tokens"]   # callers may still lower max_tokens
[[routes.targets]]
provider = "openai"
model = "gpt-4o"
```

#### \[routes.advanced\]

Catalog metadata and model-scoped policy for one route. The dashboard's model sheet writes it; [Per-model advanced settings](/concepts/providers-and-routes#per-model-advanced-settings) explains what each part means and the ranges the control plane accepts.

<ParamField path="model_type" type="string">
  The catalog family of the model, such as `chat` or `embedding`. A label the dashboard reads to choose which parameters and prices its model sheet offers, and `GET /v1/models` serves on the route's entry. The gateway does not enforce it.
</ParamField>

<ParamField path="capabilities" default="[]" type="list">
  Capability labels recorded for the model, such as `vision`. A label for the dashboard and for `GET /v1/models`: the gateway forwards a request whether or not a flag is set.
</ParamField>

<ParamField path="context_length" type="u32">
  The model's context window in tokens, at least `1`. Served on `GET /v1/models` as `context_length`; advertised, not enforced. Unset by default.
</ParamField>

<ParamField path="description" type="string">
  A note about the model for other operators. The dashboard and the routes API return it; it is never sent to API clients and never listed on `GET /v1/models`.
</ParamField>

<ParamField path="headers" type="table">
  Headers sent to the upstream on every call this route makes, none by default. A name must be a valid header name and a value visible ASCII on one line. `authorization`, `x-api-key`, `api-key`, `x-goog-api-key` and the headers that frame a request (`host`, `content-length`, `content-type`) are refused, because credentials belong on the provider. The routes API answers `400` for one, `rolter check` fails a file that sets one, and the gateway leaves it out and logs a config problem at startup.
</ParamField>

<ParamField path="locked_headers" default="[]" type="list">
  Names from `headers` whose value replaces a caller's header of the same name. A caller's header reaches the upstream only if `[client]` `forwarded_headers` lists it. An unlocked route header is a default that a forwarded caller header replaces.
</ParamField>

<ParamField path="guardrails" type="table">
  Per-route selection of guardrail rules, written as `[routes.advanced.guardrails]`. See below.
</ParamField>

##### \[routes.advanced.limits\]

Per-route request limits. An omitted key inherits the gateway-wide default, except `rpm`, `tpm` and `concurrency` where omitted means no cap. See [Route limits](/concepts/providers-and-routes#route-limits). A request over an `rpm`, `tpm` or `concurrency` cap is refused with `429` and `Retry-After`. Without `redis_url`, `rpm` and `tpm` are not enforced and `concurrency` still is. A zero fails config validation, and a stored row that carries one is served without that cap and reported by `GET /api/v1/config/problems`.

<ParamField path="timeout_secs" type="integer">
  Response headers timeout in seconds (`1` and up); replaces `[timeouts]` `request_secs` for this route. A streamed body is not cut short.
</ParamField>

<ParamField path="retries" type="integer">
  Extra attempts after the first (`0` to `10`); replaces `[retry]` `max_retries` for this route. `0` turns retries off.
</ParamField>

<ParamField path="output_tokens" type="integer">
  Refuses a request whose `max_tokens`, `max_completion_tokens` or `max_output_tokens` is above this limit.
</ParamField>

<ParamField path="rpm" type="integer">
  Requests per minute cap across all gateway replicas via Redis (`1` and up).
</ParamField>

<ParamField path="tpm" type="integer">
  Tokens per minute cap across all gateway replicas via Redis (`1` and up).
</ParamField>

<ParamField path="concurrency" type="integer">
  Maximum concurrent requests in flight per gateway replica (`1` to `10000000`). Works without Redis.
</ParamField>

##### \[routes.advanced.visibility\]

Who may see and call the model. Keys from a config file carry no organization and are not narrowed.

<ParamField path="minimum_role" type="string">
  Minimum role required to access or see the model.
</ParamField>

<ParamField path="allowed_team_ids" default="[]" type="list">
  Team IDs allowed to access the route.
</ParamField>

<ParamField path="allowed_key_ids" default="[]" type="list">
  Virtual key IDs allowed to access the route.
</ParamField>

<ParamField path="allowed_user_ids" default="[]" type="list">
  User IDs allowed to access the route.
</ParamField>

<ParamField path="project_only" default="false" type="boolean">
  When `true`, serves the route only to keys minted in its own project.
</ParamField>

<Note>
  Earlier releases also accepted `base_url`, `insecure_tls`, `[routes.advanced.pricing]` and a `context_window` limit, and stored them, but the gateway never read them ([#2924](https://github.com/rolter-ai/rolter/issues/2924)). They are gone, and a file that still sets one loads with the key ignored and reported as unknown. A second endpoint for a model is a second provider with its own `api_base`; to trust a private certificate authority use `[tls]` `ca_bundles` or the provider's `ca_bundles`; token rates belong in `[[model_prices]]`. The `rpm`, `tpm` and `concurrency` limits went the same way and came back with a reader ([#2936](https://github.com/rolter-ai/rolter/issues/2936)); a limit on a caller rather than a model is set under `[[rate_limits]]`.
</Note>

##### \[routes.advanced.guardrails\]

Switches rules of the global [`[guardrails]`](/security/guardrails) set on or off for one route. Rules are named, so adding a rule globally still reaches every route that has not opted out of it by name. A name that matches no configured rule fails config validation instead of being ignored, because a typo in `disable` would otherwise read as "this rule is off here" while the rule kept running.

<ParamField path="disable" default="[]" type="list">
  Rules that do not apply on this route.
</ParamField>

<ParamField path="enable" default="[]" type="list">
  Rules that apply on this route. Wins over `disable` when a rule is in both.
</ParamField>

```toml
[[routes]]
model = "internal-support-bot"

[routes.advanced.guardrails]
disable = ["email"]   # this route legitimately handles addresses
enable  = ["card"]
```

#### \[routes.cache\]

Opts a route into the exact-match response cache. A route caches only when this table is present with `enabled = true` and the global `[cache]` switch is on; a virtual key's `cache` setting can override the route. See [Response cache](/concepts/response-cache).

<ParamField path="enabled" default="true" type="boolean">
  Whether this route caches. Writing the table opts the route in; set `false` to opt it out again.
</ParamField>

<ParamField path="ttl_secs" type="integer">
  Entry lifetime in seconds for this route. When omitted, `default_ttl_secs` from `[cache]` applies.
</ParamField>

<ParamField path="per_key" default="false" type="boolean">
  Isolate cached entries by virtual key so responses are not shared across keys. When `false`, every caller of the route shares the cache.
</ParamField>

<ParamField path="semantic" type="table">
  Adds a semantic lookup after the exact-match one, written as `[routes.cache.semantic]`. See below.
</ParamField>

##### \[routes.cache.semantic\]

A semantic hit replays a reply only to a request that matches it exactly on everything but the user and assistant wording. Multimodal and tool-call conversations skip the semantic lookup.

<ParamField path="provider" type="string" required>
  The `name` of the configured provider used for `/v1/embeddings`. It reuses that provider's authentication, proxy and CA policy.
</ParamField>

<ParamField path="model" type="string" required>
  The embeddings model sent upstream.
</ParamField>

<ParamField path="threshold" default="0.92" type="float">
  The minimum cosine similarity for a hit.
</ParamField>

<ParamField path="max_candidates" default="256" type="integer">
  How many of the most recent semantic entries are compared per request, which bounds the cost of a lookup.
</ParamField>

```toml
[[routes]]
model = "gpt-4o"

[routes.cache]
enabled = true
ttl_secs = 600
per_key = false

[routes.cache.semantic]
provider = "openai"
model = "text-embedding-3-small"
threshold = 0.92
max_candidates = 256
```

---

### \[models\]

Model presets that behave like routes but come in the two tiers described under "Config tiers" above. Each entry takes the same keys as a `[[routes]]` entry, including its `[[models.readonly.targets]]` tables.

<ParamField path="readonly" default="[]" type="array of tables">
  Routes served directly from the file and rejected by the CRUD API, written as `[[models.readonly]]`. A readonly model is added to `[[routes]]`, and its `model` must not repeat the name of a route.
</ParamField>

<ParamField path="default" default="[]" type="array of tables">
  Routes seeded once into the default project of a database-backed control plane, written as `[[models.default]]`, and then owned and edited in the dashboard. A `model` that conflicts with a readonly model or a route is a config error.
</ParamField>

```toml
[[models.readonly]]
model = "company-safe"
targets = [{ provider = "openai", model = "gpt-4o-mini" }]

[[models.default]]
model = "starter-chat"
targets = [{ provider = "openai", model = "gpt-4o-mini" }]
```

---

### \[\[virtual_keys\]\]

Virtual keys are the bearer tokens clients present to the gateway. Remove the entire `[[virtual_keys]]` section to run in open (unauthenticated) mode, which is suitable only for local development.

<ParamField path="key" type="string" required>
  The bearer token value. Clients send this in the `Authorization: Bearer <key>` header.
</ParamField>

<ParamField path="name" type="string">
  A human-readable label for this key, used in logs and the dashboard.
</ParamField>

<ParamField path="models" default="[]" type="list">
  Model allow-list. An empty list grants access to all models. When non-empty, requests for models not on the list are rejected.
</ParamField>

<ParamField path="providers" default="[]" type="list">
  Upstream provider allow-list. An empty list allows selecting any provider on an allowed route. When non-empty, requests authenticated with this key can only route to the named providers.
</ParamField>

<ParamField path="disabled" default="false" type="boolean">
  Set to `true` to revoke the key without deleting it from the config file. Useful for auditing.
</ParamField>

<ParamField path="expires_at" type="string">
  An RFC3339 expiry timestamp. The key stops authenticating at or after this instant, e.g. `"2027-01-01T00:00:00Z"`.
</ParamField>

<ParamField path="cache" type="boolean">
  Per-key response-cache override, independent of a route's own opt-in. Omit to inherit the route decision, `false` to force responses to bypass the cache, or `true` to cache responses even on a route that did not opt in. The global `[cache]` switch is still required.
</ParamField>

```toml
[[virtual_keys]]
key = "sk-rolter-dev"
name = "local-dev"
models = []        # empty = all models allowed
providers = []     # empty = all providers allowed
disabled = false
# expires_at = "2027-01-01T00:00:00Z"
# cache = false
```

---

### \[\[db_virtual_keys\]\]

Not a `rolter.toml` table. Virtual keys created on the dashboard or through the API live in the database, and the control plane puts them in the snapshot as `db_virtual_keys`. Each entry carries the key's peppered digest, never the plaintext, with its id, the scope its usage is attributed to (organization, team, project, creator, business unit and customer) and the same `models`, `providers`, `disabled`, `expires_at` and `cache` settings as a `[[virtual_keys]]` entry. While a rotation's overlap is running (see [Virtual keys](/concepts/virtual-keys)), an entry also carries `previous`: the digest of the replaced secret and the `valid_until` instant up to which it still authenticates as the same key. A gateway matches a presented key against those digests alongside the keys of `[[virtual_keys]]`, which stay the file's own list. See [Virtual keys](/concepts/virtual-keys).

---

### \[\[mcp_servers\]\]

Not a `rolter.toml` table. MCP servers belong to an organization and are registered on the dashboard or through the management API; the control plane unseals their credentials and ships them in the snapshot. Each entry carries `id`, `org_id`, `slug`, `url`, `transport`, `required_scopes`, `auth_kind`, `auth_header_name`, `credential`, `connect_timeout_ms`, `request_timeout_ms`, `max_retries`, `tools` and `enforce_manifest_by_default`, which is true when the server's organization enforces manifests until it saves MCP settings of its own. With a database, the snapshot carries the control plane's own rows and replaces whatever a bootstrap file holds for this table. See [MCP credentials and transport settings](/configuration/mcp-servers) for what each field means.

---

### \[\[mcp_tool_groups\]\]

Not a `rolter.toml` table. Each entry is an enabled MCP tool group, which the control plane adds to the snapshot and the MCP proxy enforces: a server that at least one group names serves only the tools those groups list. Entries carry `id`, `org_id`, `slug`, `tools`, a list of `server_id` and `tool` pairs, and `assignments`, the `kind` (`virtual_key`, `project` or `team`) and `id` of each caller the group is assigned to, empty for a group that applies to the whole organization. Groups are managed on the dashboard or through the management API; see [MCP](/api/mcp#manage-the-registry).

---

### \[\[mcp_settings\]\]

Not a `rolter.toml` table. Each entry is one organization's saved MCP settings, which the MCP proxy applies per request: `org_id`, `connect_timeout_ms`, `request_timeout_ms`, `max_retries`, `fail_open` and `allow_unlisted_tools`. An organization that never saved any has no entry and keeps the deployment defaults. They are managed on the dashboard or through the management API; see [MCP](/api/mcp#manage-the-registry).

---

### \[\[mcp_oauth_sessions\]\]

Not a `rolter.toml` table. Each entry is a live OAuth session, one user's access token for one MCP server, that the control plane adds to the snapshot after the user completes the consent flow and drops when the session expires. The tokens cross only the protected `/internal/snapshot` channel and are never shown on the dashboard or through the public API. Entries carry `id`, `server_id`, `user_id`, `scopes`, `expires_at` and `access_token`. There is nothing to write by hand; see [MCP OAuth](/security/mcp-oauth) for the consent flow.

---

### \[\[model_prices\]\]

Define per-model token pricing so Rolter can compute a `cost_usd` value on every request log. Models without a price entry log a cost of `0`. Rates are in USD per million tokens, matching the convention used by major providers. Every rate must be zero or more: a negative rate is a config error, because it would turn a request into a credit against its budget.

<ParamField path="model" type="string" required>
  The public model name this price applies to. Must match a route's `model` field.
</ParamField>

<Note>
  Prices and budget limits are written as numbers but computed in exact decimal
  arithmetic, so a rate like `0.1` is a tenth rather than its nearest binary
  approximation. See [Budgets & Rate Limits](/security/budgets-and-rate-limits)
  for what that does and does not cover.
</Note>

<ParamField path="input_per_mtok" type="float">
  USD per million input (prompt) tokens.
</ParamField>

<ParamField path="output_per_mtok" type="float">
  USD per million output (completion) tokens.
</ParamField>

<ParamField path="cached_input_per_mtok" type="float">
  USD per million cache-hit input tokens. When omitted, falls back to `input_per_mtok`.
</ParamField>

<ParamField path="cache_write_per_mtok" type="float">
  USD per million input tokens written to the provider's prompt cache, which Anthropic bills above the input rate. When omitted, falls back to `input_per_mtok`. See [Cache writes](/observability/logs-and-cost#cache-writes).
</ParamField>

<ParamField path="cache_write_1h_per_mtok" type="float">
  USD per million input tokens written to the provider's 1 hour prompt cache, for a provider that reports the split and bills it apart from the 5 minute cache (Anthropic). When omitted, falls back to `cache_write_per_mtok`, and from there to `input_per_mtok`. See [Cache writes](/observability/logs-and-cost#cache-writes).
</ParamField>

<ParamField path="image_per_unit" type="float">
  Price of one generated image on `/v1/images/generations`, in `currency`, for a model the provider bills per image. The count is the length of the response's `data` array, or the request's `n` when the response has none. The cost is added to the token cost of the same request. When omitted, an image request that reports no tokens is logged `unpriced`. See [Images, audio and speech](/observability/logs-and-cost#images-audio-and-speech).
</ParamField>

<ParamField path="audio_input_per_minute" type="float">
  Price of one minute of input audio on `/v1/audio/transcriptions` and `/v1/audio/translations`, in `currency`. The minutes are the duration the response states: `duration` in a `verbose_json` answer, or a `usage` object of type `duration`. A response that states none is logged with `usage_unknown = 1` and no cost. When omitted, such a request that reports no tokens is logged `unpriced`.
</ParamField>

<ParamField path="speech_per_mchar" type="float">
  Price per million characters of text sent to `/v1/audio/speech`, in `currency`. The characters are counted from the request's `input`, because the audio that comes back states no length. When omitted, a speech request is logged `unpriced`.
</ParamField>

<ParamField path="audio_input_per_mtok" type="float">
  Price per million audio input tokens, for a realtime or audio model that bills the audio it is sent apart from the text. The share of the prompt that is audio is the one the response's usage names (see [Audio and text tokens](/observability/logs-and-cost#audio-and-text-tokens)). Defaults to `input_per_mtok`.
</ParamField>

<ParamField path="audio_output_per_mtok" type="float">
  Price per million audio output tokens. Defaults to `output_per_mtok`.
</ParamField>

<ParamField path="cached_audio_input_per_mtok" type="float">
  Price per million cached audio input tokens, for a provider that says how many of its cache hits were audio (a realtime session does). Defaults to `cached_input_per_mtok`, and from there to the audio input rate.
</ParamField>

<ParamField path="currency" default="USD" type="string">
  The currency the rates above are written in. It defaults to `USD`, not to the `base` of `[currency]`, so on a deployment whose base is another currency set it on every price or add a `USD` rate. A currency with no rate in `[currency]` is a config error.
</ParamField>

```toml
[[model_prices]]
model = "gpt-4o"
input_per_mtok = 2.5
output_per_mtok = 10.0
cached_input_per_mtok = 1.25
cache_write_per_mtok = 3.125
cache_write_1h_per_mtok = 5.0

# a model billed by the image, the minute or the character: token rates of 0,
# and the unit rate its provider charges
[[model_prices]]
model = "dall-e-3"
input_per_mtok = 0
output_per_mtok = 0
image_per_unit = 0.04
```

---

### \[currency\]

The settlement currency and a static rate table. Budgets and accumulated spend are denominated in `base`; a model priced in any other currency is converted through `rates` before it reaches a budget counter, so pricing in euros while budgeting in dollars is exact. Rates are written by hand, which keeps this working air-gapped.

The dashboard's currency chooser offers `base` followed by every code in `rates`, so adding a row here is the whole of making a currency selectable.

<ParamField path="base" default="USD" type="string">
  The currency budgets and spend are denominated in. Codes are open strings (ISO 4217 today), compared without regard to case or surrounding whitespace.
</ParamField>

<ParamField path="rates" type="table">
  How many units of `base` one unit of the keyed currency is worth: with a base of `USD`, `EUR = 1.09` means one euro costs 1.09 dollars. The base itself is always 1.0 and needs no row. Every rate must be positive, and the table is empty by default. A `[[model_prices]]` entry in a currency with no rate is a config error, because charging face value bills the wrong number and charging zero bills nothing.
</ParamField>

```toml
[currency]
base = "USD"
rates = { EUR = 1.09, GBP = 1.27, RUB = 0.011 }
```

<Note>
  The database has no currency table, so a control plane reads this one from the file it was started with (`rolter control --config rolter.toml`) and checks the currency of every price written through its API against it.
</Note>

---

### \[\[budgets\]\]

Spend caps block requests with HTTP `402` once a scope's cumulative cost reaches its limit for the current period. Budgets are enforced using Redis-tracked costs; without a Redis connection, budgets fail open. The most restrictive matching budget across the scope chain wins.

<Note>
  The `id` field matches the UUID of an entity defined in the database (org, team, project, or virtual key). Config-file virtual keys carry no scope IDs, so `[[budgets]]` entries in `rolter.toml` are most useful when combined with the full control-plane stack.
</Note>

<ParamField path="scope" type="string" required>
  The scope level this budget applies to. One of `org`, `team`, `project`, `key`, `business_unit`, `customer` or `user`. A `user` budget counts every key that person minted, whatever project it lives in.
</ParamField>

<ParamField path="id" type="string" required>
  The UUID of the scoped entity. In a file that `rolter-seed --import` reads, the entity's portable name instead: an org slug, `org/team`, `org/team/project`, or, for a `user` budget, the account's email ([Config export](/configuration/config-export#budgets-and-rate-limits)).
</ParamField>

<ParamField path="limit_usd" type="float" required>
  The spend cap in USD for the current window. Must be a positive number.
</ParamField>

<ParamField path="period" default="monthly" type="string">
  The window the budget resets on. One of `monthly`, `daily`, or `total`. `total` is a lifetime cap that never resets.
</ParamField>

<ParamField path="unpriced_policy" type="string">
  This budget's own answer to traffic the gateway cannot price: `ignore`, `warn` or `block`. Omit it to inherit the deployment-wide `unpriced_policy`. The most restrictive answer across the scope chain and the deployment setting wins, so a budget can tighten what the deployment asked for but never loosen it. See [Per-budget override](/security/budgets-and-rate-limits#per-budget-override).
</ParamField>

<ParamField path="within" type="table">
  Narrows a `user` budget to the keys the person minted in one org, `{ org = "org-uuid" }`, or one project, `{ project = "project-uuid" }`; in an import file the same places by slug path, `{ org = "acme" }` or `{ project = "acme/team/project" }`. Omit it to count all of their keys. Has no meaning on any other scope, and an import refuses it there.
</ParamField>

```toml
[[budgets]]
scope = "org"
id = "org-uuid"
limit_usd = 500.0
period = "monthly"
```

---

### unpriced_policy

A top-level key deciding what happens to traffic the gateway cannot price. A model with no `[[model_prices]]` entry accrues zero spend, so it can never exhaust a budget however much is served — without this setting, budgets fail open on exactly the models you added most recently.

Unlike budgets, this needs no Redis: being unpriced is a property of the config, not of a counter, so it is enforced even while budget counters are failing open.

<ParamField path="unpriced_policy" default="ignore" type="string">
  One of `ignore`, `warn`, or `block`. `ignore` serves the request and accrues nothing. `warn` serves it but logs the model once per hour. `block` refuses at admission with HTTP `402` and code `model_unpriced`, before any upstream call. Defaults to `ignore` so that upgrading cannot start refusing traffic on a deployment that has not priced its models yet.
</ParamField>

```toml
unpriced_policy = "block"
```

<Note>
  The built-in `fake-llm` model is served before any price lookup, so `block` never makes it unreachable.
</Note>

---

### \[\[rate_limits\]\]

Throughput caps reject excess requests with HTTP `429` and a `Retry-After` header. Limits use a Redis sliding-window counter; without Redis they fail open. At least one of `rpm` or `tpm` must be set — a limit with neither never blocks anything.

<ParamField path="scope" type="string" required>
  The scope level this limit applies to. One of `org`, `team`, `project`, `key`, `business_unit`, `customer` or `user`. A `user` limit counts every key that person minted.
</ParamField>

<ParamField path="id" type="string" required>
  The UUID of the scoped entity. In a file that `rolter-seed --import` reads, the entity's portable name instead: an org slug, `org/team`, `org/team/project`, or, for a `user` limit, the account's email ([Config export](/configuration/config-export#budgets-and-rate-limits)).
</ParamField>

<ParamField path="rpm" type="u32">
  Requests per minute cap. Omit to leave request rate uncapped.
</ParamField>

<ParamField path="tpm" type="u32">
  Tokens per minute cap (prompt \+ completion combined). Omit to leave token rate uncapped.
</ParamField>

<ParamField path="within" type="table">
  Narrows a `user` limit to the keys the person minted in one org, `{ org = "org-uuid" }`, or one project, `{ project = "project-uuid" }`; in an import file the same places by slug path, `{ org = "acme" }` or `{ project = "acme/team/project" }`. Omit it to count all of their keys. Has no meaning on any other scope, and an import refuses it there.
</ParamField>

```toml
[[rate_limits]]
scope = "key"
id = "vk-uuid"
rpm = 600
tpm = 150000
```

---

### \[retry\]

On a transient upstream failure (HTTP 408, 429, 5xx, or a connection error), Rolter re-picks a different target and retries. Retries only happen before any response bytes have been streamed to the client, so no partial response is ever duplicated.

In a database-backed deployment the store owns this table: the snapshot carries what is saved under **Settings → Performance Tuning**, see [Performance tuning](/reliability/performance-tuning). It takes effect from the file for a gateway that reads `rolter.toml` itself.

<ParamField path="max_retries" default="2" type="u32">
  Extra attempts after the first. Set to `0` to disable retries entirely.
</ParamField>

<ParamField path="base_backoff_ms" default="100" type="u64">
  The base delay in milliseconds for the first retry. Actual delay uses exponential backoff with full jitter. A `Retry-After` header from the upstream overrides the computed delay.
</ParamField>

<ParamField path="max_backoff_ms" default="2000" type="u64">
  The ceiling for the backoff delay in milliseconds.
</ParamField>

```toml
[retry]
max_retries = 2
base_backoff_ms = 100
max_backoff_ms = 2000
```

---

### \[cooldown\]

After a target returns a transient failure, Rolter parks it for `base_secs` and the balancer skips it, shifting load to healthy siblings. When every sibling is parked, the gateway fails open rather than rejecting the request.

<ParamField path="base_secs" default="5" type="u64">
  How long a failing target is parked, in seconds. Set to `0` to disable cooldowns entirely.
</ParamField>

<ParamField path="max_secs" default="300" type="u64">
  The ceiling for a cooldown duration derived from a `429 Retry-After` header.
</ParamField>

```toml
[cooldown]
base_secs = 5
max_secs = 300
```

---

### \[cache\]

Global exact-match response-cache settings. Backed by Redis; off by default. Individual routes must also opt in or carry a virtual-key cache override.

<ParamField path="enabled" default="false" type="boolean">
  Master switch for global response caching.
</ParamField>

<ParamField path="default_ttl_secs" default="60" type="u64">
  Default entry time-to-live in seconds, overridable per route.
</ParamField>

<ParamField path="namespace" default="rolter:cache" type="string">
  Redis key prefix for cache entries to isolate and flush entries easily.
</ParamField>

<ParamField path="max_entry_bytes" default="1048576" type="u64">
  Largest response body in bytes that may be cached (defaults to 1 MiB).
</ParamField>

```toml
[cache]
enabled = true
default_ttl_secs = 60
namespace = "rolter:cache"
max_entry_bytes = 1048576
```

---

### \[responses\]

Routing records for the OpenAI Responses lifecycle endpoints. When a native OpenAI provider creates a response, the gateway remembers which provider, target and credential served it, so `GET` and `DELETE /v1/responses/{id}`, cancellation and input-item retrieval go back to the same place. Records are isolated by virtual key and removed after a successful delete.

The registry lives in the gateway process. Records do not survive a restart, and a deployment with several gateway replicas has to keep a response's lifecycle requests on the replica that created it. An unknown, expired, deleted or cross-key record returns `404 response_not_found`.

<ParamField path="registry_ttl_secs" default="86400" type="integer">
  How long a record lives after the response is created, in seconds (24 hours). `0` disables registration.
</ParamField>

<ParamField path="registry_max_entries" default="100000" type="integer">
  The most live records one gateway process keeps. `0` disables registration too. Expired and overflowing records are cleaned up lazily, on lookup and insert.
</ParamField>

```toml
[responses]
registry_ttl_secs = 86400
registry_max_entries = 100000
```

---

### \[timeouts\]

Timeouts bound how long the gateway waits for an upstream before treating the request as a transient failure and feeding it into the retry and cooldown machinery. `request_secs` applies to time-to-response-headers only, so legitimately long SSE streams are never cut short.

In a database-backed deployment the store owns this table: the snapshot carries what is saved under **Settings → Performance Tuning**, see [Performance tuning](/reliability/performance-tuning). It takes effect from the file for a gateway that reads `rolter.toml` itself.

<ParamField path="connect_secs" default="10" type="u64">
  The connection-establishment timeout in seconds. Set to `0` to disable.
</ParamField>

<ParamField path="request_secs" default="60" type="u64">
  The time-to-response-headers timeout in seconds. Set to `0` to disable.
</ParamField>

```toml
[timeouts]
connect_secs = 10
request_secs = 60
```

---

### \[compatibility\]

Cross-dialect compatibility settings, applied when Rolter translates a request between the OpenAI and Anthropic wire formats. Both values have no equivalent on the other side of the translation, so they have to come from somewhere.

<ParamField path="anthropic_version" default="2023-06-01" type="string">
  The `anthropic-version` header sent to Anthropic upstreams on a translated request. It is forwarded verbatim, so it must be a dated release string; the dashboard and the API refuse anything else.
</ParamField>

<ParamField path="default_max_tokens" default="1024" type="integer">
  The `max_tokens` Rolter fills in when it translates an OpenAI request that set neither `max_tokens` nor `max_completion_tokens`, since the Anthropic Messages API requires one. A request that sets its own cap always wins. The dashboard accepts 1 to 1000000.
</ParamField>

```toml
[compatibility]
anthropic_version = "2023-06-01"
default_max_tokens = 1024
```

<Note>
  **Database-backed deployments.** Behind a control plane with a database, the compatibility policy is owned by the store: the snapshot carries what is saved under **Settings → Compatibility**, and `rolter-seed --import` does not read this table from the file. Edit it there, see [Compatibility](/configuration/compatibility). This table takes effect for a gateway that reads `rolter.toml` itself.
</Note>

---

### \[client\]

Header handling on the upstream leg of a request, and the base URL the dashboard advertises to clients.

<ParamField path="public_base_url" type="string">
  The address clients point their SDK at, such as `https://gateway.example.com`, without `/v1`. The dashboard builds its connection snippets from it. Advisory only: the gateway never reads it, and setting it does not change what the gateway listens on. When unset, the dashboard shows no connection snippet.
</ParamField>

<ParamField path="forwarded_headers" default="[]" type="list">
  Inbound client headers passed through to the upstream provider, written in lowercase. Anything not listed is dropped. Trace-context headers (`traceparent`, `tracestate`, `b3`) and every `anthropic-*` header to an Anthropic-dialect upstream propagate whether or not they are listed. Never list `authorization`, `x-api-key`, `api-key`, `host`, `content-length`, `content-type`, `transfer-encoding`, `connection` or `cookie`: the dashboard and the API refuse them, because forwarding one would hand a caller the gateway's own credentials or corrupt the upstream request, and the gateway does not check a file for them.
</ParamField>

<ParamField path="injected_headers" type="table">
  Static header names and values added to every upstream request, none by default and written with lowercase names: a partner id, a routing tag, a billing dimension your provider expects. A trace header of the same name still wins. The reserved names listed under `forwarded_headers` apply here too.
</ParamField>

<ParamField path="request_id_header" default="x-request-id" type="string">
  The header carrying the end-to-end request id. The gateway reads the caller's id from it, mints a UUID when it is missing or blank, echoes the id in it on the response and stores it as the request-log row's `request_id`. With another name set, `x-request-id` is an ordinary header the gateway neither reads nor sets. The name cannot be one that carries a credential, session state or message framing (`authorization`, `x-api-key`, `api-key`, `x-goog-api-key`, `proxy-authorization`, `cookie`, `set-cookie`, `host`, `content-length`, `content-type`, `transfer-encoding`, `connection` and the other hop-by-hop headers): a file naming one fails validation. See [Client settings](/configuration/client-settings#request-id-header).
</ParamField>

```toml
[client]
public_base_url = "https://gateway.example.com"
forwarded_headers = ["x-tenant-id"]
injected_headers = { "x-partner-id" = "acme" }
request_id_header = "x-request-id"
```

<Note>
  **Database-backed deployments.** A control plane with a database owns this table: the snapshot carries what is saved under **Settings → Client Settings**, and `rolter-seed --import` does not read it from the file. See [Client settings](/configuration/client-settings). The table takes effect for a gateway that reads `rolter.toml` itself. A control plane started with `--config` as well as a database serves the file's table until something other than the defaults is saved on the screen, and the saved one after that.
</Note>

---

### \[model_defaults\]

Inference parameters Rolter fills in when a client leaves them out. Every field only ever fills a gap: a value the client sent, an explicit `null` included, is never overwritten, so turning this on cannot change the meaning of a request that was already explicit.

<ParamField path="enabled" default="false" type="boolean">
  Master switch. While `false`, request bodies reach the provider untouched, whatever the other fields hold.
</ParamField>

<ParamField path="default_model" type="string">
  The `model` used when a request omits one or sends an empty one. It can name a routed model or a `group-slug/model` address. Leave it unset to keep rejecting such requests.
</ParamField>

<ParamField path="temperature" type="float">
  The sampling temperature applied when the client sent none. The dashboard accepts 0 to 2.
</ParamField>

<ParamField path="top_p" type="float">
  The nucleus-sampling value applied when the client sent none. The dashboard accepts 0 to 1.
</ParamField>

<ParamField path="max_tokens" type="integer">
  The completion-length cap applied when the client sent none, written under the key the upstream dialect reads (`max_tokens`, or `max_output_tokens` for the Responses API). A request that already set `max_completion_tokens` on OpenAI chat has stated its cap, so none is added. The dashboard accepts 1 to 1000000.
</ParamField>

These apply to `/v1/chat/completions`, `/v1/messages` and `/v1/responses`. Embeddings, audio and image endpoints take no sampling parameters and are left alone, and Gemini, which nests them under `generationConfig`, does not take defaults from here.

```toml
[model_defaults]
enabled = true
default_model = "gpt-4o-mini"
temperature = 0.7
max_tokens = 2048
```

<Note>
  **Database-backed deployments.** A control plane with a database owns this table: the snapshot carries what is saved under **Settings → Model Settings**, and `rolter-seed --import` does not read it from the file. See [Model settings](/configuration/model-settings), whose API names the fields `default_temperature`, `default_top_p` and `default_max_tokens`. The table takes effect for a gateway that reads `rolter.toml` itself. A control plane started with `--config` as well as a database serves the file's table until something other than the defaults is saved on the screen, and the saved one after that.
</Note>

---

### \[adaptive_routing\]

The deployment-wide policy for routes on the `adaptive` strategy, which scores each target from live latency, catalog cost and in-flight load instead of using static weights. The whole feature is off by default: with `enabled` off, or before a route has gathered enough evidence to rank its targets, an `adaptive` route behaves exactly like the `pipeline` strategy, so moving a route onto it shifts no traffic until you say so.

<ParamField path="enabled" default="false" type="boolean">
  The master kill switch. While `false`, every `adaptive` route is served by the `pipeline` fallback stack.
</ParamField>

<ParamField path="latency_weight" default="1.0" type="float">
  How strongly observed latency pulls traffic toward a target.
</ParamField>

<ParamField path="cost_weight" default="0.5" type="float">
  How strongly a low catalog price pulls traffic toward a target.
</ParamField>

<ParamField path="load_weight" default="0.25" type="float">
  How strongly in-flight load pushes traffic away from a target.
</ParamField>

<ParamField path="exploration_ratio" default="0.05" type="float">
  The share of picks made uniformly at random, so a target the blend would otherwise starve keeps producing fresh latency samples. Values above `0.5` are clamped to `0.5`.
</ParamField>

<ParamField path="min_samples" default="50" type="integer">
  The number of requests a **route** must have served before the blend engages, counted across all its targets. While `latency_weight` is above `0` the blend also needs latency samples for at least two targets. Until then the route is served by the fallback stack.
</ParamField>

Only the ratio between the three weights matters, and a negative weight is treated as `0`. If all three are `0` the blend has nothing to rank by, and Rolter treats the policy as not configured. See [Adaptive routing](/concepts/adaptive-routing) for how to tune the blend and read the telemetry.

```toml
[adaptive_routing]
enabled = true
latency_weight = 1.0
cost_weight = 0.5
load_weight = 0.25
exploration_ratio = 0.05
min_samples = 50
```

<Note>
  **Database-backed deployments.** A control plane with a database owns this table: the snapshot carries what is saved under **Adaptive Routing → Settings**, and `rolter-seed --import` does not read it from the file. The table takes effect for a gateway that reads `rolter.toml` itself.
</Note>

---

### \[queue\]

Rolter keeps a bounded dispatch queue and a worker set for each provider. A
provider that stalls before sending response headers therefore backs up only its
own queue; requests selected for other providers continue independently. Once
headers arrive, response bodies (including SSE) stream directly to the caller.
The settings are applied to subsequent requests immediately after a config
snapshot reload.

In a database-backed deployment the store owns this table: the snapshot carries what is saved under **Settings → Performance Tuning**, see [Performance tuning](/reliability/performance-tuning). It takes effect from the file for a gateway that reads `rolter.toml` itself.

<ParamField path="enabled" default="true" type="boolean">
  Set to `false` to bypass queues and forward directly.
</ParamField>

<ParamField path="capacity" default="256" type="usize">
  Maximum waiting requests per provider. Active workers do not count against this limit.
</ParamField>

<ParamField path="workers" default="8" type="usize">
  Concurrent upstream requests waiting for response headers for each provider:
  a provider with `workers = 8` has up to eight such calls outstanding, and the
  next one waits for a worker. A worker is released when the headers arrive, so
  a streamed response keeps generating upstream after its worker has moved on
  and does not count against this limit.
</ParamField>

<ParamField path="backpressure" default="error" type="drop | block | error">
  `drop` sheds a full queue with a 503; `error` returns a 429 `queue_full` response. `block` waits for capacity up to `block_timeout_ms` and then returns 429 `queue_timeout`.
</ParamField>

<ParamField path="block_timeout_ms" default="1000" type="u64">
  Maximum capacity wait when `backpressure = "block"`. It must be greater than zero for that policy.
</ParamField>

```toml
[queue]
enabled = true
capacity = 256
workers = 8
backpressure = "error"
block_timeout_ms = 1000
```

A `[queue]` change takes effect for requests that start after the reload. Requests already buffered in the replaced queue are not dropped: its workers finish them, so for that short window a provider can see the old and the new `workers` at the same time.

Queue shedding is exposed through `rolter_provider_queue_rejections_total`; timed-out blocking admissions use `rolter_provider_queue_timeouts_total`. Depth, in-flight calls and the time requests wait for a worker are per-provider series described under [Provider queue](/observability/metrics#provider-queue).

---

### \[health\]

When enabled, a background task periodically probes each provider with a lightweight `GET` request. A provider that times out, fails to connect, or returns `5xx` is marked unhealthy and its targets are skipped. Any non-5xx response (including `401` or `404`) counts as healthy. Disabled by default.

<ParamField path="enabled" default="false" type="boolean">
  Master switch for active health probing.
</ParamField>

<ParamField path="interval_secs" default="10" type="u64">
  Seconds between probe sweeps.
</ParamField>

<ParamField path="timeout_secs" default="2" type="u64">
  Per-probe timeout in seconds.
</ParamField>

<ParamField path="path" default="/" type="string">
  The request path appended to each provider's `api_base` when probing. Left at the default `/`, Rolter probes each provider kind's free non-inference liveness endpoint instead (`/v1/models` for `openai`/`openai_compatible`, and `/v1/models` with an `anthropic-version` header for `anthropic`) — no token cost. Set an explicit path (e.g. `/health`) to override this for all providers.
</ParamField>

<ParamField path="probe_concurrency" default="2" type="usize">
  Maximum probes in flight at once during a sweep. Each provider also starts at a stable offset of up to a quarter of the interval (at most two seconds), so a sweep never hits every upstream at the same moment. A value of `0` is treated as `1`.
</ParamField>

<ParamField path="consecutive_failure_threshold" default="3" type="u32">
  Consecutive probe failures before a provider is marked unhealthy and its targets are skipped. One failed probe never removes a provider on its own. A value of `0` is treated as `1`.
</ParamField>

<ParamField path="recovery_success_threshold" default="2" type="u32">
  Consecutive probe successes before an unhealthy provider returns to rotation, so a flapping upstream does not bounce in and out. A value of `0` is treated as `1`.
</ParamField>

<ParamField path="catalogue_interval_secs" default="900" type="u64">
  Seconds between refreshes of each provider's model catalogue, which `GET /v1/models` lists as `provider-slug/model`. The refresh is one free model-list request per provider and runs even with `enabled = false`. `0` turns it off. See [Models](/api/models).
</ParamField>

<ParamField path="status_page_interval_secs" default="60" type="u64">
  Seconds between polls of provider status pages. Only providers with a `status_page_url` are polled, and the poller runs even with `enabled = false`. A status page is a secondary signal: it records a `status_page` event and never changes routing on its own. See [Health checks](/reliability/health-checks).
</ParamField>

```toml
[health]
enabled = true
interval_secs = 10
timeout_secs = 2
path = "/"
probe_concurrency = 2
consecutive_failure_threshold = 3
recovery_success_threshold = 2
catalogue_interval_secs = 900
status_page_interval_secs = 60
```

---

### \[breaker\]

The circuit breaker complements the short-lived cooldown with a longer-lived state machine. After `failure_threshold` consecutive transient failures a target trips **open** and is skipped for `open_secs`. The first request after that window probes the target in a **half-open** state, and other requests skip the target until that probe answers; a success closes the breaker, a failure re-opens it. Disabled by default.

<ParamField path="enabled" default="false" type="boolean">
  Master switch for the circuit breaker.
</ParamField>

<ParamField path="failure_threshold" default="5" type="u32">
  Consecutive transient failures required to trip a target open. Setting this to `0` also disables the breaker.
</ParamField>

<ParamField path="open_secs" default="30" type="u64">
  How long a tripped target stays open before a half-open probe, in seconds.
</ParamField>

```toml
[breaker]
enabled = true
failure_threshold = 5
open_secs = 30
```

---

### \[metrics_scrape\]

When enabled, a background task periodically pulls each provider's Prometheus `/metrics` endpoint and extracts scheduler queue depth (vLLM `num_requests_waiting`, with SGLang/TGI fallbacks). That depth feeds into the balancer's in-flight load view so load-aware strategies (`power_of_two`, `pipeline`) steer away from backed-up engines. Disabled by default.

<ParamField path="enabled" default="false" type="boolean">
  Master switch for background metrics scraping.
</ParamField>

<ParamField path="interval_secs" default="5" type="u64">
  Seconds between scrape sweeps.
</ParamField>

<ParamField path="timeout_secs" default="2" type="u64">
  Per-scrape timeout in seconds.
</ParamField>

<ParamField path="path" default="/metrics" type="string">
  The request path appended to each provider's `api_base` when scraping.
</ParamField>

```toml
[metrics_scrape]
enabled = true
interval_secs = 5
timeout_secs = 2
path = "/metrics"
```

---

### \[realtime\]

Resource limits and metering cadence for [`/v1/realtime`](/api/realtime) WebSocket sessions. The three limits are per gateway process and guard the gateway's own resources; tenant policy comes from `[[budgets]]` and `[[rate_limits]]`, which a session meets at admission and while it runs. Set a limit to `0` to disable it.

<ParamField path="max_connections" default="1000" type="u64">
  Concurrent realtime sessions this gateway process admits.
</ParamField>

<ParamField path="max_session_secs" default="3600" type="u64">
  Hard limit on a session's duration, in seconds.
</ParamField>

<ParamField path="idle_timeout_secs" default="300" type="u64">
  Closes a session when neither side has sent a frame for this many seconds.
</ParamField>

<ParamField path="usage_flush_secs" default="15" type="u64">
  How often a live session's finished responses are written to the request log, charged to budgets and `tpm` windows, and followed by a budget check that closes the session once a budget is spent. It bounds how far a session can overspend. `0` flushes after every response instead of on a timer, and re-checks the budgets every second while the session is quiet; metering cannot be switched off.
</ParamField>

```toml
[realtime]
max_connections = 1000
max_session_secs = 3600
idle_timeout_secs = 300
usage_flush_secs = 15
```

---

### \[usage_recording\]

After a response completes, Rolter adds the request's cost to its applicable budget counters and its tokens to its applicable rate-limit windows. Both write to Redis, so both run off the response path — through a **bounded** queue rather than one detached task per request, so a slow counter store sheds records instead of accumulating work without limit.

Dropping is the deliberate failure mode: a lost record under-counts spend for a single request, while an unbounded queue grows worst exactly when the gateway is already under stress. Overflow is never silent — watch `rolter_usage_records_dropped_total`, and `rolter_usage_records_queued` for pressure before it turns into drops.

This section is inert without a Redis URL: with no counter store there is nothing to record.

<ParamField path="queue_capacity" default="10000" type="integer">
  Records waiting to be written. New records are dropped (and counted) once the queue is full; the response path never blocks on it.
</ParamField>

<ParamField path="workers" default="4" type="integer">
  Concurrent writers draining the queue. Each record is one Redis round trip, so a single consumer would serialise them. Raise this if `rolter_usage_records_queued` sits high against a healthy Redis.
</ParamField>

```toml
[usage_recording]
queue_capacity = 10000
workers = 4
```

---

### \[security\]

Ingress rules applied to every request on the `/v1/*` surface. A gateway managed by the control plane takes these from the dashboard's Security screen; a file-configured gateway reads them from this section. Both keys are optional and the section adds no rules when omitted.

<ParamField path="required_headers" type="table">
  Lowercase header name to the exact value every request must carry. A request that is missing one,
  or carries a different value, is refused with `403` before authentication. This is an ingress
  filter (a mesh identity header, a WAF marker), not an authorization decision. The `/healthz` and `/readyz`
  probes are exempt.
</ParamField>

<ParamField path="auth_bypass_routes" type="array of strings">
  Exact `/v1/...` paths served without authentication. Matching is exact: no prefixes and no
  wildcards.
</ParamField>

```toml
[security]
auth_bypass_routes = ["/v1/models"]

[security.required_headers]
x-mesh-identity = "gateway"
```

---

### \[logging\]

Request and cost logs are written asynchronously in batches, off the hot request path. Logging is disabled entirely when `clickhouse_url` is unset. When the internal queue fills, records are dropped and counted rather than blocking request processing.

<ParamField path="clickhouse_url" type="string">
  The base URL of the ClickHouse HTTP interface, e.g. `http://localhost:8123`. Logging is disabled when this is not set.
</ParamField>

<ParamField path="batch_max" default="1000" type="integer">
  Flush a batch once it reaches this many rows.
</ParamField>

<ParamField path="flush_ms" default="1000" type="integer">
  Also flush a partial batch at least this often, in milliseconds.
</ParamField>

<ParamField path="queue_capacity" default="10000" type="integer">
  The maximum number of log records buffered in-flight. Records are dropped when this queue is full.
</ParamField>

<ParamField path="sample_rate" default="1.0" type="float">
  The fraction of requests written to the request log, from `0` to `1`, applied before a row is written. `1` keeps every record and `0` writes no request-log rows. The decision is made per request from a hash of the request id, and each kept row records its weight so the analytics endpoints scale counts back up. Budgets, rate limits and the Prometheus metrics still count every request. See [Log settings](/observability/log-settings#sampling).
</ParamField>

<ParamField path="payload_capture" type="table">
  Opt-in capture of raw prompt and completion bodies next to the log metadata, written as `[logging.payload_capture]`. Off by default, because payloads hold whatever the caller sent. See below.
</ParamField>

<ParamField path="ui_events" default="true" type="boolean">
  Whether the control plane records the dashboard's own usage events: screen views, load times, abandoned forms and error states. They carry structure only (a screen, an action, a duration and ids), never a form value or a prompt, which is why this is on by default where `payload_capture` is not. `false` drops them at the ingest endpoint. The stream is inert anyway without `clickhouse_url`.
</ParamField>

```toml
[logging]
clickhouse_url = "http://localhost:8123"
batch_max = 1000
flush_ms = 1000
queue_capacity = 10000
sample_rate = 1.0
ui_events = true
```

#### \[logging.payload_capture\]

Captured bodies go to ClickHouse's short-retention `request_payloads` table. Redaction happens before the write, so a redacted value never reaches storage, and turning a field on later does not clean what is already stored.

<ParamField path="enabled" default="false" type="boolean">
  The master switch. Capture is off unless this is explicitly `true`.
</ParamField>

<ParamField path="max_bytes" default="32768" type="integer">
  The most UTF-8 bytes kept of each request or response body; anything longer is truncated.
</ParamField>

<ParamField path="redact_fields" default='["api_key", "authorization", "password", "secret", "token"]' type="list">
  JSON object keys whose values are replaced before storage. Matching ignores case and reaches nested objects, tool arguments included.
</ParamField>

<ParamField path="redact_detectors" default="all built-in detectors" type="list">
  Built-in detectors that scan the text of every captured string and replace a match with a typed marker such as `[redacted:aws_key]`: `private_key`, `anthropic_key`, `openai_key`, `google_api_key`, `github_token`, `slack_token`, `stripe_key`, `aws_key`, `jwt`, `bearer_token` and `connection_string` (only the password of a `scheme://user:password@host` string goes). An empty list turns the built-ins off. An unknown name is rejected when saved through the dashboard.
</ParamField>

<ParamField path="redact_patterns" default="[]" type="list">
  Your own value patterns, each a table with a `name` (lowercase letters, digits or `_`, up to 32 characters, which becomes the marker `[redacted:<name>]`) and a `pattern` (a regular expression of at most 256 bytes). Written as `[[logging.payload_capture.redact_patterns]]`. Matching uses a linear-time engine with no look-around or backreferences, at most 16 patterns are allowed, and a pattern that does not compile is rejected when saved.
</ParamField>

<ParamField path="models" default="[]" type="list">
  Public model names to capture. Empty captures every route; a non-empty list limits capture to the routes named.
</ParamField>

<ParamField path="virtual_key_ids" default="[]" type="list">
  Database virtual-key ids to capture. Empty captures every key that `models` allows; a non-empty list makes capture key-specific.
</ParamField>

```toml
[logging.payload_capture]
enabled = true
max_bytes = 32768
redact_fields = ["api_key", "authorization", "password", "secret", "token"]
models = ["support-agent"]
virtual_key_ids = []

[[logging.payload_capture.redact_patterns]]
name = "employee_id"
pattern = 'EMP-\d{6}'
```

<Note>
  **Database-backed deployments.** Behind a control plane with a database, `sample_rate`, `payload_capture` and `ui_events` are owned by the store: the snapshot carries what is saved under **Observability → Logs Settings**, see [Log settings](/observability/log-settings). `rolter-seed --import` copies `[logging.payload_capture]` and `ui_events` into the database when the file writes them, and does not read `sample_rate`. All three take effect from the file only for a gateway that reads `rolter.toml` itself. `clickhouse_url` follows a different rule: a gateway that names none inherits the control plane's own ClickHouse URL from the snapshot, and one that names its own keeps it.
</Note>

---

### \[feature_flags\]

Deployment-wide switches for the subsystems that are safe to turn on and off at runtime. They belong to the control plane: an admin edits them under **Settings → Feature Flags**, the control plane stores them and applies them to the snapshot every gateway polls, and a gateway picks the change up without a restart. A gateway that reads `rolter.toml` itself does not apply this table, so writing it there does nothing. See [Feature flags](/configuration/feature-flags) for what each flag gates and when one is reported unavailable.

All six default to `true`.

<ParamField path="response_cache" default="true" type="boolean">
  Serving repeated completions from the shared response cache. In the snapshot it sets `[cache] enabled`.
</ParamField>

<ParamField path="cache_aware_routing" default="true" type="boolean">
  Scoring targets by KV-cache affinity. While `false`, a route on `cache_aware`, `precise_cache_aware` or `lmcache_aware` is served by `power_of_two` instead.
</ParamField>

<ParamField path="circuit_breaker" default="true" type="boolean">
  Tripping a failing target out of the pool and probing it before restoring traffic. In the snapshot it sets `[breaker] enabled`.
</ParamField>

<ParamField path="active_health_checks" default="true" type="boolean">
  Polling targets in the background so an unhealthy one is parked before a live request finds it. In the snapshot it sets `[health] enabled`.
</ParamField>

<ParamField path="complexity_routing" default="true" type="boolean">
  Estimating prompt complexity and routing cheap requests to smaller models. While `false`, a route's complexity policy is dropped.
</ParamField>

<ParamField path="guardrails" default="true" type="boolean">
  Running the configured guardrail rules. In the snapshot it sets `[guardrails] enabled`, so `false` pauses every built-in, custom and config-file rule.
</ParamField>

---

### \[guardrails\]

Built-in regex guardrails and PII redaction, evaluated inside the gateway with no external service and no network hop. Disabled by default, and a disabled or empty block adds no cost to a request. It complements the custom [guardrail webhook](/security/guardrails#external-providers) and the external [PII sanitizer](/security/pii-sanitizer), and never replaces them. [Guardrails](/security/guardrails) describes the dashboard screens and how rules defined here interact with the ones added there.

<ParamField path="enabled" default="false" type="boolean">
  The master switch. While `false` no rule runs. On a control-plane-managed gateway the `guardrails` feature flag is the kill switch on top of it.
</ParamField>

<ParamField path="max_scan_bytes" default="262144" type="integer">
  The most request text scanned per request, in bytes (256 KiB). Content past the cap is forwarded unscanned, which keeps the work bounded.
</ParamField>

<ParamField path="streaming_post_call" default="reject" type="string">
  What a request with `"stream": true` does on a route that has `post_call` rules. Output masking needs the whole completion, because a match can straddle any number of token boundaries, so it cannot run on a stream. `reject` refuses the request with `400 guardrail_streaming_unsupported`, which fails closed. `passthrough` serves the stream with the output rules not applied; `pre_call` rules still run. The setting only matters when a `post_call` rule exists. The dashboard shows the effective value beside each response rule but cannot change it.
</ParamField>

<ParamField path="rules" type="array of tables">
  The rules, written as `[[guardrails.rules]]` entries and run in the order given. See below.
</ParamField>

#### \[\[guardrails.rules\]\]

Provide exactly one of `builtin` or `pattern`. Patterns use a linear-time, RE2-style engine with no catastrophic backtracking, and are compiled under a bounded program size while the config is validated, so an invalid or unbounded pattern fails at startup and never on the request path. The gateway never logs the text a rule matched; metrics only count blocks and redactions.

<ParamField path="name" type="string" required>
  A stable, unique name for the rule. It appears in telemetry, never alongside the matched text, and is how a route's `enable` and `disable` lists refer to it.
</ParamField>

<ParamField path="builtin" type="string">
  A starter detector: `email`, `phone`, `api_token` or `payment_card`. Mutually exclusive with `pattern`. The card detector matches 13 to 19 digit candidates and does not check the Luhn digit.
</ParamField>

<ParamField path="pattern" type="string">
  A custom regular expression. Mutually exclusive with `builtin`.
</ParamField>

<ParamField path="stage" default="pre_call" type="string">
  When the rule runs. `pre_call` scans the request before it goes upstream. `post_call` masks a non-streamed response body before the client sees it, and a streamed one follows `streaming_post_call`.
</ParamField>

<ParamField path="action" default="annotate" type="string">
  What a match does. `annotate` counts it and forwards the content unchanged. `block` rejects the request with an OpenAI-compatible `403 guardrail_blocked` error naming the rule. `redact` replaces each match with `replacement`.
</ParamField>

<ParamField path="replacement" type="string">
  The token that replaces a match under `redact`, up to 64 bytes. When omitted, empty or only whitespace, the detector's default token is used (`[REDACTED:EMAIL]`, `[REDACTED:PHONE]`, `[REDACTED:API_TOKEN]` or `[REDACTED:CARD]`), or `[REDACTED]` for a custom pattern, so a match is never deleted outright.
</ParamField>

<ParamField path="include_system" default="false" type="boolean">
  Also scan operator-authored `system` and `developer` messages. They are trusted and skipped unless you opt in.
</ParamField>

Request rules scan OpenAI `/v1/chat/completions` and `/v1/responses` (`messages` and `input`), `/v1/completions` (`prompt`) and Anthropic `/v1/messages` (`system` and `messages`), including string, string-array and typed `text` parts. Output rules mask `choices[].message.content` and `choices[].text`, the top-level `content` parts of an Anthropic reply, and `output[].content[].text` and `output_text` of a Responses reply, plus the arguments of tool calls. A `block` on a response rule runs after the provider has generated and billed the answer, so the request is still billed.

```toml
[guardrails]
enabled = true
max_scan_bytes = 262144
streaming_post_call = "reject"

[[guardrails.rules]]
name = "email"
builtin = "email"
action = "redact"

[[guardrails.rules]]
name = "card"
builtin = "payment_card"
action = "block"

[[guardrails.rules]]
name = "leaked-key"
builtin = "api_token"
stage = "post_call"
action = "redact"

[[guardrails.rules]]
name = "internal-ticket"
pattern = "TICKET-[0-9]{6}"
action = "annotate"
```

To switch a rule off for one route, name it in that route's `[routes.advanced.guardrails]` `disable` list, described under `[[routes]]` above. The old `default_on` key was removed: it documented a per-request opt-in that was never implemented, and a config that still sets it loads with the key ignored.

---

### \[egress\]

Where the gateway may send upstream traffic. A provider's `api_base` and `egress_proxy` are written by an operator, so without a destination policy a crafted or compromised provider row turns the gateway into a way to reach whatever its network position can reach, and the prize on a cloud host is instance metadata at `169.254.169.254`. [Egress policy](/configuration/egress) lists every URL the policy covers.

Self-hosted and air-gapped upstreams live on loopback and private networks, so only the link-local range is denied by default.

<ParamField path="block_link_local" default="true" type="boolean">
  Deny link-local destinations (`169.254.0.0/16` and `fe80::/10`), which is where cloud instance-metadata services live. Leave it on unless you have a concrete reason.
</ParamField>

<ParamField path="block_loopback" default="false" type="boolean">
  Deny loopback destinations (`127.0.0.0/8` and `::1`) and the unspecified addresses `0.0.0.0` and `::`, which reach the local host when connected to. Off by default so sidecar and single-host deployments can serve models on localhost.
</ParamField>

<ParamField path="block_private" default="false" type="boolean">
  Deny private and unique-local destinations (`10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16` and `fc00::/7`). Off by default because on-premises clusters are the common case.
</ParamField>

<ParamField path="allow_hosts" default="[]" type="list">
  Hosts exempt from every check above: an IP literal or a hostname, matched against the host the URL reaches whatever its spelling, so `169.254.169.254` also covers `http://0xa9fea9fe/`. The escape hatch for a deployment that must reach one otherwise-denied address.
</ParamField>

IP literals are classified when the config is validated, so a bad `api_base` is rejected at startup or when the control plane writes it. A hostname cannot be classified without DNS, so it is checked at connect time against the address the lookup actually returned, which also makes the policy safe against DNS rebinding. A denied address fails the request and is not retried elsewhere.

```toml
[egress]
block_link_local = true
block_loopback = false
block_private = true
allow_hosts = ["vllm.internal.example.com"]  # the one private upstream we mean to reach
```

<Note>
  The database has no egress policy, so a control plane reads this table from the file it was started with (`rolter control --config rolter.toml`) and applies it to every URL written through its API.
</Note>

---

### \[guardrail_webhook\]

A vendor-neutral hook to a self-hosted semantic guardrail service, such as Guardrails AI or LLM Guard. Before it proxies a request, the gateway POSTs a JSON envelope to your endpoint, and the service answers with a decision. Disabled by default, and it complements the built-in [`[guardrails]`](/security/guardrails) rather than replacing them. [Guardrails](/security/guardrails#external-providers) describes registering providers on the dashboard instead. Since [semantic guard policies](/security/semantic-guard-policies) this hook is one provider among those of `[semantic_guards]` and the dashboard: it runs beside them rather than displacing one, and a policy can list it by `name`.

<ParamField path="enabled" default="false" type="boolean">
  The master switch. A disabled block is never a problem, whatever else is unset.
</ParamField>

<ParamField path="name" default="" type="string">
  A label recorded with this hook's outcomes in LLM Logs (`provider_blocked:<name>`). Optional; empty records the bare `provider_blocked` / `provider_transformed`. A provider registered on the dashboard supplies its own name.
</ParamField>

<ParamField path="url" type="string">
  The `http://` or `https://` endpoint the envelope is POSTed to. Required when `enabled` is `true`. The URL is checked against the [egress policy](/configuration/egress) when the config loads and again before every call, and Rolter does not follow a redirect from it.
</ParamField>

<ParamField path="stage" default="pre_call" type="string">
  When the service is consulted. `pre_call` inspects the request before it goes upstream; `post_call` inspects the response before it is delivered. A streamed response is refused or buffered for a `post_call` provider, per the policy's `streaming`.
</ParamField>

<ParamField path="timeout_ms" default="2000" type="integer">
  The per-call timeout in milliseconds. It must be greater than zero.
</ParamField>

<ParamField path="max_retries" default="0" type="integer">
  Extra attempts after a transient failure: a connect error, a timeout, an unreadable body, or a `429` / `503`. Another error status and an answer that is not a decision are not retried.
</ParamField>

<ParamField path="failure_mode" default="fail_open" type="string">
  What happens when the service gives no verdict: it is unreachable, times out, is overloaded, errors, or answers something that is not a decision. `fail_open` forwards the content unchanged, favouring availability. `fail_closed` rejects the request with an OpenAI-compatible error (or withholds the response), favouring enforcement.
</ParamField>

<ParamField path="max_body_bytes" default="65536" type="integer">
  The most content forwarded to the service, in bytes. Oversized content is sent as a truncated preview with `truncated: true`. It must be greater than zero.
</ParamField>

<ParamField path="auth" type="table">
  An optional credential, resolved from the environment at call time and never written inline. `{ bearer = { token_env = "GUARD_TOKEN" } }` sends `Authorization: Bearer <value of GUARD_TOKEN>`, and `{ shared_secret = { secret_env = "GUARD_SECRET" } }` sends the value in an `X-Rolter-Guardrail-Secret` header.
</ParamField>

**The contract.** The envelope is `{ direction, stage, model, route, trace_id, tenant: { org, team, project, key }, truncated, content, policy?, streamed? }`, and only those fields are sent; the gateway never logs the prompt. `policy` names the [policy](/security/semantic-guard-policies) that selected the provider, and `streamed: true` marks a buffered stream, sent as `content: { "stream": true, "text": … }`. The service answers `{ "action": "allow" | "block" | "transform" | "annotate", "content"?, "reason"?, "annotations"?, "categories"? }`, where `categories` maps category names to scores from 0 to 1; a body with `categories` and no `action` is an allow carrying scores. An answer that is not a decision follows `failure_mode`, like a transport failure. The trace id travels in the `X-Rolter-Trace-Id` header.

```toml
[guardrail_webhook]
enabled = true
url = "https://guard.internal/check"
failure_mode = "fail_closed"
auth = { bearer = { token_env = "GUARD_TOKEN" } }
```

---

### \[semantic_guards\]

Several guardrail providers side by side, and the scoped policies that pick which of them judge a request (#459). Empty by default. [Semantic guard policies](/security/semantic-guard-policies) explains the precedence, thresholds, failure handling and streaming in full. The dashboard's **Guardrails → Providers** and **Guardrails → Policies** add to what the file sets; the file wins a name, and an enabled file policy wins its scope.

<ParamField path="providers" type="array of tables">
  The providers, each a `[[semantic_guards.providers]]` table below.
</ParamField>

<ParamField path="policies" type="array of tables">
  The policies, each a `[[semantic_guards.policies]]` table below.
</ParamField>

#### \[\[semantic_guards.providers\]\]

The same keys as [`[guardrail_webhook]`](#guardrail_webhook), one table per provider. Only enabled providers run.

<ParamField path="enabled" default="false" type="boolean">
  Whether the provider runs.
</ParamField>

<ParamField path="name" type="string">
  Required and unique, including against `[guardrail_webhook]`'s name. Policies list providers by it, and it labels the provider's metrics and log events.
</ParamField>

<ParamField path="url" type="string">
  The `http://` or `https://` endpoint, checked against the [egress policy](/configuration/egress) at load and before every call.
</ParamField>

<ParamField path="stage" default="pre_call" type="string">
  `pre_call` judges the request, `post_call` the response.
</ParamField>

<ParamField path="timeout_ms" default="2000" type="integer">
  The per-call timeout in milliseconds.
</ParamField>

<ParamField path="max_retries" default="0" type="integer">
  Extra attempts after a transient failure.
</ParamField>

<ParamField path="failure_mode" default="fail_open" type="string">
  `fail_open` or `fail_closed`, applied when this provider gives no verdict.
</ParamField>

<ParamField path="max_body_bytes" default="65536" type="integer">
  The most content forwarded, in bytes.
</ParamField>

<ParamField path="auth" type="table">
  An optional credential, in the same two forms as `[guardrail_webhook]`.
</ParamField>

#### \[\[semantic_guards.policies\]\]

<ParamField path="name" type="string">
  Required and unique. It labels the policy's metrics and log events.
</ParamField>

<ParamField path="enabled" default="true" type="boolean">
  A paused policy matches nothing, but its providers stay confined to policies.
</ParamField>

<ParamField path="scope" default="deployment" type="string">
  `deployment`, `org`, `project`, `route` or `key`. The most specific enabled match wins, in the order key, route, project, org, deployment, and replaces the broader ones.
</ParamField>

<ParamField path="scope_id" type="string">
  The org, project or key id, or the route's public model name. Required for every scope but `deployment`, which takes none. One enabled policy per scope.
</ParamField>

<ParamField path="providers" default="[]" type="array of strings">
  Provider names, consulted in this order at each stage.
</ParamField>

<ParamField path="thresholds" default="{}" type="table">
  Category name to a score from 0 to 1; a score at or above it violates the policy. `"*"` covers every category not named.
</ParamField>

<ParamField path="on_violation" default="block" type="string">
  `block` refuses the request or withholds the response; `annotate` only records the violation.
</ParamField>

<ParamField path="streaming" default="reject" type="string">
  What a streamed request gets when the policy has a `post_call` provider: `reject` refuses it with `400 guardrail_streaming_unsupported`, `buffer` holds the whole stream, checks it once and then replays or withholds it. No value skips the check.
</ParamField>

```toml
[[semantic_guards.providers]]
name = "toxicity"
enabled = true
url = "https://guard.internal/toxicity"
stage = "post_call"

[[semantic_guards.policies]]
name = "support-bot"
scope = "project"
scope_id = "1f0e…"
providers = ["toxicity"]
thresholds = { violence = 0.8, "*" = 0.95 }
streaming = "buffer"
```

---

### \[pii_sanitizer\]

An external PII sanitizer: a self-hosted de-identification service, Microsoft Presidio or anything that speaks the same small JSON contract, that replaces each detected entity with a placeholder before the content reaches a provider and can reverse the substitution on the way back. Unlike the guardrail webhook, which returns a decision, it returns transformed content. The placeholder-to-plaintext mapping stays on the sanitizer, so Rolter never holds the values. Disabled by default. [PII sanitizer](/security/pii-sanitizer) explains the contract, restoration, streaming and the response cache in full.

<ParamField path="enabled" default="false" type="boolean">
  The master switch.
</ParamField>

<ParamField path="url" type="string">
  The `http://` or `https://` endpoint that sanitizes content and returns placeholders. Required when `enabled` is `true`.
</ParamField>

<ParamField path="restore_url" type="string">
  The endpoint that turns placeholders back into plaintext. Required unless `restoration` is `never`.
</ParamField>

<ParamField path="direction" default="request" type="string">
  Which legs the sanitizer sees: `request` keeps personal data away from a third-party model, `response` scans what the model generated, and `both` does each with a separate call.
</ParamField>

<ParamField path="restoration" default="never" type="string">
  When placeholders may be turned back into plaintext. `never` leaves them in the reply. `trusted_downstream` restores every response, for callers that supplied the data in the first place. `caller_authorized` restores only when the request carries `x-rolter-pii-restore: true`.
</ParamField>

<ParamField path="streaming" default="reject" type="string">
  What a streamed request does while the response leg is active. `reject` refuses it with `400 pii_streaming_unsupported`; `passthrough` serves the stream with the response leg skipped; `restore` restores the stream through `restore_stream_url`. `restore` covers restoration only: when `direction` also covers the response, a streamed request is refused as under `reject`. It needs `restore_stream_url` and a `restoration` other than `never`. A request-only setup streams normally.
</ParamField>

<ParamField path="restore_stream_url" type="string">
  The endpoint that restores a stream over one full-duplex NDJSON exchange per streamed response. Required when `streaming` is `restore`. [PII sanitizer](/security/pii-sanitizer#restoring-a-stream) describes the protocol.
</ParamField>

<ParamField path="timeout_ms" default="5000" type="integer">
  The per-call timeout in milliseconds. It must be greater than zero.
</ParamField>

<ParamField path="max_retries" default="0" type="integer">
  Extra attempts after a transient failure.
</ParamField>

<ParamField path="failure_mode" default="fail_open" type="string">
  `fail_open` forwards the unsanitized body when the sanitizer is unreachable, which sends untouched personal data upstream during an outage. `fail_closed` refuses the request with `502 pii_sanitizer_unavailable`. Choose `fail_closed` when the sanitizer is a compliance control.
</ParamField>

<ParamField path="max_body_bytes" default="262144" type="integer">
  The most content sent to the sanitizer, in bytes. Oversized content is sent as a truncated preview with `truncated: true`. It must be greater than zero.
</ParamField>

<ParamField path="entities" default="[]" type="list">
  The entity types to detect, such as `EMAIL_ADDRESS` or `PERSON`. Empty means whatever the service is configured for.
</ParamField>

<ParamField path="auth" type="table">
  An optional credential, in the same two forms as `[guardrail_webhook]`, resolved from the environment at call time.
</ParamField>

All three URLs are checked against the [egress policy](/configuration/egress) when the config loads and again before every call.

```toml
[pii_sanitizer]
enabled = true
url = "https://presidio.internal/sanitize"
restore_url = "https://presidio.internal/restore"
direction = "request"
restoration = "never"
failure_mode = "fail_closed"
entities = ["EMAIL_ADDRESS", "PERSON", "CREDIT_CARD"]
auth = { bearer = { token_env = "PII_SANITIZER_TOKEN" } }
```

---

### \[prompt_templates\]

Centrally managed, versioned prompt templates. A template adds system, assistant or user messages around a caller's own, so applications reuse approved instructions without being granted arbitrary prompt-authoring rights. Disabled by default, and a disabled or empty block adds no cost to a request. [Prompt templates](/concepts/prompt-templates) covers the dashboard repository, scopes and the request-time errors.

<ParamField path="enabled" default="false" type="boolean">
  The master switch.
</ParamField>

<ParamField path="templates" type="array of tables">
  The templates, written as `[[prompt_templates.templates]]` entries. Each is one immutable version, and you list exactly the versions to activate. See below.
</ParamField>

#### \[\[prompt_templates.templates\]\]

<ParamField path="id" type="string" required>
  A stable identifier. It appears in safe metadata, never in content logs. The `(id, version)` pair must be unique.
</ParamField>

<ParamField path="version" type="integer" required>
  The immutable version, at least `1`.
</ParamField>

<ParamField path="routes" default="[]" type="list">
  The public model names the template applies to. Empty means every route.
</ParamField>

<ParamField path="variables" type="array of tables">
  The named variables a decorator may reference as `{{ name }}`, written as `[[prompt_templates.templates.variables]]`.
</ParamField>

<ParamField path="decorators" type="array of tables" required>
  The messages injected around the caller's own, written as `[[prompt_templates.templates.decorators]]`. A template needs at least one.
</ParamField>

##### \[\[prompt_templates.templates.variables\]\]

<ParamField path="name" type="string" required>
  The variable name, matching `[A-Za-z_][A-Za-z0-9_]*`.
</ParamField>

<ParamField path="required" default="false" type="boolean">
  The caller must supply the variable on every request the template applies to, whether or not a decorator references it. Mutually exclusive with `default`.
</ParamField>

<ParamField path="default" type="string">
  The value used when the caller omits the variable. An optional variable with no default renders as an empty string.
</ParamField>

##### \[\[prompt_templates.templates.decorators\]\]

<ParamField path="role" default="system" type="string">
  The role of the injected message: `system`, `assistant` or `user`. For Anthropic, a `system` decorator folds into the top-level `system` field.
</ParamField>

<ParamField path="position" default="prepend" type="string">
  `prepend` places the message before the caller's messages and `append` places it after, both in the order declared.
</ParamField>

<ParamField path="content" type="string" required>
  The message text, which may contain `{{ variable }}` placeholders. Every placeholder is checked at config load to name a declared variable.
</ParamField>

Callers pass values in a `rolter_template_vars` object on the request body, which is always stripped before the request goes upstream. A caller's value overrides the declared default. An unknown variable, a missing required variable, a value over 4 KiB or a rendered message over 16 KiB is refused with `invalid_prompt_template`. Substitution is structural: each rendered message is emitted as a JSON string through the serializer, never concatenated into raw JSON, so a value cannot break out of its string or inject extra messages.

Templates are applied to `/v1/chat/completions`, `/v1/responses` and Anthropic `/v1/messages`. Surfaces without a message array, such as `/v1/completions`, are not decorated.

```toml
[prompt_templates]
enabled = true

[[prompt_templates.templates]]
id = "support-preamble"
version = 3
routes = ["gpt-4o"]

[[prompt_templates.templates.variables]]
name = "persona"
default = "a helpful support assistant"

[[prompt_templates.templates.decorators]]
role = "system"
position = "prepend"
content = "You are {{ persona }}. Follow the company policy and be concise."
```

In a database-backed deployment, templates are authored on the Prompt Repository screen, and `rolter-seed --import` seeds the templates in this table into the database, creating missing versions and refusing to overwrite an existing version with different content.

---

### \[plugins\]

Not a `rolter.toml` table. It holds the enabled webhook plugin instances that the control plane builds from the plugin registry (**Plugins** on the dashboard) and ships in the snapshot, for the gateway's `pre_route`, `pre_upstream` and `post_response` stages. The list is empty by default and costs nothing on the request path. Each instance carries its `slug`, `org_id`, an optional `project_id`, `stage`, `position`, `failure_mode`, `endpoint` and an optional `auth`. See [Plugins](/configuration/plugins), which is experimental, for scope, ordering and failure policy.

---

<Note>
  The repository ships a fully annotated `rolter.example.toml` that shows every table a `rolter.toml` can set, with inline comments explaining each field; most blocks are commented out with their defaults. Copy it to `rolter.toml` to get started, then remove or uncomment the sections you need. It leaves out the tables the control plane owns and a file does not set: `feature_flags`, `db_virtual_keys`, `mcp_servers`, `mcp_oauth_sessions`, `mcp_tool_groups`, `mcp_settings` and `plugins`.
</Note>