One deployment-wide runtime policy governs how Rolter talks to upstreams under load: how hard it retries, how long it waits, and how many requests it admits at once. Edit it from Settings → Performance Tuning in the dashboard, or through the management API.
Retries#
| Setting | Range | Meaning |
|---|---|---|
retry_max_retries | 0–10 | Retry budget for a single client request. |
retry_base_ms | 0–60000 | First backoff interval. |
retry_max_ms | 0–600000 | Ceiling the exponential backoff grows to. |
The cap cannot be lower than the base — an incoherent pair is rejected rather than silently clamped.
Timeouts#
timeout_connect_s (0–300) bounds establishing the upstream connection; timeout_request_s (0–3600) bounds the wait for the upstream’s response headers. Once the headers arrive the body is not timed, so a streaming response that has started runs to completion however long it takes. Zero disables the corresponding timeout.
Admission queue#
Rolter creates an independent bounded queue and worker set per provider, so one slow provider cannot consume the admission capacity that healthy providers need.
| Setting | Range | Meaning |
|---|---|---|
queue_capacity | 1–100000 | Requests that may wait for a worker. |
queue_workers | 1–2048 | Requests per provider waiting for response headers (does not cap streams). |
queue_backpressure | drop / block / error | What happens when the queue is full. |
queue_block_ms | 0–120000 | How long block waits for capacity. |
On the dashboard the queue fields are disabled while the queue is off, and the block timeout unless the policy is block. A disabled field is not validated, so an out-of-range value already stored never blocks Save, and saving keeps it unchanged. It is checked again once the field is enabled.
The dashboard shows each problem under the field it belongs to, all at once, and the backpressure dropdown offers Drop, Block and Error for the wire values below. Save and Discard changes wait for an edit; a panel holding one is marked Edited, the footer counts the unsaved settings, and leaving the screen with unsaved edits asks first.
The three backpressure policies:
drop— shed the request immediately with an overload response.block— wait for capacity untilqueue_block_msexpires. This requires a non-zero timeout; blocking forever would park callers indefinitely, so the combination is rejected.error— return a structuredqueue_fullerror immediately. This is the default.
Shed requests never fail over to a sibling target and never retry: admission backpressure is a signal to the caller, not an upstream fault. Each policy has a fixed wire shape:
| Policy | HTTP status | error.code | error.type |
|---|---|---|---|
drop | 503 | queue_dropped | overloaded_error |
block (timed out) | 429 | queue_timeout | rate_limit_error |
error | 429 | queue_full | rate_limit_error |
Rejections and timeouts are counted separately on the gateway metrics endpoint as
rolter_provider_queue_rejections_total and
rolter_provider_queue_timeouts_total, so backpressure is visible as a real
counter rather than only as a client-side status code.
API#
Both endpoints are for a superadmin or an operator (a deployment role). Every update is audited.
Send the full policy, not a patch. A successful update bumps config_version, so gateways pick it up on their next snapshot poll without a restart.