Rolter exposes three operational endpoints that require no authentication and are designed to be scraped or probed directly by your infrastructure tooling. Use /healthz for liveness checks, /readyz for readiness checks, and /metrics for Prometheus scraping.
GET /healthz#
The liveness endpoint returns 200 OK with the body ok when the process is running and its HTTP server is accepting connections. No authentication is required.
It checks no dependency — not PostgreSQL, not Redis, not ClickHouse, not an upstream provider. That is deliberate: a failing liveness probe restarts the pod, so nothing outside the process may be able to trigger a restart.
Use this endpoint for:
- Kubernetes
livenessProbeandstartupProbeconfiguration - Docker
HEALTHCHECKinstructions - Load balancer liveness checks in front of multiple gateway instances
GET /readyz#
The readiness endpoint answers a different question: can this instance serve traffic right now? A failing readiness probe removes the pod from the Service and nothing more, so unlike /healthz it is free to check dependencies.
Returns 200 OK normally, and 503 Service Unavailable with the body draining once the instance has been drained from the dashboard’s cluster screen. A drained gateway therefore leaves the Service and finishes its in-flight requests instead of being killed mid-request.
It also returns 503 with the body snapshot_format_unsupported while it has never received a configuration snapshot and the control plane serves no snapshot format it reads. This keeps a gateway that was upgraded ahead of its control plane out of the Service until the control plane catches up. A gateway that already holds a snapshot stays ready on its last config instead. See Upgrading.
Returns a JSON body naming each check, with 200 when ready and 503 otherwise:
| Check | Ready when |
|---|---|
database | The PostgreSQL pool hands out a connection within 2 seconds |
migrations | Every migration embedded in this binary is applied to the database |
kek | ROLTER_KEK parses when it is set (reported not configured when unset, which is not a failure) |
A check that fails reports unavailable rather than the database driver’s error, since this endpoint answers without authentication. The detail is in the control plane’s log.
Redis and ClickHouse are not checked. The control plane serves configuration without either, so their absence is degraded, not unready — removing every control pod from the Service because analytics is down would be a strictly worse outage.
Do not point both probes at the same path. If a single endpoint answers as soon as the server binds, Kubernetes routes traffic at a control plane that has not migrated yet. If it instead checks PostgreSQL, a database blip fails liveness and restarts every control pod at once, turning a recoverable dependency outage into a restart storm.
GET /metrics#
The metrics endpoint serves Prometheus exposition-format text. Scrape it with any Prometheus-compatible collector. No authentication is required.
The default path is /metrics. You can change it in rolter.toml to avoid collisions with an upstream application or sidecar that already owns /metrics behind the same reverse proxy:
The metrics path must start with / and must not collide with any of Rolter’s built-in request paths (/healthz, /v1/models, /v1/chat/completions, /v1/completions, /v1/messages). Rolter validates this at startup and reports an error if a collision is detected.
Key metrics#
The following metrics are included in the Prometheus output. All are counters or histograms scoped to the gateway process.
| Metric | Type | Description |
|---|---|---|
rolter_requests_total | Counter | Total number of inference requests proxied by the gateway |
rolter_request_latency_ms_bucket | Histogram | End-to-end request latency in milliseconds, labelled by model route |
rolter_request_ttft_ms_bucket | Histogram | Time-to-first-token in milliseconds for streaming requests, labelled by model route |
rolter_upstream_errors_total | Counter | Number of requests that exhausted all retries and returned an upstream error |
rolter_auth_failures_total | Counter | Number of authentication failures (missing, invalid, expired, or revoked keys) |
rolter_config_version | Gauge | Monotonically increasing version of the active configuration snapshot; increments on each hot reload |
Additional counters#
| Metric | Description |
|---|---|
rolter_retries_total | Attempts made again after a transient upstream failure (see below) |
rolter_cooldowns_tripped_total | Number of times a target was placed on cooldown after a failure |
rolter_breaker_opened_total | Number of times a circuit breaker tripped open |
rolter_breaker_closed_total | Number of times a circuit breaker closed after a successful probe |
rolter_rate_limit_blocks_total | Requests rejected by an RPM, TPM or route concurrency limit |
rolter_budget_blocks_total | Requests and realtime sessions refused or closed by a spent budget |
rolter_retries_total counts an attempt only when another one follows it: a retryable status (408, 429, 5xx) or a connection error, and a target left to try, whether a sibling target, the same target on another key of a multi-key provider, or the next candidate of a variant. A route with a single target, or the last target of several, has nothing to retry on: its failure is answered to the caller and the counter does not move.
Prometheus scrape config#
Add the following job to your prometheus.yml to scrape the default metrics path:
If you customised metrics_path in rolter.toml, update the metrics_path field in the scrape config to match:
Security note#
Neither /healthz nor /metrics require authentication. In production, restrict access to /metrics at your reverse proxy layer to prevent leaking operational data. A typical nginx rule: