> ## Documentation index > Fetch the complete documentation index at: https://rolter-ai-rolter.docs7.io/llms.txt > Ask it with GET https://rolter-ai-rolter.docs7.io/ask?q=. Short keywords rank best. > Use these to discover all available pages before exploring further. # Health and Metrics Endpoints for the Rolter Gateway > Use GET /healthz for liveness probes, GET /readyz for readiness probes, and GET /metrics for Prometheus scraping to integrate Rolter into your monitoring and orchestration stack. Rolter exposes three operational endpoints that require no authentication and are designed to be scraped or probed directly by your infrastructure tooling. Use `/healthz` for liveness checks, `/readyz` for readiness checks, and `/metrics` for Prometheus scraping. ## GET /healthz The liveness endpoint returns `200 OK` with the body `ok` when the process is running and its HTTP server is accepting connections. No authentication is required. It checks **no dependency** — not PostgreSQL, not Redis, not ClickHouse, not an upstream provider. That is deliberate: a failing liveness probe restarts the pod, so nothing outside the process may be able to trigger a restart. Use this endpoint for: - **Kubernetes** `livenessProbe` and `startupProbe` configuration - **Docker** `HEALTHCHECK` instructions - **Load balancer** liveness checks in front of multiple gateway instances ```bash curl http://localhost:4000/healthz # → 200 OK # → ok ``` ```yaml Kubernetes liveness probe livenessProbe: httpGet: path: /healthz port: 4000 initialDelaySeconds: 5 periodSeconds: 10 ``` ## GET /readyz The readiness endpoint answers a different question: can _this_ instance serve traffic right now? A failing readiness probe removes the pod from the Service and nothing more, so unlike `/healthz` it is free to check dependencies. Returns `200 OK` normally, and `503 Service Unavailable` with the body `draining` once the instance has been drained from the dashboard's cluster screen. A drained gateway therefore leaves the Service and finishes its in-flight requests instead of being killed mid-request. It also returns `503` with the body `snapshot_format_unsupported` while it has never received a configuration snapshot and the control plane serves no snapshot format it reads. This keeps a gateway that was upgraded ahead of its control plane out of the Service until the control plane catches up. A gateway that already holds a snapshot stays ready on its last config instead. See [Upgrading](/deployment/upgrading#upgrade-order). A gateway started with `--snapshot-url` answers `503` with the body `awaiting_snapshot` until it applies its first snapshot from the control plane, whatever the reason none has arrived yet. Until then it serves only its own config file and refuses every key, so it stays out of the Service. A gateway whose config file carries virtual keys, or sets `server.require_auth = false`, can serve before that and is ready. Once it has applied a snapshot it stays ready when a later poll fails. When both answers apply, `snapshot_format_unsupported` is the one returned, since it says why no snapshot arrived. ```bash curl -i http://localhost:4000/readyz # → 200 OK # → ok ``` Returns a JSON body naming each check, with `200` when ready and `503` otherwise: ```bash curl -s http://localhost:4001/readyz ``` ```json { "status": "ready", "checks": { "database": "ok", "migrations": "ok", "kek": "ok" } } ``` | Check | Ready when | | --- | --- | | `database` | The PostgreSQL pool hands out a connection within 2 seconds | | `migrations` | Every migration embedded in this binary is applied to the database | | `kek` | `ROLTER_KEK` parses when it is set (reported `not configured` when unset, which is not a failure) | A check that fails reports `unavailable` rather than the database driver's error, since this endpoint answers without authentication. The detail is in the control plane's log. Redis and ClickHouse are **not** checked. The control plane serves configuration without either, so their absence is degraded, not unready — removing every control pod from the Service because analytics is down would be a strictly worse outage. ```yaml Kubernetes readiness probe readinessProbe: httpGet: path: /readyz port: 4000 initialDelaySeconds: 3 periodSeconds: 5 ``` Do not point both probes at the same path. If a single endpoint answers as soon as the server binds, Kubernetes routes traffic at a control plane that has not migrated yet. If it instead checks PostgreSQL, a database blip fails *liveness* and restarts every control pod at once, turning a recoverable dependency outage into a restart storm. ## GET /metrics The metrics endpoint serves Prometheus exposition-format text. Scrape it with any Prometheus-compatible collector. No authentication is required. ```bash curl http://localhost:4000/metrics ``` The default path is `/metrics`. You can change it in `rolter.toml` to avoid collisions with an upstream application or sidecar that already owns `/metrics` behind the same reverse proxy: ```toml [server] metrics_path = "/internal/metrics" ``` The metrics path must start with `/` and must not collide with any of Rolter's built-in request paths (`/healthz`, `/v1/models`, `/v1/chat/completions`, `/v1/completions`, `/v1/messages`). Rolter validates this at startup and reports an error if a collision is detected. ## Key metrics The following metrics are included in the Prometheus output. All are counters or histograms scoped to the gateway process. | Metric | Type | Description | | ---------------------------------- | --------- | ---------------------------------------------------------------------------------------------------- | | `rolter_requests_total` | Counter | Total number of inference requests proxied by the gateway | | `rolter_request_latency_ms_bucket` | Histogram | End-to-end request latency in milliseconds, labelled by model route | | `rolter_request_ttft_ms_bucket` | Histogram | Time-to-first-token in milliseconds for streaming requests, labelled by model route | | `rolter_upstream_errors_total` | Counter | Number of requests that exhausted all retries and returned an upstream error | | `rolter_auth_failures_total` | Counter | Number of authentication failures (missing, invalid, expired, or revoked keys) | | `rolter_config_version` | Gauge | Monotonically increasing version of the active configuration snapshot; increments on each hot reload | ### Additional counters | Metric | Description | | -------------------------------- | ------------------------------------------------------------------ | | `rolter_retries_total` | Attempts made again after a transient upstream failure (see below) | | `rolter_cooldowns_tripped_total` | Number of times a target was placed on cooldown after a failure | | `rolter_breaker_opened_total` | Number of times a circuit breaker tripped open | | `rolter_breaker_closed_total` | Number of times a circuit breaker closed after a successful probe | | `rolter_rate_limit_blocks_total` | Requests rejected by an RPM, TPM or route concurrency limit | | `rolter_budget_blocks_total` | Requests and realtime sessions refused or closed by a spent budget | `rolter_retries_total` counts an attempt only when another one follows it: a retryable status (`408`, `429`, `5xx`) or a connection error, and a target left to try, whether a sibling target, the same target on another key of a multi-key provider, or the next candidate of a variant. A route with a single target, or the last target of several, has nothing to retry on: its failure is answered to the caller and the counter does not move. ## Prometheus scrape config Add the following job to your `prometheus.yml` to scrape the default metrics path: ```yaml scrape_configs: - job_name: rolter static_configs: - targets: - "localhost:4000" metrics_path: /metrics ``` If you customised `metrics_path` in `rolter.toml`, update the `metrics_path` field in the scrape config to match: ```yaml scrape_configs: - job_name: rolter static_configs: - targets: - "localhost:4000" metrics_path: /internal/metrics ``` ## Security note Neither `/healthz` nor `/metrics` require authentication. In production, restrict access to `/metrics` at your reverse proxy layer to prevent leaking operational data. A typical nginx rule: ```nginx location /metrics { allow 10.0.0.0/8; deny all; } ```