> ## Documentation index > Fetch the complete documentation index at: https://rolter-ai-rolter.docs7.io/llms.txt > Ask it with GET https://rolter-ai-rolter.docs7.io/ask?q=. Short keywords rank best. > Use these to discover all available pages before exploring further. # Realtime > Connect to the OpenAI-compatible Realtime WebSocket relay at /v1/realtime, and how a session is admitted, metered and closed by budgets and rate limits. **Experimental.** `/v1/realtime` carries the `realtime` [stability marker](/api/stability) and may change shape in a minor release. Budgets, rate limits, usage recording, guardrails, the guardrail webhook and `pre_upstream` plugins apply to a session's text events, and guardrails to the arguments of the model's function calls. Plugin calls made on a session reach the request log on its next row, so the Plugins screen counts them. The PII sanitizer and `pre_route` / `post_response` plugins do not run on a session. A `fail_closed` PII sanitizer refuses the session with `sanitizer_unsupported_on_realtime`; under `fail_open` the session is admitted unsanitized. `/v1/realtime` is a WebSocket relay compatible with the OpenAI Realtime API. Rolter authenticates the request and selects an upstream before accepting the client's WebSocket upgrade, then pins that upstream and its provider key for the life of the session. ## Connect The gateway's `GET /openapi.json` describes this upgrade as `GET /v1/realtime`, including its required `model` query parameter, the `101` response and the HTTP admission failures returned before a session is accepted. Send the usual gateway virtual key and the public route model as a query parameter: ```text wss://gateway.example.com/v1/realtime?model=gpt-realtime ``` Authenticate with `Authorization: Bearer ` the same way you would for `/v1/chat/completions`. The `model` value selects a route the same way it does everywhere else in the gateway. ## What is relayed Text, binary audio and WebSocket control frames are relayed in both directions without application-level buffering, including `session.update`, `input_audio_buffer.*`, `response.*` and function-call events. If the upstream connection drops, the client must reconnect. Rolter does not fail a live session over to another target, because replaying audio or tool events mid-session is not safe to do automatically. ## Budgets, rate limits and usage A realtime session meets the same [budgets and rate limits](/security/budgets-and-rate-limits) as an HTTP request, on the same key, project, team and org scope. **Opening a session** is checked before any upstream is dialled. A refused upgrade gets the HTTP status and error body a chat request would: | Refusal | Status | `error.code` | | ---------------------------------------------------------------- | ------ | --------------------- | | a budget in the key's scope is spent | `402` | `insufficient_quota` | | the model has no price and `unpriced_policy` resolves to `block` | `402` | `model_unpriced` | | an `rpm` or `tpm` window is full | `429` | `rate_limit_exceeded` | | the gateway is shutting down | `503` | `null` | Opening a session counts as one request against `rpm`, and a `429` carries `Retry-After`. **While the session runs**, rolter meters it per response. Each `response.done` event reports the tokens that response used, and rolter turns it into one row in the request log, priced and attributed like any other request, so realtime spend appears in [cost tracking](/observability/logs-and-cost) and analytics. Its cost is added to your budgets and its tokens to your `tpm` limits every `usage_flush_secs` seconds (15 by default), not only when the session closes. A session that stays open all day is accounted as it goes. **Audio and text tokens are priced apart.** A `response.done` names the audio share of its input and output tokens, and when the model's price row sets `audio_input_per_mtok`, `audio_output_per_mtok` or `cached_audio_input_per_mtok` those tokens are charged at them; a row without them prices every token at the text rates. See [Audio and text tokens](/observability/logs-and-cost#audio-and-text-tokens). **Input-audio transcription is metered as its own model.** With `input_audio_transcription` on, the upstream runs a second model (`whisper-1`, `gpt-4o-transcribe`) over the caller's audio and reports its usage on `conversation.item.input_audio_transcription.completed`. Each such event becomes one more row in the request log, and its cost is added to your budgets and its tokens to your `tpm` limits with the session's turns. The row's `model` is the transcription model the session named in `session.update` (`session.audio.input.transcription.model`, or `session.input_audio_transcription.model` in the beta shape) and it is priced from that model's own price row: - A model billed by tokens (`gpt-4o-transcribe`) is priced from the event's token usage, with its audio input at `audio_input_per_mtok` when set. - A model billed by the minute (`whisper-1`) is priced from the event's duration at `audio_input_per_minute`. - A model with no price row, a duration with no per-minute rate, or a session that never named a transcription model is logged `unpriced = 1` with no cost. - An event that carries no usage is logged `usage_unknown = 1`, unknown rather than free. A full `tpm` window does not close a live session; it refuses the key's next session or request, as it would for HTTP. **When a budget runs out mid-session**, rolter closes the session. This happens at the next flush after the budget is spent, whether this session or other traffic spent it. The client receives a Realtime `error` event first: ```json { "type": "error", "event_id": "event_rolter_…", "error": { "type": "insufficient_quota", "code": "insufficient_quota", "message": "budget exceeded for Org 'acme' (limit $500.00)", "param": null } } ``` It then receives a close frame with code `1008` (policy violation). A session can overspend by at most one flush window, plus the response in flight when the budget ran out; lower `usage_flush_secs` to tighten that. **When the key is revoked mid-session**, rolter closes the session too. Every flush tick looks the session's virtual key up again in the current configuration. A key that was disabled, has expired or was deleted, a `models` list that no longer names the session's model, a route the key can no longer reach, and a `providers` list that no longer includes the provider the session is connected to all end the session, so revoking access is not left waiting on `max_session_secs`. The client receives a Realtime `error` event with code `invalid_api_key` (key gone), `model_not_allowed`, `route_not_allowed` or `provider_not_allowed`, then a close frame with code `1008`. The bound is one flush interval, `usage_flush_secs` (one second when that is `0`), plus the time for the change to reach the gateway. **When the key is moved or its limits change mid-session**, the same tick applies the change to the open session rather than leaving it on what it was opened with: - Budgets, rate limits and prices edited while a session is open apply from its next tick. A budget lowered below what its scope already spent closes the session with `insufficient_quota`. - A key moved to another team or project in the same organization keeps its session. Later turns are logged and charged under the new team and project, and a spent budget there closes the session. - A key moved to another organization, or to a project whose `pre_upstream` plugins differ from the old one's, closes the session with an `error` event of code `key_scope_changed` and a `1008` close. The client should open a new session, which is admitted under the key's new scope. - A change to any other key leaves the session alone. **A response cut short** (the client leaves, the upstream drops, or the session is closed mid-answer) never reports its usage, although the provider bills what it generated. Its row is written with `usage_unknown = 1` so the gap is visible rather than counted as free. See [Withheld responses](/observability/logs-and-cost#withheld-responses) for how to query it. ## Gateway restarts When a gateway receives `SIGTERM` (a rolling deploy, a scale-down) or Ctrl-C, it closes every live session with WebSocket close code `1001` (going away) and refuses new ones with `503`. Reconnect, and the load balancer in front of the fleet sends you to a gateway that is still running. Before the process exits, each session's usage so far is charged to budgets and `tpm`, including turns still waiting for the next `usage_flush_secs`, and its request-log rows are queued. A response in flight at that moment is logged with status `503` and `usage_unknown = 1`. The gateway waits up to 10 seconds for this, then up to 5 more seconds for the request-log writer, the health-event writer and the budget and `tpm` recording queue to flush what they hold, so give the process a termination grace period of at least 15 seconds on top of your longest HTTP request. If ClickHouse or Redis is unreachable at that moment, the rows and charges still queued are lost after those 5 seconds. ## Current limits - **WebRTC/browser ephemeral-token handoff is not exposed.** Only the WebSocket-first flow is supported today. - **No failover.** A dropped upstream ends the session; there is no session-affinity retry the way HTTP routes get. - **Content policy is narrower than on chat.** Built-in guardrails, the guardrail webhook and `pre_upstream` plugins check client text events (`conversation.item.create`, `session.update` instructions, `response.create`) and, for guardrails, the model's text and transcript output. A blocked event is not forwarded and the client receives an `error` event with code `guardrail_blocked` (or `plugin_blocked`); the session stays open, and a blocked response is cancelled. Output is checked as it streams, so the part of a match before the delta that completes it can already have been delivered. The PII sanitizer and `pre_route`/`post_response` plugins do not run on realtime events (a `fail_closed` sanitizer refuses the session with `sanitizer_unsupported_on_realtime`), audio itself is not inspected, and a binary frame is refused while a policy applies. See [Content policy](https://github.com/rolter-ai/rolter/blob/master/docs/dev-docs/architecture/realtime-metering.md#content-policy-on-a-bidirectional-stream-1880) for the detail. - **Audio rates need the split.** An audio rate applies only to the tokens a `response.done` names as audio. An upstream that reports totals alone is priced at the text rates. - **A transcription model is named by the session.** A session that turns transcription on in its defaults, without a `session.update` or a `session.updated` naming the model, has its transcription logged `unpriced`. ## `[realtime]` configuration The gateway's `[realtime]` block holds per-process resource limits, which bound what one gateway will hold rather than what a tenant may spend, and the metering cadence: - `max_connections` (default `1000`) — concurrent sessions this gateway process admits - `max_session_secs` (default `3600`) — hard session-duration limit - `idle_timeout_secs` (default `300`) — closes a session when neither side sends a frame - `usage_flush_secs` (default `15`) — how often finished responses are logged, charged to budgets and `tpm`, and followed by a budget check. `0` does this after every response instead of on a timer, and re-checks the budgets every second while the session is quiet Set any of the first three to `0` to disable that limit. Metering itself cannot be switched off. See [Configuration](/configuration/config-file) for where `[realtime]` lives in `rolter.toml`.