> ## Documentation index > Fetch the complete documentation index at: https://rolter-ai-rolter.docs7.io/llms.txt > Ask it with GET https://rolter-ai-rolter.docs7.io/ask?q=. Short keywords rank best. > Use these to discover all available pages before exploring further. # Per-Request Logs and Cost Tracking with ClickHouse > Enable ClickHouse logging to record every proxied request with token counts, cost in USD, latency, and error details for analysis and billing. Rolter can write a structured row to ClickHouse for every request it proxies. Each row captures everything you need to audit usage, track spend, and investigate issues — token counts, cost in USD, latency, error details, and the A/B variant that served the request. Writes are asynchronous and batched off the request path, so enabling logging adds zero latency for your users. ClickHouse logging is entirely optional. If you don't need per-request records or cost tracking, you can skip this section and Rolter will still expose Prometheus metrics and traces. ## Enabling logging Set `clickhouse_url` in the `[logging]` section of your config file to point at your ClickHouse HTTP interface: ```toml rolter.toml [logging] clickhouse_url = "http://localhost:8123" ``` Logging is disabled when `clickhouse_url` is unset — there's no placeholder or no-op mode. The dashboard reads the rows back through the control plane, which needs its own `CLICKHOUSE_URL` (gateways with no address of their own inherit it). A control plane without one shows **LLM Logs** as a note that request logs need ClickHouse, **MCP Logs** as a note that tool-call logs need it, and the **Dashboard** as a note that traffic and spend need it, each naming `CLICKHOUSE_URL`. **Cost Attribution** shows the same kind of note in place of its spend figures, and **Account & keys** shows one above your keys (each key reads _usage: unavailable_), while the rest of both screens keeps working. It is information about the deployment rather than an error, so it offers no retry: nothing changes until the setting does. ## What gets logged Every request from a caller the gateway has identified writes one row to the `request_logs` table with the following fields, whatever happened to it: a success, a failure after the upstream errored, or a refusal before any upstream was involved (see [Failed and refused requests](#failed-and-refused-requests)). With a [`sample_rate`](/observability/log-settings#sampling) below `1` only the sampled requests do, each stamped with a `sample_weight`; a raw `count()` or `sum()` over the table shrinks with the rate, and `sum(sample_weight)` or `sum(cost_usd * sample_weight)` restores it. | Field | Type | Description | | ----------------------- | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ts` | DateTime64(3) | When the request **started** — the instant `latency_ms` is measured from | | `request_id` | String | Unique request identifier, echoed in the request id response header (`x-request-id` by default) | | `log_id` | String | A key the gateway mints for each request; it ties a row to its captured body. Empty on rows written before it existed | | `trace_id` | String | The caller's distributed-trace id (W3C `traceparent` or B3); empty when the caller sent none | | `model` | String | The public model name the client requested | | `provider` | String | The upstream provider that served the request (`builtin` for the built-in `fake-llm` model) | | `target` | String | The specific upstream target within the provider | | `status` | UInt16 | HTTP status code returned to the client | | `upstream_status` | UInt16 | HTTP status of the last upstream attempt that answered, `0` when none did (a refusal, a cache hit, a connection that failed) | | `attempts` | UInt8 | Upstream attempts the request made, the one that answered included; `0` when it never reached an upstream | | `prompt_tokens` | UInt32 | Prompt / input token count reported by the upstream, prompt-cache reads and writes included | | `completion_tokens` | UInt32 | Completion / output token count reported by the upstream | | `cost_usd` | Float64 | Computed cost in USD (see [Cost tracking](#cost-tracking)) | | `unpriced` | UInt8 | `1` when the model had no price row, so `cost_usd` is unknown rather than zero (see [Unpriced traffic](#unpriced-traffic)) | | `withheld` | UInt8 | `1` when the provider answered and billed, but a guardrail or plugin refused delivery (see [Withheld responses](#withheld-responses)) | | `usage_unknown` | UInt8 | `1` when the upstream answered successfully but reported no token usage, so the zero tokens are unknown rather than free | | `latency_ms` | UInt32 | End-to-end request latency in milliseconds | | `ttft_ms` | UInt32 | Time-to-first-token in milliseconds (streaming requests) | | `stream` | UInt8 | `1` when the response was streamed | | `cache_hit` | UInt8 | `1` when Rolter answered from its own response cache, so no upstream was called and `provider` and `target` are empty | | `cache_read_tokens` | UInt32 | Prompt tokens the upstream reused from its own prompt cache | | `cache_write_tokens` | UInt32 | Prompt tokens the upstream wrote to its prompt cache for this request | | `cache_write_1h_tokens` | UInt32 | The part of `cache_write_tokens` the upstream wrote to its 1 hour prompt cache (Anthropic only); `0` when the provider reports no split | | `error` | String | Why the request failed, including the upstream's own reason; empty on success (see [Failed and refused requests](#failed-and-refused-requests)) | | `variant` | String | Name of the A/B variant that served the request (blank when not using variants) | | `lifecycle_operation` | String | Which call on a stored response the row records (`retrieve`, `delete`, `cancel`, `input_items`, `compact`, `input_tokens`); empty otherwise | | `fallback_reason` | String | The failure class that sent the request on to the target that answered it (see [Fallback targets](/reliability/fallback-targets)); empty when the first dial answered | | `sample_weight` | Float64 | How many real requests the row stands for: `1 / sample_rate` when it was written, `1` when unsampled or written before the column | The built-in `fake-llm` model is logged like any provider. It has no upstream, so its rows carry `provider = builtin` and `target = fake-llm`, and the token counts are the ones the fake response reports (a streamed chat completion included, though its stream carries no usage object). Its `cost_usd` is exactly `0`, and its rows are never marked `unpriced` or `usage_unknown`: a model that costs nothing is not an accounting gap. They show up on **LLM Logs** and in the analytics screens with no special casing, which makes `fake-llm` a quick way to check that logging works end to end. Requests it refuses, for example a `400` for a missing `input`, are logged with that status. Requests to `fake-llm` are not charged against token-per-minute limits or budgets, and its rows carry no captured payload. ### Failed and refused requests `status` is what the caller was told. When the gateway answers with an error of its own, it does not say which upstream caused it, so two more columns do: `upstream_status` is the HTTP status of the last upstream attempt that answered, and `attempts` is how many upstream attempts the request made. A provider that answers `429` on a route with one target reaches the caller as a `429` with `code` `upstream_rate_limited` and a `Retry-After` when the provider sent one, not as an overload, and its row reads `status = 429`, `upstream_status = 429`, `attempts = 1`. When every target of a route failed with another status, the caller gets a `503` with `code` `upstream_unavailable` that names the last upstream status, and the row carries the same. `error` keeps the upstream's reason, so **LLM Logs** shows a sentence such as `upstream returned 429 after 1 attempt: Provider returned error: google/gemma-4-31b-it:free is temporarily rate-limited upstream`. It is read from the error body the provider sent, reduced to one line of at most 300 characters, with any URL in it masked, and it is never the raw body. A connection that failed before any status line has `upstream_status = 0` and the transport error in `error`. A request the gateway refuses before it reaches an upstream is logged too, with an empty `provider` and `target`, `attempts = 0`, and the message the caller received as `error`: an unknown model (`404`), a model or route the key may not use (`403`), a spent budget or an unpriced model the deployment refuses (`402`), a rate limit (`429`), a request a guardrail, plugin or prompt template rejects (`400`), and a route with no targets (`503`). They are written to the log only; they add no latency or error series to `/metrics`, which have their own counters for these refusals. **LLM Logs** shows the two columns where they explain a status. The detail drawer prints one sentence under the status badge: `Upstream returned 500 after 3 attempts` when an upstream answered, `Upstream did not respond after 2 attempts` when the connection failed before any status line, and `Refused before reaching an upstream` for a request the gateway turned away itself. A request that succeeded on its first attempt gets no sentence, and neither does a cache hit or a `fake-llm` call, which make no attempts either. In the table, a request whose status differs from the upstream's carries the upstream's status beside its own, such as `503` with `500` after it: a `503` the gateway made after every target of a route failed, with the sentence as the tooltip. A provider's error handed to the caller as it came has the same number on both sides, so it gets no hint, and the drawer still says how many attempts it took. A row written before the columns existed has nothing to show and is drawn as it always was. What is **not** logged is anything that happens before the gateway knows who is asking: a body that is not JSON, a request with no `model`, a missing, unknown or expired key, and a request refused for a missing required header or an oversized body. No tenant owns such a row, and anonymous traffic would otherwise be able to write to your log without limit. `rolter_auth_failures_total` counts the rejected keys, and `rolter_rejected_requests_total{reason}` splits them by reason (`missing_key`, `invalid_key`, `required_header_missing`), so "how many 401s, and why" is answerable even though none of them has a row. Everything after the key is resolved is logged, including a budget block, a rate limit and a guardrail refusal: the key bounds the write volume, since an attacker cannot mint valid keys. Rows written before `clickhouse/015_upstream_attempts.sql` default both columns to `0`. #### The outcome of each request Every row also names what happened to the request in its `outcome` column, so a refusal is selected by its reason rather than inferred from an empty provider and a 4xx: | `outcome` | The request | | -------------------- | ------------------------------------------------------------------------------------------------------------------------- | | `served` | got a successful answer, from an upstream, the [response cache](/concepts/response-cache) or the built-in `fake-llm` | | `upstream_error` | reached an upstream and ended in an error: the provider's own, or the gateway's after every attempt failed | | `withheld` | was answered and billed, and a post-call policy withheld the answer (see [Withheld responses](#withheld-responses)) | | `cancelled` | lost its caller before the answer was delivered | | `refused_budget` | was refused before routing because a budget is spent (`402`) | | `refused_rate_limit` | was refused before routing by a rate limit or a route's own cap (`429`) | | `refused_guardrail` | was refused before routing by a guardrail or the PII sanitizer | | `refused_plugin` | was refused before routing by a plugin | | `refused_access` | was refused before routing: an unknown model, or one the key may not use (`403`, `404`) | | `refused_invalid` | was refused before routing because it cannot be served as sent: a missing parameter, an endpoint the route's servers lack | | `refused` | was refused for another reason, or no target was left to try | | `unknown` | was logged before the gateway recorded an outcome, or by a gateway older than the column | The `refused_*` rows are the requests refused after the key resolved, and read as described above: no `provider` or `target` and `attempts = 0`. The gateway picks the reason from the error code it answered with (`insufficient_quota`, `rate_limit_exceeded`, `guardrail_blocked`, `plugin_blocked`, `model_not_found`, ...), and from the status where the answer has no code. New outcomes may be added in a later release; treat a name you do not know as "see `status` and `error`". The column is added by `clickhouse/030_request_outcome.sql`. Every row older than the file reads `unknown`, and a gateway upgraded before you apply it keeps logging and loses only the outcome. Filter by it in **LLM Logs** with **Outcome**, or with `outcome=` on [`GET /api/v1/analytics/invocations`](#paging-the-invocation-log). **Calls on a stored response** are logged too: `GET`, `DELETE`, `cancel` and `input_items` on `/v1/responses/{id}` write one row each, like any other request from a known caller. The row's `model` is the model the response was created for, and its `provider` and `target` are the ones that hold it. `lifecycle_operation` says which call it was. The row has no tokens and no cost, and it is not marked `unpriced` or `usage_unknown`: the call generates nothing, so there is nothing to price. These calls are not billable, so they do not count against a budget, a token limit or a request limit, and they are left out of the per-model latency series on `/metrics`. Failures read as they do for a model request. A provider that answered with an error has its status in `upstream_status` and `attempts = 1`; one that could not be reached is a `502` with `upstream_status = 0`. A call the gateway turned away itself has an empty `provider` and `target` and `attempts = 0`: an id that does not exist, has expired or belongs to another tenant (`404`, with no `model` since there is no stored response to take one from), access that was revoked since the response was created (`403`), or an operation the response's provider has no API for (`501`). **LLM Logs** shows which call it was. In the table a small outline label with the operation sits beside the model name: **Retrieve**, **Delete**, **Cancel**, **Input items**, **Compact** or **Count input tokens**. Hover it for the sentence, such as `Call on a stored response: Retrieve`. The detail drawer starts its **Routing** group with the same words under **Stored response call**. A call on an id that does not exist has no model, so the label stands alone in the model column and names the row in the drawer's title. A request that ran a model, and a row written before the column, show neither the label nor the drawer row. Rows written before `clickhouse/016_lifecycle_operation.sql` have an empty `lifecycle_operation`, as does every row of a request that ran a model. `GET /api/v1/analytics/invocations` returns the column on every row, as an empty string in those two cases, so an older row needs no special case. On a ClickHouse that has not had `016` applied the column reads as an empty string as well, so the file is not a condition of upgrading the control plane (see [Upgrading](/deployment/upgrading)). The API has no filter on the operation: look a call up by its `request_id`, or read the column in ClickHouse. A [realtime](/api/realtime) session writes one row per model response rather than one per session: `request_id` is the upgrade request's id with the response's ordinal appended (`:1`, `:2`, ...), all rows of a session share its `trace_id`, and `latency_ms` and `ttft_ms` describe that one response. Rows arrive every `[realtime] usage_flush_secs` while the session is open. A response the session ended before it finished is logged with `usage_unknown = 1`. `ts` is the moment the request began, so `ts + latency_ms` is when it finished. Rows written by a Rolter older than 1.0 have no `ts` of their own and fall back to the ClickHouse column default, which records the moment their **batch was flushed** — in that data a burst of requests shares one millisecond and ordering inside a batch is arbitrary. Only rows written by a current gateway carry the true request time. `ts` is stored as `DateTime64(3)`, so requests that genuinely begin inside the same millisecond still share a timestamp. `seq` tells such requests apart: each row also carries the gateway's start sequence, and `GET /api/v1/analytics/invocations` orders by `ts desc, seq desc, request_id desc`, so a tied pair keeps a stable, repeatable order, which is what stops a row appearing on two pages or on none. `seq` is the number of nanoseconds on the gateway process's monotonic clock between its start and the request's, counted from 1, so within one gateway instance a larger `seq` is a later start. It exists only per instance: two gateways' sequences are unrelated, and a restart begins again near 1. Two requests in the same millisecond are therefore ordered by when they began only when one gateway served both. Across gateways their order is stable and repeatable but not chronological. `ts` and the table's sorting key are unchanged, and no existing row is rewritten: rows older than the column, and rows from a gateway that predates it, have `seq` `0`. The column comes from `clickhouse/021_request_log_seq.sql` ([Upgrading](/deployment/upgrading)). The list works before you apply it, reading `seq` as `0`. The next page cursor now reads `timestamp|seq|request_id`, and a cursor handed out before the upgrade still resumes, at worst repeating rows of its one millisecond. The request log can also be downloaded as a CSV ([CSV exports](/api/dashboard-analytics#csv-exports)). ## Configuration reference All `[logging]` fields except `clickhouse_url` have sensible defaults and are optional: ```toml rolter.toml [logging] clickhouse_url = "http://localhost:8123" # required to enable logging # flush when batch reaches this many rows (default: 1000) batch_max = 1000 # also flush partial batches at least this often, in milliseconds (default: 1000) flush_ms = 1000 # bounded queue capacity; rows are dropped and counted when full (default: 10000) queue_capacity = 10000 ``` **Type:** String · **Default:** unset (logging disabled) Base URL of your ClickHouse HTTP interface, e.g. `http://clickhouse:8123`. Rolter uses the HTTP interface — not the native TCP protocol — so no special driver is required. Logging is disabled entirely when this is unset. **Type:** Integer · **Default:** `1000` Rolter accumulates log rows in an in-memory batch and flushes to ClickHouse once the batch reaches this size. Larger values reduce ClickHouse insert pressure but increase memory usage and the number of rows lost if the gateway crashes before a flush. **Type:** Integer · **Default:** `1000` Maximum time in milliseconds between flushes. Even if `batch_max` hasn't been reached, a partial batch is flushed at least this often. At the default value of `1000`, log rows appear in ClickHouse within about a second of the request completing. **Type:** Integer · **Default:** `10000` The bounded in-memory queue between the request path and the background log writer. When the queue is full (e.g. ClickHouse is slow or unreachable), incoming rows are dropped rather than blocking the request. Dropped rows are counted by the `rolter_log_dropped_total` Prometheus counter so you can alert on them. ## Cost tracking The `cost_usd` field on each log row is computed from the `[[model_prices]]` entries in your config. You define the input and output price per million tokens for each model, and Rolter calculates the cost automatically: ```toml rolter.toml [[model_prices]] model = "gpt-4o" input_per_mtok = 2.5 output_per_mtok = 10.0 cached_input_per_mtok = 1.25 # optional; defaults to input_per_mtok when absent cache_write_per_mtok = 3.125 # optional; defaults to input_per_mtok when absent ``` Add a `[[model_prices]]` block for each model you want to track. You can define prices for managed models (OpenAI, Anthropic) and self-hosted models alike. After each completed request, Rolter multiplies the upstream-reported token counts by your configured rates. Cached input tokens use `cached_input_per_mtok` when set, falling back to `input_per_mtok`. Providers name the cached share differently; every spelling in [Cached prompts and thinking tokens](#cached-prompts-and-thinking-tokens) is read, on buffered and streamed answers alike. The same holds when the request crosses dialects (a Messages client served by a Chat Completions provider, a chat or Responses client served by an Anthropic or Gemini one): Rolter carries the cached share across, so the discount applies whichever dialect the client spoke. Anthropic reports its cache reads and writes beside `input_tokens`, not inside it, so Rolter adds them into `prompt_tokens`. Tokens written to the cache use `cache_write_per_mtok` when set, falling back to `input_per_mtok`; see [Cache writes](#cache-writes). Rows without a matching model price record `cost_usd = 0` **and `unpriced = 1`** — they still appear in logs, they just don't contribute to spend reports until you add a price. A rate is a number from `0` up to `999999.999999`, the most a price column holds. A negative rate would turn a request into a credit against its budget, so `PUT /api/v1/model-prices` refuses one, and `NaN`, infinity and a rate of a million or more, with a `400` that names the field in `error.field`. A `rolter.toml` with a negative rate fails its startup check. A price stored with a bad rate by an older release is left out of the config snapshot and reported by `GET /api/v1/config/problems`, so its model is **unpriced** rather than priced at zero; see [Upgrading](/deployment/upgrading#model-price-rates) for how to find and correct the row. ### Cached prompts and thinking tokens Rolter reads the prompt-cache hit from the field the provider reports it in, and records it as `cache_read_tokens`. Every spelling below counts the hit **inside** the prompt total except Anthropic's, which Rolter adds back in, so `prompt_tokens` is always the whole prompt and the cached share is priced at `cached_input_per_mtok`: | Provider | Where the cache hit is reported | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ | | OpenAI, Azure OpenAI, OpenRouter, Groq, xAI, Qwen, Z.ai/GLM, Mistral, Together, Fireworks, MiniMax, Doubao, Perplexity, Gemini (OpenAI-compatible endpoint), Meta Llama API, Nebius, Sarvam, Hunyuan, YandexGPT | `usage.prompt_tokens_details.cached_tokens` (Chat Completions) | | OpenAI, Bedrock's OpenAI models, Meta Llama API and Sarvam (Responses API) | `usage.input_tokens_details.cached_tokens` | | Ollama, llama.cpp, vLLM | `usage.prompt_tokens_details.cached_tokens`, when the server reports one | | Anthropic | `usage.cache_read_input_tokens`, beside `input_tokens` rather than inside it | | Fireworks (dedicated deployments) | the `fireworks-cached-prompt-tokens` response header, read when the body states none | | Databricks-hosted Claude | `usage.cache_read_input_tokens`, at the top of `usage` beside `prompt_tokens` | | DeepSeek | `usage.prompt_cache_hit_tokens` (`prompt_tokens` is the hits plus the misses) | | Kimi (Moonshot), some Together models | `usage.cached_tokens`, at the top of `usage` | | GigaChat | `usage.precached_prompt_tokens` | | Gemini (native `generateContent`), Gemini Interactions | `cachedContentTokenCount`, `total_cached_tokens`; carried across by the translator | A provider that caches but is not in the table logs `cache_read_tokens = 0` and is priced as fresh input until its field is added, so report the provider if you run into one. The table lists the providers whose documentation, or a response someone captured, names the field, not every provider that caches. Google documents no `usage` block for its OpenAI-compatible endpoints; Gemini's is in the table from a captured response, but no response with a cache hit was found for Vertex AI's, and Cohere's, Upstage's, OVHcloud's, Scaleway's and Ernie's compatible endpoints are not documented either, so a cache hit from any of them is read only if it uses one of the spellings above. Doubao's, Hunyuan's and Gemini's compatible endpoints are listed on the strength of the shape they copy, not of a primary source or a captured hit, so check `cache_read_tokens` on a request you know repeats a long prompt before relying on it. Tokens a provider **wrote** to its cache are recorded as `cache_write_tokens`, and are part of `prompt_tokens`. Rolter reads Anthropic's `usage.cache_creation_input_tokens` and the OpenAI-shaped spellings `usage.prompt_tokens_details.cache_write_tokens` (OpenRouter, Perplexity), `usage.input_tokens_details.cache_write_tokens` (Bedrock's OpenAI models), `usage.prompt_tokens_details.cache_creation_input_tokens` (Qwen's explicit cache) and `usage.prompt_tokens_details.created_cache_tokens` (vLLM). They are priced at `cache_write_per_mtok`, and the share Anthropic reports as written to its 1 hour cache at `cache_write_1h_per_mtok` (see [Cache writes](#cache-writes)). Gemini bills its **thinking tokens as output**, and reports them beside the answer's count, not inside it. Rolter adds `thoughtsTokenCount` (`total_thought_tokens` on the Interactions API) to `completion_tokens`, so the request log, the budget and the `usage` block your client reads all count what the provider charged. The tokens a built-in tool such as URL context fed back to the model (`toolUsePromptTokenCount`, `total_tool_use_tokens`) are charged as input and are added to `prompt_tokens`. The `usage` Rolter shows a client always adds up: `total_tokens` is `prompt_tokens` plus `completion_tokens`. The thinking share is **part of** `completion_tokens`, never on top of it, and a chat or Responses client served by a thinking model is shown how much of the completion it was: `usage.completion_tokens_details.reasoning_tokens` on Chat Completions, `usage.output_tokens_details.reasoning_tokens` on the Responses API. This is OpenAI's own convention (its `completion_tokens` includes the reasoning tokens, `reasoning_tokens` is the breakdown, and `total_tokens` is the prompt plus the completion), and it holds for every provider: Together, Fireworks, Doubao, Perplexity, Upstage, vLLM and OpenAI state it that way themselves, and for the three that count their thinking _beside_ the completion (xAI, Vertex AI and Gemini, below) Rolter moves it inside before the client sees it. A Messages client has no such field and reads the completion alone, and an answer whose provider reported no thinking states none. xAI reports its reasoning tokens beside `completion_tokens`, with only `total_tokens` adding them up (its API reference prints 32 + 9 + 110 = 151); Rolter sees that from the total and adds them to the completion, which is how xAI bills them. Google does not document the `usage` of its OpenAI-compatible endpoints, but responses captured from them agree: **`completion_tokens` leaves the thinking out**. Gemini's endpoint (provider `kind = "gemini"`, `generativelanguage.googleapis.com/v1beta/openai`) has no field for it, so a thinking model's `total_tokens` is more than `prompt_tokens` plus `completion_tokens` by exactly the thinking tokens; Vertex AI's endpoint names them in `usage.completion_tokens_details.reasoning_tokens` beside the same short `completion_tokens`, and its total adds the three up. Both are billed as output, so Rolter adds them to the completion in the request log, the cost and the budget spend: for Gemini what the total has beyond the prompt and the completion, for Vertex AI and xAI the `reasoning_tokens` they name. Only the `gemini` kind is read from the total alone, because no other provider's total is known to mean that; every other kind has only a reasoning count it names itself added. **What a client is shown.** One rule for every provider and every dialect: the thinking is inside the completion and named, and `total_tokens` is unchanged. A Chat Completions client of the `gemini`, `vertex` or `xai` kind gets `completion_tokens` with the thinking in it, `completion_tokens_details.reasoning_tokens` naming it, and the provider's own `total_tokens`, so `prompt_tokens + completion_tokens == total_tokens` and what the client adds up matches what Rolter bills. A Messages client gets `output_tokens` with the thinking in it, and a Responses client `output_tokens` plus `output_tokens_details.reasoning_tokens`. The rest of a Chat Completions response, streamed or not, is the provider's own; only an answer or stream event whose `usage` has thinking beside the completion is rewritten, and an event with no usage is forwarded as it came. Nothing is added when the total is no more than the prompt and the completion (a model that did not think, or an endpoint that later counts thinking in `completion_tokens` itself), so an answer that already follows the rule is never changed and never counted twice. ### Cache writes Writing a prompt into a provider's cache can cost more than reading it as ordinary input: Anthropic charges a cache write at a multiple of the input rate (1.25 times for its 5 minute cache), and a higher one for its 1 hour cache (2 times). Rolter records the count as `cache_write_tokens` and prices it at `cache_write_per_mtok`, with a separate `cache_write_1h_per_mtok` for the 1 hour cache: ```toml rolter.toml [[model_prices]] model = "claude" input_per_mtok = 3.0 output_per_mtok = 15.0 cached_input_per_mtok = 0.3 cache_write_per_mtok = 3.75 cache_write_1h_per_mtok = 6.0 ``` The prompt total holds every token class once, so each token is charged at one rate: the cache reads at `cached_input_per_mtok`, the cache writes at `cache_write_per_mtok` (or `cache_write_1h_per_mtok` for those written to the 1 hour cache), and the rest of the prompt at `input_per_mtok`. A prompt of 2000 tokens that read 1500 from the cache and wrote 100 to it costs the 400 remaining tokens at the input rate, the 1500 at the cached rate and the 100 at the write rate. If a provider reports more cache tokens than the prompt holds, the reads are charged first and the writes take what is left, so a token is never charged twice. - **Absent means the input rate.** A price without `cache_write_per_mtok` charges written tokens as ordinary input, exactly as every price did before the field existed, so upgrading changes no existing cost. - **A separate rate for the 1 hour cache.** Anthropic reports a write to each cache apart, in `usage.cache_creation` (`ephemeral_5m_input_tokens` and `ephemeral_1h_input_tokens`), beside the combined `cache_creation_input_tokens` ([Anthropic's Messages API reference](https://platform.claude.com/docs/en/api/messages), [prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching)). Rolter keeps `cache_write_tokens` as the total and records the 1 hour part as `cache_write_1h_tokens`, so the 5 minute part is the difference. The 1 hour part is charged at `cache_write_1h_per_mtok`, which falls back to `cache_write_per_mtok` and from there to the input rate. A price that sets only `cache_write_per_mtok` charges both caches at it, exactly as before the field existed. A 100-token write that put 60 tokens in the 5 minute cache and 40 in the 1 hour cache costs 60 at `cache_write_per_mtok` and 40 at `cache_write_1h_per_mtok`, and the two parts are never charged on top of the total. - **Which providers report the split.** Only Anthropic's Messages API (a provider of kind `anthropic`, including one behind your own `api_base` that passes Anthropic's `usage` through). It sends the split with the first event of a stream, so a streamed answer is split too. No other provider listed under [Cached prompts and thinking tokens](#cached-prompts-and-thinking-tokens) documents one: their writes are all priced at `cache_write_per_mtok`, whatever `cache_write_1h_per_mtok` says, and their rows log `cache_write_1h_tokens = 0`. A Chat Completions or Responses client served by an Anthropic provider is shown the split as `cache_write_1h_tokens` beside `cache_write_tokens` in `prompt_tokens_details` / `input_tokens_details`, and a Messages client served by a provider that stated one is shown Anthropic's own `cache_creation` breakdown. - **Apply the ClickHouse migration to keep the share.** `cache_write_1h_tokens` is the column `clickhouse/017_cache_write_1h.sql` adds to `request_logs`. A gateway that is upgraded before you apply it keeps logging and prices the request correctly, but cannot store the share: ClickHouse drops the field it has no column for, so only the requests that wrote to the 1 hour cache lose it, and the row's `cost_usd` is unaffected. Rows older than the column read `0`. `GET /api/v1/analytics/invocations` returns the share as `cache_write_1h_tokens` on every row, a number beside `cache_write_tokens` (see [Optional columns in the invocation list](#optional-columns-in-the-invocation-list)); the 5 minute share is the difference, `cache_write_tokens - cache_write_1h_tokens`. The row detail in **LLM Logs** shows the split as **Cache write by lifetime**, the 5 minute and the 1 hour share side by side, beside the cache read and write figures. It appears only for a request whose `cache_write_1h_tokens` is above `0`, so a request that wrote to the 5 minute cache only, one from a provider that reports no split, and a row older than the file show just the total. Applying the file is not a condition of upgrading the control plane (see [Upgrading](/deployment/upgrading)). - **Only reported writes are priced.** OpenAI and Gemini report no write count (their caching is not billed per write), so their rows log `cache_write_tokens = 0` and the field has no effect on them. A write count reaches Rolter from Anthropic's `cache_creation_input_tokens` and from the OpenAI-shaped spellings listed under [Cached prompts and thinking tokens](#cached-prompts-and-thinking-tokens): `prompt_tokens_details.cache_write_tokens` is OpenRouter's and the one Rolter uses when it translates an Anthropic answer for a Chat Completions or Responses client, and Qwen and vLLM spell theirs differently. - **Setting it in the dashboard.** The **Pricing** screen has a _5 minute cache-write price per Mtok (optional)_ field and a _1 hour cache-write price per Mtok (optional)_ field beside the cached-input one, and a price that sets either shows a `5 minute cache write` or `1 hour cache write` badge on its card. The first field is `cache_write_per_mtok`: Anthropic's 5 minute cache, and the writes of any other provider that reports a write count. The second is `cache_write_1h_per_mtok`, which only an Anthropic provider uses. The **5 minute cache-write** and **1 hour cache-write** inputs in a model's **Pricing override** section edit the same two rates; they are saved with the price, so they need the input or output rate too. Leave the 5 minute field empty to price writes at the input rate, and the 1 hour field empty to price those writes at the 5 minute rate. Emptying a rate that was set clears it, and saving a price without touching a field keeps the rate the price holds, so a rate set over the API or by an import survives an edit of something else. A rate the control plane refuses (one above 999999.999999) is shown at its own field. Earlier releases showed the model sheet's input but stored its value on the route, where nothing read it: a rate entered there never changed a cost, and the field now reads and writes the price instead, so enter it again if you relied on it. - **Setting it over the API.** The rates are stored with the rest of the price row as `cache_write_per_mtok` and `cache_write_1h_per_mtok`. A `PUT /api/v1/model-prices` that leaves either field out keeps the stored rate, so a client that has never heard of it cannot reset one. `null` clears `cache_write_per_mtok` back to the input rate and `cache_write_1h_per_mtok` back to `cache_write_per_mtok`. This differs from the other rates, which a `PUT` replaces. Re-importing a `rolter.toml` with `rolter-seed --import` follows the file instead: a price without the key has no write rate afterwards. ### Images, audio and speech Some models are billed by something other than tokens: an image by the image, a transcription by the minute of audio, a text-to-speech request by the character of input. A price row carries a rate for each, in the row's `currency`, beside the token rates: ```toml rolter.toml [[model_prices]] model = "dall-e-3" input_per_mtok = 0 output_per_mtok = 0 image_per_unit = 0.04 [[model_prices]] model = "whisper-1" input_per_mtok = 0 output_per_mtok = 0 audio_input_per_minute = 0.006 [[model_prices]] model = "tts-1" input_per_mtok = 0 output_per_mtok = 0 speech_per_mchar = 15.0 ``` The rates above are what OpenAI's [pricing page](https://openai.com/api/pricing/) lists for those three models. They are examples: Rolter ships no price list, so check your provider's before relying on one. Which rate a request uses follows its endpoint, and the count comes from the request or the response: | Endpoint | Rate | Count | | ---------------------------------------------------- | ------------------------ | -------------------------------------------------------------------------------------------------------------------- | | `/v1/images/generations` | `image_per_unit` | the length of the response's `data` array; the request's `n` (default 1) when the response has none | | `/v1/audio/transcriptions`, `/v1/audio/translations` | `audio_input_per_minute` | the duration the response states, in seconds: `duration` of a `verbose_json` answer, or a `usage` of type `duration` | | `/v1/audio/speech` | `speech_per_mchar` | the characters (not bytes) of the request's `input` | How it combines and what it records: - **Added to the token cost.** A model that reports tokens as well (`gpt-image-1` does) is charged both: the tokens at the token rates and the images at `image_per_unit`. Set only the rate your provider actually bills; leave the other unset. A unit rate never replaces a token rate and a token rate never replaces a unit rate. - **Token rates are still required.** A price row must name `input_per_mtok` and `output_per_mtok`, so a model billed only by the unit takes `0` for both. Zero is a rate, not a missing one. - **Unset means `unpriced`.** An image, transcription or speech request that reports no tokens on a price with no rate for its unit is logged `unpriced = 1` with `cost_usd = 0`, the same as a model with no price row. Before this, such a request on a row that priced only tokens was logged as priced and free. - **A duration that is not stated is unknown, not free.** A transcription returned as `json`, `text`, `srt` or `vtt` carries no duration. With `audio_input_per_minute` set, the row is logged `usage_unknown = 1` and `cost_usd = 0`. Ask for `verbose_json`, or use a model that reports its audio duration in `usage`, to have it priced. - **Failed requests cost nothing.** A request the upstream refused records no unit cost. A response a guardrail or plugin withheld after the provider produced it is still billed: images by `n`, speech by its input, a transcription by the duration the provider stated, read before the answer was withheld. - **Budgets count it.** `cost_usd` is the figure budgets add, so unit-priced requests advance an org, team, project, key, business-unit or customer budget exactly as token-priced ones do, and a rate in another currency is converted to the base first. - **Setting it.** The **Pricing** screen has an _image_, an _audio minute_ and a _speech character_ field, and a price that sets one shows a badge on its card. Over the API the rates are `image_per_unit`, `audio_input_per_minute` and `speech_per_mchar` on `PUT /api/v1/model-prices`, with the same rule as the cache-write rates: a body that leaves one out keeps the stored rate, `null` clears it. The **Pricing override** section of an image model's sheet edits the image rate, and that of an audio model's sheet the audio minute and speech character rates. Saving the sheet without touching one keeps the stored rate, and emptying one clears it. ### Audio and text tokens A realtime or audio model bills the audio tokens it is sent and speaks at a different rate from its text tokens, and its usage says how many of each. A price row carries up to three audio rates beside the text ones, per million tokens: ```toml rolter.toml [[model_prices]] model = "gpt-realtime" input_per_mtok = 4.0 output_per_mtok = 16.0 cached_input_per_mtok = 0.4 audio_input_per_mtok = 32.0 audio_output_per_mtok = 64.0 cached_audio_input_per_mtok = 0.4 ``` These figures are illustrative. Rolter ships no price list, so take the rates from your provider's pricing page. Where the split comes from, and what it applies to: | Source | Audio input | Audio output | Cached audio input | | -------------------------------------------------------------------------- | ------------------------------------------ | ---------------------------------------------- | -------------------------------------------------------------- | | A realtime `response.done` | `usage.input_token_details.audio_tokens` | `usage.output_token_details.audio_tokens` | `usage.input_token_details.cached_tokens_details.audio_tokens` | | Chat Completions with an audio model | `usage.prompt_tokens_details.audio_tokens` | `usage.completion_tokens_details.audio_tokens` | not reported | | `/v1/audio/transcriptions` with a token-billed model (`gpt-4o-transcribe`) | `usage.input_token_details.audio_tokens` | not reported | not reported | Each count is a share of the matching total (`input_tokens`, `output_tokens`), so a token is priced once: - **Unset falls back to the text rate.** A row with no audio rate prices every token at `input_per_mtok` or `output_per_mtok`, exactly as before the audio rates existed. A response that names no audio share has nothing to apply them to, whatever the row sets. - **Cached audio has its own fallback.** A cached audio token is charged at `cached_audio_input_per_mtok`, then at `cached_input_per_mtok`, then at the audio input rate. The cache hits are split into audio and text only when the provider names the audio share; an unsplit hit is read as text. - **Inconsistent counts are clamped.** An audio count larger than the total it belongs to is capped at that total, so a response is never charged for more tokens than it reported. - **One cost.** The figure on the request-log row is the one budgets and `tpm` windows see; there is no second audio-only cost. - **Setting it.** The **Pricing** screen has an _audio input_, an _audio output_ and a _cached audio input_ field, and so does the **Pricing override** section of a model whose type is _audio_, under the same names; a field left empty there says which text rate it falls back to, and saving the model without touching one keeps the rate the price holds. Over the API they are `audio_input_per_mtok`, `audio_output_per_mtok` and `cached_audio_input_per_mtok` on `PUT /api/v1/model-prices`, and they keep the rule the cache-write rates keep: a body that leaves one out keeps the stored rate, `null` clears it, and a number sets it. For how a realtime session is metered, including its input-audio transcription, see [Realtime](/api/realtime). ### Prices are yours, not ours Rolter fetches no pricing data at runtime, ever. Every rate comes from your `[[model_prices]]` entries, from the dashboard's **Pricing** screen or the **Pricing override** section of a model's sheet (which write the same rows), or from an explicit [catalogue import](#importing-catalogue-prices). That is deliberate: published list prices go stale, and most deployments have a negotiated rate, a committed-use discount or a self-hosted model whose real cost is a GPU bill. Rolter multiplies the tokens it counted by the number it holds for the model. The dashboard's pricing section links here for that reason — it is the answer to "where does this number come from": from whoever typed it, or from a catalogue snapshot you imported on purpose. #### Importing catalogue prices To save typing the list price of a model, you can import it from a catalogue. The control plane carries a **vendored snapshot** of [LiteLLM's `model_prices_and_context_window.json`](https://github.com/BerriAI/litellm) (MIT, notice shipped beside it), cut down to the per-token input, output, cache read and cache write costs of the chat, completion, responses and embedding models. It is compiled into the binary, so importing works with no network, and the snapshot is refreshed by a maintainer running `just refresh-prices`, never by your deployment. - `GET /api/v1/model-prices/catalogue?q=gpt-4o` searches it and states its `source`, `licence`, `currency` and `snapshot` date. - `POST /api/v1/model-prices/import` with `{"models": ["gpt-4o", …]}` (at most 500 ids; superadmin, like any price write) creates the prices. Each result is `created`, `updated`, `kept_override` or `not_in_catalogue`. In the dashboard, **Pricing → Import from catalogue** (superadmin) opens a searchable list of the snapshot's models with their rates, the snapshot date and the source and licence. Pick the models and import them: the result states, per model, whether it was created, refreshed from the catalogue, kept as your own price or missing from the catalogue. The Source column of the Pricing screen marks each price as a catalogue value or as your override. Three rules hold for every import: - **Your price wins.** A price you set by hand (the Pricing screen, the API, `[[model_prices]]`) is an _override_: an import reports it as `kept_override` and does not touch it. Editing an imported price in the dashboard turns it into an override. To go back to the catalogue value, delete your price and import the model again. - **Catalogue prices are marked.** An imported row carries its `source` and the date of the snapshot it came from (`source_imported_at`), and the dashboard shows both on the Pricing screen and in the model sheet. - **They are list prices, subject to change.** The file is a third party's best effort and drifts from the provider's real price list, and the choice of LiteLLM as the source is itself provisional. Check an imported rate against your invoice before you rely on it for budgets. The file states no currency; the importer states **USD** explicitly, so an import needs USD to be convertible in your deployment (it is the default base currency). ### Cache hits and overall p95 `GET /api/v1/analytics/summary` also reports `cache_hit_requests`, the requests answered from rolter's response cache (a provider's prompt cache is not counted here; it shows in each row's `cache_read_tokens`), `cache_read_tokens`, the prompt tokens the provider served from its own prompt cache, beside `prompt_tokens` (a read share is the first over the second, and both are scaled by the sampling weight), and `p95_latency_ms`, the 95th percentile of latency over the whole window. The hit count is scaled by the log's sampling weight like every other count, so the response-cache hit rate is `cache_hit_requests / requests`; the percentile is read from the sampled rows as they are, since a uniform sample already estimates it. The dashboard home shows both. ### Withheld and unknown-usage traffic Two more flags keep a total honest. A **withheld** response is one the provider answered and billed but an output guardrail or a `post_response` plugin refused to deliver: the caller got a 403, and the cost is still in `cost_usd`. **Unknown usage** is the other way round: the upstream answered successfully but reported no token counts, so a zero there means "not reported", not "free". `GET /api/v1/analytics/summary` and `GET /api/v1/analytics/by-model` report `withheld_requests`, `withheld_cost` (the part of `cost_usd` that bought an answer nobody saw) and `usage_unknown_requests`, scaled by the log's sampling weight like every other count. Every row from `GET /api/v1/analytics/invocations` carries `withheld` and `usage_unknown` as `0` or `1`. The **Dashboard** says so whenever either count is non-zero in the window: a notice gives the withheld requests and the part of the spend they account for, and another gives the successful requests with no reported usage. Each links to **LLM Logs** over the same window and filters, narrowed to those rows with `withheld=true` or `usage_unknown=true` on `GET /api/v1/analytics/invocations`. The server applies both flags before the page is cut, inside the caller's row scope, so a page of 50 is 50 matching rows. **LLM Logs** marks a withheld row with its 403, the billed cost and the policy named in `error`, and shows a muted dash with an explanation instead of `0` for a row whose usage is unknown, the way it does for an unpriced one. ### Unpriced traffic A zero `cost_usd` is ambiguous on its own: it means either "this request cost nothing" or "we have no price for this model". The `unpriced` column tells them apart, so a spend total can say when it is incomplete rather than presenting a partial figure as final. ```sql -- how much of a window's spend is missing, and which models are responsible select countIf(unpriced = 1) as unpriced_requests, uniqIf(model, unpriced = 1) as unpriced_models, round(sum(cost_usd), 6) as known_spend from request_logs where ts >= now() - interval 24 hour; ``` The dashboard reports the same two figures on `GET /api/v1/analytics/summary` (`unpriced_requests`, `unpriced_models`) and `GET /api/v1/analytics/by-model` (`unpriced_requests`), and shows a notice above the spend tile whenever a window contains unpriced traffic. The flag is exposed **per invocation** too, not only in those summaries: every row from `GET /api/v1/analytics/invocations` carries `unpriced` beside its `cost_usd`, so a single request can be read the same way the totals are. The **LLM Logs** screen does the same per request: a row whose model has no price shows an **unpriced** label in the cost column instead of `$0.0000`, because a zero there claims the request was free and a dash reads as a missing value. The table explains the label on hover, and the detail drawer prints the explanation as a sentence under it. It reads the row's own `unpriced` flag to decide, which is the judgement the gateway made against the price catalogue **as it was when the request was served**. That matters once prices change: a request served before you priced a model stays unpriced in the log even after you add the price, and one served under a price you later delete keeps the cost it was actually charged. The notice links to **Model Prices**, which lists the models that carried unpriced traffic in the last 7 days, most requests first, each with its request count and an **Add price** button that opens the price form with the model filled in. It reads `unpriced_requests` from `GET /api/v1/analytics/by-model`, scaled by each row's sample weight when the request log is sampled, and leaves out a model that has a price now. The list is absent when nothing is unpriced, and on a deployment without ClickHouse the screen says it cannot name them instead of showing an empty list. The price catalogue below it is a table whose price columns sort and whose model column filters by name; figures follow the dashboard language. The **Models** screen names the cause. A model with no price row shows **no price set** in the in/out price column instead of a dash, and an **Unpriced** filter chip above the table counts them and narrows the list to exactly those models — so the fix for an incomplete spend total is one click from the number that reported it. If the price catalogue itself cannot be read, the column falls back to plain figures rather than reporting every model as unpriced on no evidence. Unpriced traffic accrues **no spend**, so it cannot exceed a budget no matter how much is served — budgets simply do not apply to a model you have not priced. `unpriced_policy` decides what to do about that: ```toml unpriced_policy = "block" # ignore (default) | warn | block ``` | policy | behaviour | | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ignore` | Serve it and accrue nothing. Budgets silently do not cover it. | | `warn` | Serve it, but log the model once per hour and treat any budget covering the window as an incomplete accounting. | | `block` | Refuse at admission with `402` (`model_unpriced`), naming the model. Nothing is forwarded, so no upstream tokens are spent on traffic that would never have been counted. | `ignore` is the default so that upgrading cannot start refusing traffic for a deployment that has not priced its models yet — which, on a fresh install, is all of them. Once your price catalogue is complete, `block` costs nothing and closes the gap; the [unpriced-request count](#unpriced-traffic) above is how you tell when you are there. The built-in `fake-llm` model is answered by the gateway itself, before any price is looked up, so `block` never takes away the model you smoke-test a deployment with. Two Prometheus counters track this: `rolter_unpriced_served_total` (requests served with no price — non-zero means your spend figures are a floor) and `rolter_unpriced_blocks_total` (requests refused by `block`). Rows written before the `unpriced` column existed default to `0`. That under-reports the unpriced share of historical data rather than inventing one. ### Withheld responses An output [guardrail](/security/guardrails) with `block`, or a `post_response` [plugin](/configuration/plugins) that blocks, runs **after** the provider has generated the answer. The caller gets a `403` and never sees the answer. The provider still charged for it, so Rolter bills it anyway: - The row's `status` is `403` (what the caller received) and `withheld` is `1`. - `prompt_tokens`, `completion_tokens` and `cost_usd` are the provider's figures, and they count toward budgets and token rate limits like any delivered request. - `error` names the policy, either `guardrail_blocked: ` or `plugin_blocked: `. The rejected answer itself is never stored, even with payload capture on. Usage is read from the provider's response **before** any guardrail, plugin or PII sanitizer runs. A plugin that rewrites or strips the `usage` object changes what the caller sees, not what you are billed. A response-cache hit that a rule blocks is logged with `cache_hit = 1`, `withheld = 1` and zero cost, since nothing was generated upstream for it. The original miss was billed when it happened. To see what blocked output is costing you: ```sql select sum(cost_usd) as withheld_spend, count() as withheld_requests from request_logs where withheld = 1 and ts > now() - interval 30 day; ``` `rolter_withheld_responses_total` counts the same refusals in Prometheus. A sustained rate usually means a prompt is steering the model into output your policy forbids, and you pay for every attempt. `usage_unknown = 1` marks a different gap: the upstream answered but reported no usage at all, so the row's zero tokens and cost are unknown rather than free. Most often it is an OpenAI-style stream sent without `stream_options: {"include_usage": true}`. A streamed Responses API answer (`/v1/responses`) needs nothing extra: its token counts arrive on the final `response.completed` (or `response.incomplete`) event and are logged from there. It is also set on a `502` whose upstream answered and then failed to deliver the body the gateway was reading (a reset, a truncated response): the provider may have billed an answer nobody received. Rows written before these columns existed default to `0`. ## MCP tool calls When ClickHouse is configured, the gateway records each `tools/call` it proxies on `/mcp/{server}` as one row in **Observability → MCP Logs**. Nothing else is recorded: `initialize`, `tools/list` and notifications leave no row, and with no `clickhouse_url` the gateway records nothing. Each row carries the server, the tool, the latency and a status: `success`, `error` (the tool reported `isError: true` or the server answered with a JSON-RPC error), `auth_denied` (the gateway refused the caller's OAuth session, or the server answered `401`/`403`), `timeout`, or `transport_error`. It is attributed to the calling virtual key's organization, team and project, and to the user who owns the key, which is the user whose OAuth session the call used. The call's arguments and result are stored only when `[logging.payload_capture]` is `enabled`, redacted with `redact_fields` and cut to `max_bytes`, the same policy as LLM payloads. Recording never slows or fails the call: if the write queue is full the row is dropped and `rolter_mcp_events_dropped_total` goes up. For a streamed (SSE) reply the row is written when the stream ends, so its latency includes the whole stream. ## Time windows Every usage, spend, provider-health and MCP-log read takes an optional `since` (inclusive) and `until` (exclusive). Left out, or sent empty, they cover the last 7 days up to now. Each accepts: - RFC 3339, which is what `Date.prototype.toISOString()` and most SDKs write: `2026-07-01T00:00:00Z`, `2026-07-01T00:00:00.123+02:00` - a date and time with no offset, read in the ClickHouse server's time zone: `2026-07-01 00:00:00`, `2026-07-01T00:00:00.000` - a bare date: `2026-07-01` Anything else is refused with `400` and an error naming the parameter: ```json { "error": { "message": "since must be an RFC 3339 timestamp such as 2026-07-01T00:00:00Z", "type": "invalid_request_error", "param": "since", "code": "invalid_time_bound" } } ``` That covers a date the calendar does not have (`2026-02-30`), a lowercase `t` or `z`, and a Unix timestamp; send those as RFC 3339 instead. The check exists because ClickHouse reads a bound it cannot parse as 1970-01-01, so without it a typo would scan the whole log rather than fail. Percent-encode the `+` of a positive offset as `%2B`. A query string decodes a bare `+` to a space, so `since=2026-07-01T00:00:00+02:00` arrives as `2026-07-01T00:00:00 02:00` and is refused, with a message that says so. ## Filtering LLM Logs **Filters** opens a rail beside the **LLM Logs** table that narrows the requests of one time window, the last 24 hours unless you pick another. The window is counted back from each refresh, so a screen left open keeps showing a single day. The control plane applies each filter before it cuts the page, so a page of 50 is 50 matching requests. - **Time window** is one of the [reporting windows](#time-windows). The address carries it as `window=7d`. - **Status** is one of **All**, **Errors** and **OK**. **Errors** is every request answered with a status of 400 or above, plus every request that got no answer at all (status `0`, shown as `ERR` in the table and **No response** in the drawer), since an upstream that never answered is the failure you are looking for. **OK** is every request answered with a status below 400, which includes a redirect as well as a `2xx`, and never a request that got no answer. **All** applies no filter. - **Model** takes one model, because the API filters on one exact model name. Picking a second model replaces the first. - **Virtual key** takes one key, picked by name from the keys of the project you are looking at. The address carries it as `key=`. - **Cost** has three checks. **Unpriced only** keeps the requests recorded as unpriced (see [Unpriced traffic](#unpriced-traffic)); the API takes it as `unpriced=true`. **Withheld responses only** keeps the requests a policy refused to deliver after the provider billed them, and **Unknown usage only** keeps the successes whose upstream reported no usage (see [Withheld and unknown-usage traffic](#withheld-and-unknown-usage-traffic)); the API takes them as `withheld=true` and `usage_unknown=true`. They combine with each other and with every other filter, but a withheld response is a `403` and a response with unknown usage was a success, so ticking both returns nothing. Like every filter here they narrow the rows your roles can read and never widen them. - **Business unit** and **Customer** each take several values. A request matches when it was charged to any of the ones you picked. - **Outcome** keeps the requests the gateway recorded with one [outcome](#the-outcome-of-each-request), such as **Refused: budget spent** for the requests a spent budget turned away. **Not recorded** keeps the rows written before the gateway recorded one. The API takes it as `outcome=refused_budget`; a value it does not know is a `400`. - **Plugin** appears when the address names one (`plugin=blocked:`), and keeps the requests that plugin acted on. **Clear filters** drops it. **Clear filters** at the top of the rail drops every filter at once. When nothing matches, the empty table offers the same button. The filters are part of the page's address, so a reload keeps them and a copied link opens the same view. Whoever opens it still sees only the rows their own role allows. The parameters use the API's names: ```text /logs?status=error&model=gpt-4o&business_unit=&unpriced=true ``` A link can outlive what it names. A status value the API does not know is ignored, and so is an `unpriced`, `withheld` or `usage_unknown` value other than `true`, and a model that has since been removed from the catalogue still filters the log and still shows in the picker, so you can see it and clear it. The row open in the detail drawer is not part of the address, except when it was opened by [an id lookup](#looking-a-request-up-in-the-dashboard), whose link opens the drawer again. ### Saved views **Saved views** beside **Filters** keeps the filters you have applied under a name, so you can come back to them in one click. On **LLM Logs** a view holds the time window, status, model, key, business unit and customer; on the Dashboard it holds the window, model, key, business unit and customer. It never holds an id lookup, the page you were on or the **Cost** checks. Pick **Apply** to set the screen's filters from the view (a **Cost** check you had ticked is cleared, since the view does not hold it), **Rename** to change its name, or the bin to delete it. Each account keeps up to 50 views per screen, and a name can be used once. Views are private: they follow your account to any browser you sign in from, and nobody else sees them. They never widen what you can read. If a view names a key, business unit or customer that you can no longer read, it is applied without that filter and the screen says how many it left out, for example _Applied without: 1 key, 2 customers_. The ids are not shown. Saving again keeps only the filters applied at that moment. The [Dashboard](/concepts/dashboard) has the same control and, applying a view, shows the same _Applied without: …_ notice for anything you can no longer read. Saved views need a signed-in account, so they are unavailable while the control plane runs in open mode. ## Paging the invocation log `GET /api/v1/analytics/invocations` returns one page of requests plus the cursor that resumes it: ```json { "data": [ /* newest first */ ], "next_cursor": "2026-07-19 12:00:01.000|req-1" } ``` Ask for the next page by handing that value straight back as `cursor`: ```bash curl -H "Authorization: Bearer $ROLTER_ADMIN_TOKEN" \ "https://control.example.com/api/v1/analytics/invocations?limit=50&cursor=2026-07-19%2012%3A00%3A01.000%7Creq-1" ``` The cursor is a position in the list's sort key — `(ts, request_id)`, newest first — not a row number, and this is what makes paging a live log correct. `request_logs` is written continuously by the gateway, so requests land above the window while you read: a row offset would count them in and either serve you a row you already saw or skip one entirely. A keyset bound cannot be moved by rows above it, so every request that existed when you started paging is returned exactly once. `request_id` breaks ties, so two requests logged in the same millisecond still have one order. Treat the cursor as opaque and pass it back unchanged. `next_cursor` is `null` only for an empty page; a short page still carries one, so stop when a page comes back with no rows. `limit` is clamped to `1..=200` and defaults to 50. A cursor that is not `timestamp|request_id`, or whose timestamp is not one of the [time window](#time-windows) forms, is a `400` with `param: "cursor"` and `code: "invalid_cursor"`. The MCP call log (`GET /api/v1/mcp/logs`) pages the same way over `(ts, event_id)` and checks its cursor the same way. Every filter — `since`, `until`, `model`, `key`, `business_unit`, `customer`, `status`, `unpriced`, `guardrail`, `plugin`, `outcome`, `request_id`, `trace_id` — must be repeated with the cursor, since it is the same query resumed rather than a stored server-side page. The **LLM Logs** screen pages the same way. **Next page** follows the cursor the current page returned and **Previous page** goes back to the one it came from, so a page you return to holds the requests it held before, however much traffic arrived in between. Only the first page follows live traffic while **Streaming** is on; later pages stay fixed on their cursor. Changing a filter starts again from the newest request, and a page after a full last page reads as the end of the log with a **Back to newest** button. ### Optional columns in the invocation list These fields of a row come from ClickHouse files that an existing deployment applies by hand (see [Upgrading](/deployment/upgrading)). Every row from `GET /api/v1/analytics/invocations` carries all of them, whether or not the file has been applied: | Field | Type | Added by | Read as when the table lacks it | | ----------------------- | ------ | ----------------------------- | ------------------------------- | | `upstream_status` | number | `015_upstream_attempts.sql` | `0` | | `attempts` | number | `015_upstream_attempts.sql` | `0` | | `lifecycle_operation` | string | `016_lifecycle_operation.sql` | `""` (empty string) | | `cache_write_1h_tokens` | number | `017_cache_write_1h.sql` | `0` | | `guardrail_events` | array | `018_guardrail_events.sql` | `[]` | | `plugin_events` | array | `019_plugin_events.sql` | `[]` | | `fallback_reason` | string | `023_fallback_reason.sql` | `""` (empty string) | | `param_adjustments` | array | `022_param_adjustments.sql` | `[]` | | `outcome` | string | `030_request_outcome.sql` | `"unknown"` | | `has_tools` | number | `031_request_shape.sql` | `0` | | `has_image_input` | number | `031_request_shape.sql` | `0` | | `has_audio_input` | number | `031_request_shape.sql` | `0` | | `has_response_schema` | number | `031_request_shape.sql` | `0` | | `turns` | number | `031_request_shape.sql` | `0` | `guardrail_events` lists what each guardrail did to the request, as `:` entries (`blocked:`, `redacted:` or `annotated:`) and `provider_blocked:` / `provider_transformed:` for the guardrail provider (the bare `provider_blocked` / `provider_transformed` on rows that predate the name, or for a provider with none); it is empty for a request no guardrail touched. See [What each rule did](/security/guardrails#what-each-rule-did). `fallback_reason` is the class of the failure that sent the request on to the target that answered it: `throttled`, `timeout`, `transport`, `server_error`, `refusal` or `refusal:`. It is empty when the first dial answered. **LLM Logs** shows a non-empty one as **Fallback reason** in the **Routing** group of the detail drawer. See [Fallback targets](/reliability/fallback-targets#seeing-why-a-request-reached-a-fallback). `plugin_events` lists what each plugin did to the request, as `:` entries (`allowed`, `blocked`, `transformed` or `annotated`) and `error::` for a call that got no verdict; it is empty for a request no plugin was consulted on. See [What each plugin did](/configuration/plugins#what-each-plugin-did). The five request-shape fields say what a request asked the model to do beyond its tokens, read by the gateway off the body the caller sent, in the OpenAI (Chat Completions, Responses) or Anthropic (Messages) dialect: `has_tools` is `1` when it offered the model tools, `has_image_input` and `has_audio_input` when an input message carried an image or audio, `has_response_schema` when it asked for output that follows a JSON schema (a bare JSON mode is not one), and `turns` is how many user turns it carried. They are `0` for a request that is not a chat-shaped model call, such as an embedding, and the [savings advisor](/observability/savings-advisor) excludes a request that has any of them set, or more than one turn. `cache_write_1h_tokens` is the part of the row's `cache_write_tokens` that went to the provider's 1 hour cache, never more than the total. The 5 minute share is the difference: `cache_write_tokens - cache_write_1h_tokens`. It is `0` for a provider that reports no split (every provider but Anthropic), for a request that wrote only to the 5 minute cache, and for every row older than the file. The control plane checks which of these columns the table has by reading `system.columns`, selects the ones that exist and fills in the default for the rest, so a request list never fails because a file is still to be applied. It remembers the answer: a table that has all of them is looked at again every ten minutes, one that lacks a column every minute, so a file you apply shows on the next page within a minute and needs no restart. A ClickHouse user that may not read `system.columns` is treated as having every column, which is how the list behaved before it checked. The check is made once the request is known to be well formed, and a ClickHouse that cannot be reached is an error as it always was, never a default. ## Finding one request by its id Every gateway response carries `x-request-id` (or the header the deployment set as its [request ID header](/configuration/client-settings#request-id-header)), and the same id is the row's `request_id`. When a client reports a failed call, that id takes you straight to it: ```bash curl -si https://gateway.example.com/v1/chat/completions ... | grep -i x-request-id # x-request-id: 3f2c9a1e-... curl -H "Authorization: Bearer $ROLTER_TOKEN" \ "https://control.example.com/api/v1/analytics/invocations?request_id=3f2c9a1e-..." ``` The answer is that one row, with its status, error, tokens, cost, latency and — if your role may read them — the captured bodies. `trace_id` looks a request up by its W3C trace id the same way, for when you start from a span in your tracing backend instead. Without a `since`, a lookup by either id searches every row still retained rather than the default last 7 days, so an older request is found rather than answered with an empty page. The lookup obeys the same visibility as the rest of the log: an id from a project you cannot read returns nothing. ### Looking a request up in the dashboard **LLM Logs** has a field under its toolbar for the same lookup. Paste the id and press **Find**. A 32-character lowercase hex string is a trace id, and so is a whole `traceparent` header value (Rolter reads the trace id out of it); anything else is taken as a request id. A request id is matched exactly. - **One row** comes back: its detail drawer opens on it. This is the usual result for a request id. - **Several rows** share the trace id: they are the list, and the toolbar says it is showing a request ID or trace ID lookup. Click a row to open it. - **Nothing** comes back: the table says **No request found**. The dashboard cannot tell an id that does not exist from one in a project your role cannot read, and neither can the API, so the message names both. A lookup searches every retained row, not the last 24 hours, and leaves out whatever the filter rail held: starting one clears those filters, so they cannot hide the request. It does not stream. **Clear lookup** (or the cross in the field) goes back to the live feed. The lookup is part of the page's address, so a link opens on the request: ```text /logs?request_id=3f2c9a1e-... /logs?trace_id=0af7651916cd43dd8448eb211c80319c ``` The command palette does the same from anywhere in the dashboard. Open it with `⌘K` (`Ctrl-K` on Windows and Linux), paste the id, and pick **Open request … in LLM Logs** (or **Open trace …**). The offer appears only when no screen or record matches what you typed, and only for an account that can open LLM Logs. A word has to look like an id to be offered: at least 8 characters, no spaces, with a digit in it. A request id that is exactly 32 lowercase hex characters, such as a UUID with its dashes removed, reads as a trace id. Use `/logs?request_id=…` to look it up as a request id. ### Reading a request in LLM Logs Clicking a row in **LLM Logs** opens it in a detail drawer, and the row stays highlighted in the table while the drawer is open. The drawer starts with the verdict: the status, when the request started, and its request id and trace id, each with a copy button. A status of **No response** means the gateway never got an answer from the upstream, because the connection was refused or timed out. A trace id reads **Not sent by the caller** when the request carried no trace header, since there is nothing to follow into a tracing backend. When the request failed, its error comes next, before anything else. The rest of the row is grouped: - **Routing**: provider → target, the A/B variant, whether Rolter answered from its response cache, whether the response was streamed, time to first token and latency. - **Usage and cost**: input and output tokens, the upstream's prompt-cache reads and writes, the 5 minute / 1 hour split of the write (**Cache write by lifetime**, only when part of it went to the 1 hour cache), and the cost. - **Attribution**: the virtual key by its name and prefix, the business unit and the customer. A key that cannot be looked up, for example one that has since been deleted, shows its id instead. The captured request and response bodies follow, or the reason they are absent (see [log settings](/observability/log-settings)). A request with neither body gets one note under a single **Request and response** heading, with one link to the log settings. A request with one body shows it, and the missing one gets its own short note. ### Reading the table The **Time** column shows the clock with milliseconds, such as `15:34:56.789`, because the milliseconds are what tell requests apart. The date is in the tooltip on the time and at the top of the detail drawer. **Status** and **Cost** stay on screen at every width, with **Time** and **Model**. **Latency**, **Tokens** and **Provider** drop out as the table gets narrower, widest first, and all three are in the detail drawer. The table's own width decides, not the window's, so opening the filter rail or the drawer hides columns instead of scrolling **Cost** out of view. Below 1280 px the drawer opens as a sheet over the table instead of beside it. ## Who reads the MCP call log `GET /api/v1/mcp/logs`, `/api/v1/mcp/logs/summary` and `/api/v1/mcp/logs/{event_id}` follow the same rule as the request log. A signed-in user reads the tool calls of the orgs, teams and projects where they hold a role, and every call made under their own MCP OAuth sessions, whatever project the gateway attributed it to. A superadmin, the admin token and open mode read everything. Someone with no role anywhere gets an empty list and a summary of zero, and a call outside their reach answers `404` exactly like one that does not exist. The list carries no tool arguments or results. The single-event read carries them only for a caller who is a member or above at the call's project, or a viewer on a project whose admin set `payload_min_role` to `viewer` (the same setting that shows LLM request bodies). Anyone else gets both fields empty and `payload_withheld: 1`. Recording an event (`POST /api/v1/mcp/events`) stays with the gateway and superadmins. Calls recorded before an MCP proxy attributed them to an org, project and user carry none of the three, so they stay visible to superadmins only. Nothing needs migrating: new calls are attributed when they are written. ## Built-in dashboard The Rolter control plane's dashboard queries ClickHouse directly to give you usage, spend, latency percentiles, and error rates — all sliced by model, project, and virtual key. No extra ETL or data pipeline is needed: the rows Rolter writes are the source of truth for the dashboard. Use the `variant` column to run A/B cost analyses. If you have two variants serving the same logical model (e.g. `gpt-4o` vs `claude-sonnet`), you can compare average `cost_usd` and `latency_ms` per variant directly in ClickHouse or your BI tool.