Log Settings: Sampling, Payload Capture, Retention and UX Events

Request logging is governed by one deployment-wide policy, edited from Observability → Logs Settings in the dashboard or through the management API. It decides four things: how much traffic is logged, whether raw payloads are captured alongside the metadata, how long each is kept, and whether the dashboard’s own usage events are recorded.

Without ClickHouse#

Sampling, payload capture and both retention windows act on rows written to ClickHouse. A control plane started without clickhouse_url (the CLICKHOUSE_URL environment variable) writes no log, so none of them has anything to act on. Logs Settings says so at the top of the screen, names that setting and links this page. The controls stay editable: the policy is stored and applies from the first row once ClickHouse is added and the control plane restarted.

The dashboard reads this from the analytics endpoints, which answer 503 when no ClickHouse is configured.

Sampling#

sample_rate is the fraction of requests written to the log, from 0 to 1. The dashboard shows it as a percentage. Lowering it cuts what lands in the log store on a high-volume deployment, and it cuts the raw row count read back out of that store by the same share.

The gateway decides per request, from a hash of the request id, and drops an unsampled row before it is written. Each kept row records its weight, sample_weight, which is 1 / sample_rate at the moment it was written (4 at 0.25; rows written unsampled, or before the column existed, weigh 1). The analytics endpoints (/api/v1/analytics/summary, timeseries, by-model, by-variant, by-attribution), the usage panel (/api/v1/me/usage) and the request_volume, spend_velocity and error_rate alert signals multiply every count and sum by it, so below 1:

• request counts, spend and token totals on the Dashboard and the cost attribution screens (Business Units and Customers) are scaled estimates of the real traffic, not the raw logged share. At 0.25 the Dashboard still shows about the real spend, with sampling noise that shrinks as volume grows. Averages and latency percentiles are not scaled, since a uniform sample already estimates them. Changing the rate does not distort history, because each row keeps the weight it was written with;
• the usage panel on Account gets the same scaling per key: requests, tokens, cost and errors are estimates of the real totals;
• LLM Logs lists only the logged requests, and a request that was not logged has no captured payload either;
• the request_volume and spend_velocity alert rules read the scaled figures, error_rate is the ratio of weighted counts, and p95_latency_ms is an estimate from the sample.

Three things count every request at any rate, because the gateway records them before it decides whether to log the row:

• budgets, which add each request’s cost to a Redis counter;
• rate limits, request and token windows alike;
• the Prometheus metrics on /metrics.

When spend read off the Dashboard has to be the real spend, keep the rate at 1 and bound storage with retention instead.

Below 100 %, Logs Settings states the consequence under the field: the share the log keeps, such as “about 1 in 4 requests”, and that counts, spend and tokens on the screens that read the log are estimates scaled up from the sample. The warning follows the form, so it shows for an unsaved edit as well as for a rate already saved. A cleared field is refused rather than saved as 0, which would log nothing. Text that is not a plain decimal, such as 0x10, is refused too.

The Dashboard, Business Units, Customers and LLM Logs carry the same note whenever the saved rate is below 100 %: the figures are estimates scaled up from a sample, and the LLM Logs list shows only the sampled requests. Every role that can open those screens sees the note: they read the rate from GET /api/v1/analytics/sampling, which answers the rate alone, while the rest of the logging settings stays a superadmin’s.

A control plane started with --config serves the file’s [logging] sample_rate until someone saves Logs Settings, since the stored settings then still hold their seeded defaults. The note follows the rate gateways actually apply, so it shows for a file rate below 100 % too. Logs Settings itself shows the stored settings, 100 % in that state, and saving the form replaces the file’s values.

Payload capture#

Payload capture stores the raw prompt and completion bodies next to the log metadata. It is off by default: payloads contain whatever the caller sent, which is the most sensitive data Rolter touches.

When it is on, four settings narrow what gets stored:

In the dashboard the two allow-lists are pickers rather than text. Models are chosen from those that carried traffic in the last 30 days, and a name that has not (a model not yet used, or a typo) is accepted but marked. Virtual keys are chosen by name and prefix from the project selected in the sidebar, and the stored value is still the key’s id. A saved key that is not among that project’s keys stays in the list, marked, until you remove it; a key of another project looks the same, so switch project to check. The API payload is the same list of names and ids as before.

Redacting secrets in text#

A key name cannot match a secret that sits inside free text: a key pasted into a prompt, a connection string in a tool argument, a bearer token in a pasted log. So capture also scans the text of every string for credential shapes and replaces each match with a typed marker, so a reader sees that something was there and what kind:

All are on by default. Add patterns of your own as name=regular expression lines under Custom patterns; the name becomes the marker. The engine runs in time linear in the input, so a pattern cannot be made to stall capture, and it has no look-around or backreferences. A policy holds at most 16 patterns of at most 256 bytes, and a pattern that does not compile is refused with a 422 naming the field. Detectors are heuristics: they catch the common shapes, not every secret, so keep payload_capture_redact_fields for keys you know by name.

Both kinds of redaction run on the gateway before the body is truncated and handed to the log writer, so no unredacted copy reaches ClickHouse. Streamed (SSE) responses are scanned as text rather than frame by frame.

Previewing redaction#

Preview redaction on the Logs Settings screen, or POST /api/v1/logging-settings/redaction-preview, runs the policy against a pasted sample and returns what capture would store plus what it found, by kind. The dashboard sends the policy as edited, saved or not; the API uses the stored policy for any part you leave out. The preview stores nothing, writes no audit entry and does not log the sample, so paste a test body rather than a live secret all the same: the sample does travel to the control plane for the check. It needs the same superadmin role as saving the settings.

Redaction happening before the write is the important part: turning a field on later does not retroactively clean what is already stored, and turning it off does not expose history that was never captured.

With the PII sanitizer on, both captured bodies are the sanitized ones: the request as it was forwarded to the provider, with placeholders in place of the personal data, and the response before any restoration: as the provider returned it, or, when the sanitizer’s direction covers the response, as the sanitizer returned it with placeholders in place of what it found. A caller under a restoration policy receives the plaintext back, but the capture store never holds it.

The dashboard states all of this at the point of decision: the switch that turns capture on carries a summary of exactly what will be stored — the truncation limit, the retention window, the fields redacted before the write, and which models are captured — computed from the values currently in the form, so an unsaved edit is reflected in it.

Who can read captured payloads#

A captured body is shown to the members and admins of the project the request belongs to — not to everyone who can see the request-log row. A viewer sees the row, its tokens, cost and status, with both bodies withheld. A project admin can let that project’s viewers read the bodies too, from the project’s settings (Project settings in the project’s ⋯ menu in the scope switcher) or with PUT /api/v1/projects/{id}/settings. The deployment-wide switch above still decides whether anything is captured at all. See who sees which request logs.

Why a payload panel is empty#

The log detail drawer explains a missing body instead of leaving an empty panel. A request with neither body stored gets one note, under a Request and response heading, and a body can be missing for three different reasons:

• payload capture was off when the request ran;
• capture was on, but the request’s model or virtual key falls outside the allow-lists above;
• the payload’s retention window has already passed. Payloads are kept for payload_retention_hours, which is deliberately shorter than the log row’s own retention, so an old row keeps its metadata long after its body is gone.

A fourth reason is certain rather than guessed: the body was captured, but your role on the request’s project is below the bar above. The server withholds it and says so, and the drawer tells you it is your role rather than listing the other three.

A request can also have one body and not the other. Capture stores the two together, so the body that is there rules out the three reasons above, and the missing one was empty when it was logged: a response the upstream never sent, for example. The drawer shows the stored body and puts a short note under the heading of the missing one, with no link to this page.

Bodies captured by a gateway from v0.1.0 or earlier also stop showing once the control plane is upgraded; see Upgrading.

The log row does not record which of the first three applies, so the note names all three. A superadmin or a security auditor — the only roles that can read this policy (deployment roles) — sees the answer narrowed to the actual reason, because the screen reads the live setting for them, and gets a link straight to this page’s settings screen. Everyone else is told that a superadmin sets payload capture, since the link would only lead them to a refusal.

Capture in the dogfooding profile#

integration/dogfood/gateway.toml is the one profile shipped with capture on. It runs against fake and self-owned providers with no real user data, and payload visibility is the point of dogfooding the Logs screen, so the drawer shows real bodies out of the box. This is a local-stack exception, not a change to the shipped default — every other profile, and every fresh deployment, still starts with capture off.

Retention#

The two halves of the log expire on separate clocks:

• retention_days — how long log metadata is kept (1–3650 days).
• payload_retention_hours — how long captured payloads are kept (1–8760 hours).

Payloads cannot outlive the metadata they belong to. payload_retention_hours must not exceed retention_days × 24, and both the dashboard and the API reject a policy that breaks it — otherwise raw prompt content would survive past the point an operator meant it to expire.

To delete one subject’s rows on request rather than wait for expiry, see Log erasure.

How expiry is enforced#

There is no purge job. Rolter translates the policy into a ClickHouse table TTL (alter table request_logs modify ttl …, and the same for request_payloads), and ClickHouse drops expired parts itself during merges. Deletion is therefore eventual, not instant: a row past its TTL may survive until its part is next merged.

The TTL is written in two places, so a control plane that could not reach ClickHouse does not end up enforcing a policy different from the one it reports:

• on update — immediately after the policy is saved. If ClickHouse is unreachable the saved policy still stands and the API still returns 200; losing the operator’s setting because the log store was briefly down would be the worse failure.
• at control-plane startup — the stored policy is re-applied. This is what converges the two after an update that could not reach ClickHouse, and after a ClickHouse restored from backup or provisioned later, either of which would otherwise sit on the schema defaults (90 days / 7 days) indefinitely.

Re-applying an unchanged policy is a no-op, so the startup pass is safe on every boot. It runs in the background and never delays serving traffic.

Dashboard usage events#

The Dashboard usage events switch (ui_events) decides whether the control plane accepts the dashboard’s own UX events — screen views, load times, abandoned forms, error states. It is on by default, unlike payload capture, because those events carry structure only: a screen, an action, a duration and ids, never a form value or prompt.

Switching it off makes the ingest endpoint answer 202 and store nothing, from the next batch onward. Events already recorded are not deleted; they expire with the table’s own TTL. The stream is inert anyway when ClickHouse is not configured.

API#

Reading is for a superadmin or a security auditor (a deployment role); updating is superadmin only. Every update is audited.

curl -H "Authorization: Bearer $ROLTER_ADMIN_TOKEN" \  https://control.example.com/api/v1/logging-settingscurl -X PUT -H "Authorization: Bearer $ROLTER_ADMIN_TOKEN" \  -H "Content-Type: application/json" \  -d '{        "sample_rate": 0.1,        "payload_capture_enabled": true,        "payload_capture_max_bytes": 65536,        "payload_capture_redact_fields": ["authorization", "api_key"],        "payload_capture_redact_detectors": ["aws_key", "jwt", "openai_key"],        "payload_capture_redact_patterns": [{"name": "employee_id", "pattern": "EMP-\\d{6}"}],        "payload_capture_models": [],        "payload_capture_virtual_key_ids": [],        "retention_days": 90,        "payload_retention_hours": 168,        "ui_events": true      }' \  https://control.example.com/api/v1/logging-settings

Send the full policy, not a patch — with one exception: ui_events may be omitted, and then keeps its stored value, so a client written before the field existed cannot switch the stream back on. A successful update bumps config_version, so gateways pick it up on their next snapshot poll without a restart.