> ## Documentation index > Fetch the complete documentation index at: https://rolter-ai-rolter.docs7.io/llms.txt > Ask it with GET https://rolter-ai-rolter.docs7.io/ask?q=. Short keywords rank best. > Use these to discover all available pages before exploring further. # Log Settings: Sampling, Payload Capture, Retention and UX Events > Control what fraction of requests Rolter logs, whether raw prompt and completion bodies are captured, which fields are redacted from them, how long each half is kept, and whether dashboard usage events are recorded. Request logging is governed by one deployment-wide policy, edited from **Observability → Logs Settings** in the dashboard or through the management API. It decides four things: how much traffic is logged, whether raw payloads are captured alongside the metadata, how long each is kept, and whether the dashboard's own usage events are recorded. --- ## Without ClickHouse Sampling, payload capture and both retention windows act on rows written to ClickHouse. A control plane started without `clickhouse_url` (the `CLICKHOUSE_URL` environment variable) writes no log, so none of them has anything to act on. **Logs Settings** says so at the top of the screen, names that setting and links this page. The controls stay editable: the policy is stored and applies from the first row once ClickHouse is added and the control plane restarted. The dashboard reads this from the analytics endpoints, which answer `503` when no ClickHouse is configured. --- ## Sampling `sample_rate` is the fraction of requests written to the log, from `0` to `1`. The dashboard shows it as a percentage. Lowering it cuts what lands in the log store on a high-volume deployment, and it cuts the raw row count read back out of that store by the same share. The gateway decides per request, from a hash of the request id, and drops an unsampled row before it is written. Each kept row records its weight, `sample_weight`, which is `1 / sample_rate` at the moment it was written (`4` at `0.25`; rows written unsampled, or before the column existed, weigh `1`). The analytics endpoints (`/api/v1/analytics/summary`, `timeseries`, `by-model`, `by-variant`, `by-attribution`), the usage panel (`/api/v1/me/usage`) and the `request_volume`, `spend_velocity` and `error_rate` alert signals multiply every count and sum by it, so below `1`: - request counts, spend and token totals on the **Dashboard** and the [cost attribution](/security/cost-attribution) screens (**Business Units** and **Customers**) are scaled estimates of the real traffic, not the raw logged share. At `0.25` the Dashboard still shows about the real spend, with sampling noise that shrinks as volume grows. Averages and latency percentiles are not scaled, since a uniform sample already estimates them. Changing the rate does not distort history, because each row keeps the weight it was written with; - the usage panel on **Account** gets the same scaling per key: requests, tokens, cost and errors are estimates of the real totals; - **LLM Logs** lists only the logged requests, and a request that was not logged has no captured payload either; - the `request_volume` and `spend_velocity` [alert rules](/observability/alerting) read the scaled figures, `error_rate` is the ratio of weighted counts, and `p95_latency_ms` is an estimate from the sample. Three things count every request at any rate, because the gateway records them before it decides whether to log the row: - [budgets](/security/budgets-and-rate-limits), which add each request's cost to a Redis counter; - [rate limits](/security/budgets-and-rate-limits), request and token windows alike; - the Prometheus [metrics](/observability/metrics) on `/metrics`. When spend read off the Dashboard has to be the real spend, keep the rate at `1` and bound storage with [retention](#retention) instead. Below 100 %, **Logs Settings** states the consequence under the field: the share the log keeps, such as "about 1 in 4 requests", and that counts, spend and tokens on the screens that read the log are estimates scaled up from the sample. The warning follows the form, so it shows for an unsaved edit as well as for a rate already saved. A cleared field is refused rather than saved as `0`, which would log nothing. Text that is not a plain decimal, such as `0x10`, is refused too. The **Dashboard**, **Business Units**, **Customers** and **LLM Logs** carry the same note whenever the saved rate is below 100 %: the figures are estimates scaled up from a sample, and the LLM Logs list shows only the sampled requests. Every role that can open those screens sees the note: they read the rate from [`GET /api/v1/analytics/sampling`](/api/dashboard-analytics#sampling), which answers the rate alone, while the rest of the logging settings stays a superadmin's. A control plane started with `--config` serves the file's `[logging] sample_rate` until someone saves **Logs Settings**, since the stored settings then still hold their seeded defaults. The note follows the rate gateways actually apply, so it shows for a file rate below 100 % too. **Logs Settings** itself shows the stored settings, `100 %` in that state, and saving the form replaces the file's values. --- ## Payload capture Payload capture stores the raw prompt and completion bodies next to the log metadata. It is **off by default**: payloads contain whatever the caller sent, which is the most sensitive data Rolter touches. When it is on, four settings narrow what gets stored: | Setting | Effect | | ---------------------------------- | ----------------------------------------------------------------------------------------------------- | | `payload_capture_max_bytes` | Truncates anything longer. Maximum `1048576` (1 MiB). | | `payload_capture_redact_fields` | JSON keys stripped **before the write**, so a redacted value never reaches storage. | | `payload_capture_redact_detectors` | Built-in value detectors, on by default; see [Redacting secrets in text](#redacting-secrets-in-text). | | `payload_capture_redact_patterns` | Your own value patterns, each `{name, pattern}`. | | `payload_capture_models` | Capture only these models. Empty means all. | | `payload_capture_virtual_key_ids` | Capture only these virtual keys. Empty means all. | In the dashboard the two allow-lists are pickers rather than text. Models are chosen from those that carried traffic in the last 30 days, and a name that has not (a model not yet used, or a typo) is accepted but marked. Virtual keys are chosen by name and prefix from the project selected in the sidebar, and the stored value is still the key's id. A saved key that is not among that project's keys stays in the list, marked, until you remove it; a key of another project looks the same, so switch project to check. The API payload is the same list of names and ids as before. ### Redacting secrets in text A key name cannot match a secret that sits inside free text: a key pasted into a prompt, a connection string in a tool argument, a bearer token in a pasted log. So capture also scans the text of every string for credential shapes and replaces each match with a typed marker, so a reader sees that something was there and what kind: | Detector | Matches | Marker | | ------------------- | --------------------------------------------------------------- | ---------------------------------- | | `private_key` | a PEM private-key block, through to the end if it was truncated | `[redacted:private_key]` | | `anthropic_key` | `sk-ant-…` | `[redacted:anthropic_key]` | | `openai_key` | `sk-…`, `sk-proj-…` | `[redacted:openai_key]` | | `google_api_key` | `AIza…` | `[redacted:google_api_key]` | | `github_token` | `ghp_…`, `gho_…`, `github_pat_…` | `[redacted:github_token]` | | `slack_token` | `xoxb-…`, `xoxp-…` | `[redacted:slack_token]` | | `stripe_key` | `sk_live_…`, `rk_test_…` | `[redacted:stripe_key]` | | `aws_key` | `AKIA…`, `ASIA…` access key ids | `[redacted:aws_key]` | | `jwt` | three dot-separated base64url segments starting `eyJ` | `[redacted:jwt]` | | `bearer_token` | `Bearer` followed by a long token | `[redacted:bearer_token]` | | `connection_string` | the password of `scheme://user:password@host`; the rest is kept | `…:[redacted:connection_string]@…` | All are on by default. Add patterns of your own as `name=regular expression` lines under **Custom patterns**; the name becomes the marker. The engine runs in time linear in the input, so a pattern cannot be made to stall capture, and it has no look-around or backreferences. A policy holds at most 16 patterns of at most 256 bytes, and a pattern that does not compile is refused with a `422` naming the field. Detectors are heuristics: they catch the common shapes, not every secret, so keep `payload_capture_redact_fields` for keys you know by name. Both kinds of redaction run on the gateway before the body is truncated and handed to the log writer, so no unredacted copy reaches ClickHouse. Streamed (SSE) responses are scanned as text rather than frame by frame. The MCP tool-call log applies the same policy, key names, detectors and your patterns alike, to the arguments and result it stores, so a credential pasted into a tool argument or returned by a tool is stored as its marker. ### Previewing redaction **Preview redaction** on the Logs Settings screen, or `POST /api/v1/logging-settings/redaction-preview`, runs the policy against a pasted sample and returns what capture would store plus what it found, by kind. The dashboard sends the policy as edited, saved or not; the API uses the stored policy for any part you leave out. The preview stores nothing, writes no audit entry and does not log the sample, so paste a test body rather than a live secret all the same: the sample does travel to the control plane for the check. It needs the same superadmin role as saving the settings. Redaction happening before the write is the important part: turning a field on later does not retroactively clean what is already stored, and turning it off does not expose history that was never captured. With the [PII sanitizer](/security/pii-sanitizer) on, both captured bodies are the sanitized ones: the request as it was forwarded to the provider, with placeholders in place of the personal data, and the response **before** any restoration: as the provider returned it, or, when the sanitizer's `direction` covers the response, as the sanitizer returned it with placeholders in place of what it found. A caller under a restoration policy receives the plaintext back, but the capture store never holds it. The dashboard states all of this at the point of decision: the switch that turns capture on carries a summary of exactly what will be stored — the truncation limit, the retention window, the fields redacted before the write, and which models are captured — computed from the values currently in the form, so an unsaved edit is reflected in it. ### Who can read captured payloads A captured body is shown to the **members and admins** of the project the request belongs to — not to everyone who can see the request-log row. A viewer sees the row, its tokens, cost and status, with both bodies withheld. A project admin can let that project's viewers read the bodies too, from the project's settings (**Project settings** in the project's **⋯** menu in the scope switcher) or with [`PUT /api/v1/projects/{id}/settings`](/api/project-settings). The deployment-wide switch above still decides whether anything is captured at all. See [who sees which request logs](/security/rbac#who-sees-which-request-logs). ### Why a payload panel is empty The log detail drawer explains a missing body instead of leaving an empty panel. A request with neither body stored gets one note, under a **Request and response** heading, and a body can be missing for three different reasons: - payload capture was off when the request ran; - capture was on, but the request's model or virtual key falls outside the allow-lists above; - the payload's retention window has already passed. Payloads are kept for `payload_retention_hours`, which is deliberately shorter than the log row's own retention, so an old row keeps its metadata long after its body is gone. A fourth reason is certain rather than guessed: the body was captured, but your role on the request's project is below the bar [above](#who-can-read-captured-payloads). The server withholds it and says so, and the drawer tells you it is your role rather than listing the other three. A request can also have one body and not the other. Capture stores the two together, so the body that is there rules out the three reasons above, and the missing one was empty when it was logged: a response the upstream never sent, for example. The drawer shows the stored body and puts a short note under the heading of the missing one, with no link to this page. Bodies captured by a gateway from v0.1.0 or earlier also stop showing once the control plane is upgraded; see [Upgrading](/deployment/upgrading#captured-payloads-from-older-gateways). The log row does not record which of the first three applies, so the note names all three. A superadmin or a security auditor — the only roles that can read this policy ([deployment roles](/security/rbac#deployment-roles)) — sees the answer narrowed to the actual reason, because the screen reads the live setting for them, and gets a link straight to this page's settings screen. Everyone else is told that a superadmin sets payload capture, since the link would only lead them to a refusal. ### Capture in the dogfooding profile `integration/dogfood/gateway.toml` is the one profile shipped with capture **on**. It runs against fake and self-owned providers with no real user data, and payload visibility is the point of dogfooding the Logs screen, so the drawer shows real bodies out of the box. This is a local-stack exception, not a change to the shipped default — every other profile, and every fresh deployment, still starts with capture off. --- ## Retention The two halves of the log expire on separate clocks: - `retention_days` — how long log **metadata** is kept (1–3650 days). - `payload_retention_hours` — how long **captured payloads** are kept (1–8760 hours). Payloads cannot outlive the metadata they belong to. `payload_retention_hours` must not exceed `retention_days × 24`, and both the dashboard and the API reject a policy that breaks it — otherwise raw prompt content would survive past the point an operator meant it to expire. To delete one subject's rows on request rather than wait for expiry, see [Log erasure](/observability/log-erasure). ### How expiry is enforced There is no purge job. Rolter translates the policy into a ClickHouse **table TTL** (`alter table request_logs modify ttl …`, and the same for `request_payloads`), and ClickHouse drops expired parts itself during merges. Deletion is therefore eventual, not instant: a row past its TTL may survive until its part is next merged. The TTL is written in two places, so a control plane that could not reach ClickHouse does not end up enforcing a policy different from the one it reports: - **on update** — immediately after the policy is saved. If ClickHouse is unreachable the saved policy still stands and the API still returns `200`; losing the operator's setting because the log store was briefly down would be the worse failure. - **at control-plane startup** — the stored policy is re-applied. This is what converges the two after an update that could not reach ClickHouse, and after a ClickHouse restored from backup or provisioned later, either of which would otherwise sit on the schema defaults (90 days / 7 days) indefinitely. Re-applying an unchanged policy is a no-op, so the startup pass is safe on every boot. It runs in the background and never delays serving traffic. --- ## Dashboard usage events The **Dashboard usage events** switch (`ui_events`) decides whether the control plane accepts the dashboard's own [UX events](/api/ui-events) — screen views, load times, abandoned forms, error states. It is **on by default**, unlike payload capture, because those events carry structure only: a screen, an action, a duration and ids, never a form value or prompt. Switching it off makes the ingest endpoint answer `202` and store nothing, from the next batch onward. Events already recorded are not deleted; they expire with the table's own TTL. The stream is inert anyway when ClickHouse is not configured. --- ## API Reading is for a **superadmin** or a **security auditor** (a [deployment role](/security/rbac#deployment-roles)); updating is **superadmin only**. Every update is audited. ```bash curl -H "Authorization: Bearer $ROLTER_ADMIN_TOKEN" \ https://control.example.com/api/v1/logging-settings curl -X PUT -H "Authorization: Bearer $ROLTER_ADMIN_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "sample_rate": 0.1, "payload_capture_enabled": true, "payload_capture_max_bytes": 65536, "payload_capture_redact_fields": ["authorization", "api_key"], "payload_capture_redact_detectors": ["aws_key", "jwt", "openai_key"], "payload_capture_redact_patterns": [{"name": "employee_id", "pattern": "EMP-\\d{6}"}], "payload_capture_models": [], "payload_capture_virtual_key_ids": [], "retention_days": 90, "payload_retention_hours": 168, "ui_events": true }' \ https://control.example.com/api/v1/logging-settings ``` Send the full policy, not a patch — with one exception: `ui_events` may be omitted, and then keeps its stored value, so a client written before the field existed cannot switch the stream back on. A successful update bumps `config_version`, so gateways pick it up on their next snapshot poll without a restart.