Guardrails

Rolter can inspect request or response content with built-in detectors, custom regular expressions, and a self-hosted HTTP guardrail service. Open Guardrails → Guardrail Rules or Guardrails → Providers in the dashboard. These screens require superadmin access.

Inspection rules#

Rules run by ascending position, after any rules defined in the [[guardrails.rules]] tables of the config file. The dashboard lists them in that order, and a new rule starts 10 after the last one. Each rule uses exactly one source:

• Built-in detector — email address, phone number, API token, or payment card.
• Custom regex — compiled by Rolter’s bounded, linear-time regex engine before the rule is published.

Choose annotate to count a match without changing traffic, block to reject matching traffic, or redact to replace matches. System messages are excluded unless Inspect system messages is enabled. A paused rule stays stored and ordered but does not enter gateway snapshots.

Try a rule before you publish it#

The rule dialog has a Try it on a sample panel. Paste some text, optionally mark it as a system message, and select Run test. The control plane runs the draft exactly as filled in, with the same matcher and the same pattern limits the gateway uses, and answers with what would happen: no match, annotated, redacted (with the redacted text), would block, or not scanned (a system message the rule does not inspect, or a sample longer than the gateway scans per request). Each match is marked in the sample. A pattern the gateway would refuse comes back with the gateway’s message, so you learn that before publishing.

The test does not save or publish anything, and the sample is not logged or stored. It runs one rule on its own. To see how it combines with the rest, select Run in the whole policy in the same panel. A sample is limited to 64 KiB. Testing needs the same access as editing a rule.

Try the whole policy#

Rules do not act alone. Config-file rules run first, then the enabled dashboard rules by position. A block ends the scan, and a redact rule hides its matches from the rules after it, so an earlier redaction can stop a later block rule from firing. Try the policy at the top of the Guardrail Rules screen runs a sample through all of it with the gateway’s own scan and lists every rule in the order it runs: where it comes from (config file, dashboard or draft), what it did, and why it did not act when it was skipped (other stage, system message, an earlier block, guardrails off). Below the list is the verdict and the text that would be forwarded after every redaction. Pick Request or Response to try the pre-call or the post-call rules.

Run in the whole policy in the rule dialog does the same with the rule you are editing slotted in at its position. If a saved rule has the same name, the draft takes its place. Route overrides are not applied; the answer is for the policy, not for one route. Like the single-rule test, it saves nothing, logs nothing and needs the access that editing a rule needs.

The detector and action pickers describe what each choice does, and the Source and Stage fields carry an (i) note beside the label that says what they choose between. payment_card matches any run of 13 to 19 digits and does not check the card number, so a 13-digit timestamp matches too; phone matches 8 to 20 digits.

A response rule that blocks runs after the provider has generated and billed the answer. The caller gets a 403 guardrail_blocked, but the request is still billed and counts toward budgets. Its log row is marked withheld and names the rule, and the blocked text is never stored. See Withheld responses.

A response rule (stage Before response) needs the whole answer, so it cannot act on a streamed one. The deployment-wide streaming_post_call setting in the [guardrails] block of rolter.toml (reference) decides what happens instead:

• reject (the default) — while the rule is enforced, a request with "stream": true on a route the rule covers is refused with 400 guardrail_streaming_unsupported. Chat clients and coding agents usually stream, so one response rule can refuse most interactive traffic.
• passthrough — the stream is served and the rule does not see it. Only non-streamed responses are checked.

The rule dialog shows which of the two applies as soon as Before response is picked, and each response rule’s card repeats it in one line. Both read the value from the effective configuration (GET /api/v1/config). When that cannot be read, they say so and describe both outcomes rather than assuming the default. The dashboard shows the setting but does not change it.

The deployment-wide guardrails feature flag remains the kill switch. Turning it off pauses every built-in and custom rule, and every config-file rule, without deleting the registry. While it is off, Guardrails → Guardrail Rules shows a banner saying no rule inspects traffic, with a link to Feature Flags, and every rule’s card says not enforced instead of enforced.

Redaction tokens#

A redact rule replaces each match with its Replacement token. Leave the field empty and the gateway writes the default token for the rule’s source:

A new rule starts with the field empty, and the empty field shows the default for the selected detector. Changing the detector or the source updates a token you have not edited; a token you typed is kept. annotate and block rules never rewrite traffic, so the dialog saves them with no replacement and their cards say the content is not rewritten. A redact rule’s card names the token it writes, the default included.

Over the API, omit replacement or send null (or an empty string) to get the default token.

What the Rules screen reports#

A card’s status comes from the policy the control plane hands to gateways, read from the effective configuration (GET /api/v1/config), not only from the rule’s own switch:

• enforced — the rule is in the effective policy and the guardrails flag is on.
• not enforced — the rule is in the policy, but the flag is off.
• paused — the rule is switched off on its card.
• overridden by config file — the config file defines a rule with the same name. The file’s rule wins a name clash, so this card’s rule does not run. The card names the file rule, which is listed below under Config-file rules. A paused rule under a file rule’s name says it would not run if resumed, and the rule dialog warns when you type a name the config file already uses.
• status unknown — the effective configuration could not be read. A banner says so and offers a retry; the screen does not assume the rules are enforced.

Rules defined in the config file are listed under Config-file rules, read-only, in the order they run. They run before the rules added on the screen and change only when the file does.

The effective configuration does not say where each rule came from, so the screen infers it from names. A rule the configuration carries is a config-file rule when no card has its name, when the card with its name is paused, or when it differs from that card in any field. A config-file rule identical in every field to an enabled card cannot be told apart from it and is shown as that card. Enforcement is the same either way, but deleting or pausing the card would leave the file’s copy running. #2249 tracks reporting the source directly.

What each rule did#

Every request a guardrail acts on leaves its outcome on the request’s row in LLM Logs: the rule’s name and what it did, never the text it matched. A rule’s card on Guardrails → Rules counts them over the last seven days.

A request counts once per rule however many times the rule matched in it. Each non-zero count on a card links to the LLM Logs rows behind it (the log’s Guardrail filter, guardrail=blocked:<rule name> in the address). A rule that has not matched says No matches in the last 7 days. The active provider’s card on Guardrails → Providers counts the requests the provider blocked and the ones it replaced the content of (provider_blocked:<provider name>, provider_transformed:<provider name>); a provider that is not the one the gateway calls shows nothing. Each entry carries the name the provider had when it acted, so after you switch providers the old one’s history stays out of the new one’s card (renaming a provider starts its count over). Requests recorded before the name was added, and those of a config-file [guardrail_webhook] that sets no name, have the bare form; the enforced provider’s card lists them apart under Recorded without a provider name. A pre-call block keeps the redacted:<rule> entries of the rules that ran before it, so the redaction counts include requests that were refused afterwards.

The counts are read from ClickHouse. Without CLICKHOUSE_URL the screens say so once, and with ClickHouse but without clickhouse/018_guardrail_events.sql applied they say the counts are not being kept (see Upgrading); neither is shown as “no matches”. Output rules that redact or annotate a response are not counted yet, only the ones that withhold it, and neither is a guardrail provider’s transport failure.

The same rollup is GET /api/v1/analytics/guardrails (see Dashboard analytics).

Turning a rule off for one route#

A route can switch a rule off for its own traffic. Name the rule in the route’s advanced.guardrails.disable list. Every other route keeps running it, and a rule added later reaches every route that does not name it. A rule the route lists under advanced.guardrails.enable runs there even if disable names it too.

A name that no rule has is a typo, and a configuration file that contains one is refused at load. The control plane is more forgiving about dashboard rules: if you pause, rename or delete a rule that a route still names, the route’s override is left out of the configuration sent to the gateways and reported on GET /api/v1/config/problems, rather than blocking every other change. Update or remove the override to clear the notice.

[[routes]]model = "internal-support-bot"[routes.advanced.guardrails]disable = ["Redact customer email"]

Rules are matched by their exact name. For a config-file route, set the lists in rolter.toml as above. For a route created in the dashboard, open its model sheet and use the Guardrails section: one list of rules to switch off on the route and one of rules to force on, picked from the rules in force. A stored name that matches no rule stays in the list, marked, so you can remove it. The sheet shows a config-file route’s lists read-only. Without the dashboard, send them in the advanced object of PUT /api/v1/routes/{id}/advanced. A name that no rule has fails config validation, so a typo in disable cannot read as “this rule is off here” while the rule keeps running. Take a rule out of every route’s lists before you pause, rename or delete it: until #2306 is fixed, a route that still names it stops config changes from reaching gateways.

Guardrails → Guardrail Rules shows each exception on the rule’s card, for dashboard and config-file rules alike. The card keeps its status, since the rule still runs on every other route, and adds a line such as Off for traffic on 2 routes with the route names under it. Past three routes the names fold behind that line; select it to list them. The routes are read from the same effective configuration as the rest of the screen. A paused card lists no routes, since its rule is not in the policy. Neither does a card the config file overrides: the override names a rule, and the rule under that name is the file’s, so its card lists the routes. An override that names a rule the policy no longer has appears on no card.

External providers#

Register any number of vendor-neutral HTTP services and enable as many as you need: enabling one never pauses another. Which providers judge a request is decided by semantic guard policies, scoped to the deployment, an organization, a project, a route or a virtual key. A provider no policy lists runs on every request no policy covers.

Each field of the provider dialog that is not obvious from its label carries an (i) note: the stage, the failure policy, the timeout, the retries, the body cap and the authentication say what they do, and the defaults where there is one. Point at the (i) or focus it to read the note; Escape closes it without closing the dialog.

Choose the failure policy deliberately:

• Fail closed rejects traffic when the provider times out, is overloaded, errors or answers something that is not a decision. This favors enforcement.
• Fail open lets traffic continue when the provider cannot decide. This favors availability.

Each provider has its own; see failure handling for every case.

The Evaluation URL starts empty and is required. The address shown in the empty field is an example, not a default, so a provider cannot be saved pointing at a host nobody typed.

The evaluation URL is checked against the egress policy when the provider is saved, so an address such as the cloud metadata service is refused with a 400 naming the field, and again before every call. Rolter does not follow a redirect from the service. The same applies to [guardrail_webhook].url in a config file.

Enabling a provider asks for confirmation. The dialog says it joins the providers already running, with nothing paused, and states what traffic it will then see: every request no policy covers when no policy lists it, or the scopes of the policies that do. It also states what its failure policy does to that traffic. Cancelling returns to the form with nothing saved. Saving a provider that is already enabled, with the switch left on, does not ask again.

Pausing a provider asks too. The dialog names it, says the other enabled providers keep running, and names the policies that lose it.

Test a provider before you activate it#

The provider dialog has a Send a test envelope panel. It works on the form as filled in, so you can test before the first save. The control plane sends the same JSON envelope the gateway sends before a request (direction: "request", a user message holding your sample, or a short harmless sentence when you leave it empty) to the URL, with the provider’s timeout (at most 10 seconds) and body cap. It makes one attempt, with no retries, so a flaky service shows up as a failure rather than being retried away.

The answer is always a result, never a server error: the decision (allow, block with its reason, transform or annotate) with the round-trip time, or the kind of failure: refused by the egress policy, credential variable not set, timeout, connection failed, TLS failed, a non-2xx status, an answer that is not a decision, or an answer over 1 MiB. An answer that is not a decision is reported as a failure, the same reading the gateway applies: there it resolves through the provider’s failure policy. A body with scored categories and no action is a decision (an allow carrying scores). For a failure, the dialog also says what the provider’s failure policy does to live traffic.

The call obeys the same egress policy as saving does, checked before connecting and again on the addresses the name resolves to, and does not follow redirects. The response body is never shown on an error status, because an endpoint can echo the credential it just rejected.

Credentials are read from the control plane’s environment, while production gateways read them from their own. If the variable named in Secret environment variable is not set on the control plane, the test says so and does not call the service; this says nothing about the gateways. Either set it on the control plane for the test, or turn on Send without the credential. A variable that holds a control-plane secret (ROLTER_KEK, ROLTER_DATABASE_URL and any other ROLTER_ name except ROLTER_GUARDRAIL_TOKEN) is refused with a 400, so a test URL cannot be used to send one out. Testing needs the same access as editing a provider.

Rolter stores only an environment variable name for bearer or shared-secret authentication. Put the secret value in every gateway’s environment; it is never returned by the control-plane API.

Both stages run: Before upstream (pre_call) judges the request and Before response (post_call) the response. A post_call provider on a streamed response is refused or buffered, never skipped; see streamed responses.

The banner at the top of Guardrails → Providers reports the providers the control plane hands to gateways, read from GET /api/v1/guardrails/effective, not which registry rows are switched on. It names every running provider, from the registry or the config file, and which of them are unattached, and shows the status as unknown when the effective configuration cannot be read.

Hot reload and config files#

Creating, editing, enabling, or deleting a registry entry increments config_version, so gateway snapshots update without a restart. Rules defined in the config file remain immutable and win a name collision; dashboard-managed rules extend them. The Rules screen lists the config-file rules and marks a dashboard rule whose name the file takes as overridden. An enabled config-file webhook runs beside the registry’s providers as one more provider; it no longer displaces them. A config-file provider or policy wins a registry entry of the same name. The control plane states where the webhook and each rule come from (see GET /api/v1/guardrails/effective below), so a config-file rule or webhook that is an exact copy of an enabled dashboard entry is still reported as the file’s, and the dashboard entry as overridden.

API endpoints#

Reading rules and providers is for a superadmin or a security auditor (a deployment role); creating, changing and deleting them is superadmin only. GET /api/v1/guardrails/effective takes the same read access and returns what the gateways run: guardrails (the feature flag), streaming_post_call, rules, and webhook (null when none is on). Each rule carries source (file or registry), rule_id (the dashboard row behind a registry rule) and overrides (the enabled dashboard row a config-file rule displaced). The webhook is a single-provider view kept for older clients: the config-file webhook when it is on, else the first enabled registry provider, with source (file, registry or none) and provider_id; overrides is always null, since nothing displaces a provider any more. The semantic object lists every provider and policy the gateways run; see semantic guard policies. Deleting a provider a policy lists is a 409.

Every mutation is audit logged. The test endpoints write nothing and are not audit logged; the rule and policy ones need guardrail_rule:update, the provider one guardrail_provider:update.

POST /api/v1/guardrails/rules/test takes sample plus the rule fields (builtin or pattern, stage, action, replacement, include_system) and system (treat the sample as a system message). It returns outcome (no_match, annotated, redacted, blocked or skipped, with skip saying why), match_count, spans (UTF-16 offsets into the sample, at most 200) and redacted. An invalid rule or a sample over 64 KiB is a 400.

POST /api/v1/guardrails/policy/test takes sample, stage (pre_call by default, or post_call), system and an optional draft: the rule fields above plus name (default draft) and position. It returns outcome, blocked_by, redacted (the text after every redaction, only when it changed) and steps, one per rule in run order, each with rule, source (file, dashboard or draft), stage, action, outcome, match_count (against the text as that rule saw it) and, for skipped, skip (system_message, over_scan_budget, other_stage, after_block, policy_disabled or not_compiled). A draft named like a config-file rule, an invalid draft or a sample over 64 KiB is a 400.

POST /api/v1/guardrails/providers/test takes url plus stage, timeout_ms, max_body_bytes, auth_kind, auth_env, an optional sample and skip_credential. It returns ok, decision or error (kind and message), credential (not_configured, sent, skipped or unset), http_status, latency_ms and truncated. Only a malformed draft is a 400; a refused address or a failing service is a 200 with ok: false. Provider bodies accept auth_kind as none, bearer, or shared_secret; auth_env is required for the latter two and contains only the environment variable name.