Rolter’s built-in guardrails redact with regular expressions inside the gateway. That is fast and self-contained, but a regex cannot tell a person’s name from any other capitalised word. When a deployment needs real named-entity recognition — and needs the original values back afterwards — point Rolter at an external sanitizer instead.
The sanitizer is a service you host: Microsoft Presidio is the reference, but anything that speaks the JSON contract below will do. It is disabled by default.
What Rolter does and does not hold#
Rolter sends the content, receives the content back with each detected entity replaced by a deterministic placeholder (<PERSON_1>, <EMAIL_ADDRESS_2>), and receives an opaque restoration token.
The mapping from placeholder to plaintext stays on the sanitizer. Rolter holds only the token. This is the point of the design: there is no gateway log line, metric label, span attribute, or cache entry that can leak the original values, because the gateway never has them after the substitution.
The token itself is scoped to the org, team, project and route of the request that produced it, and is never written to a log or returned to the caller. A token minted for one project cannot be used to restore content in another.
Configuration#
entities is passed through to the service. Leave it empty to use whatever recognizer set the service is configured for — Rolter does not second-guess your recognizers.
Direction#
request (the default) is the common case: keep personal data away from a third-party model. response scans what the model generated, which catches a model reproducing training data or echoing content from a tool call. both does each leg with a separate call.
Restoration#
Placeholders reach the caller. Nothing is ever reversed, and Rolter tells the sanitizer not to bother storing a mapping.
Every response is restored. Use this when the caller is your own application on your own network and the substitution exists only to keep data away from the model provider.
Restoration only ever reverses placeholders the request leg introduced. Content the provider generated is never restored: reversing that would hand back the very data the response leg exists to remove.
With the response cache#
The response cache stores what the provider returned, placeholders and all. It never stores a restored response. The response leg runs on every delivery, cache hits included, with the restoration token of the request being answered. Two callers whose requests sanitize to the same body share one cache entry, and each gets back only their own values.
With payload capture#
When payload capture is on, the captured request is the sanitized body Rolter forwarded, and the captured response is the body before restoration, placeholders and all. A caller under trusted_downstream or caller_authorized receives the restored plaintext, but the capture store, like the cache, only ever holds the placeholders. Token counts and cost are read from the provider’s response as usual.
Streaming#
The buffered response leg needs the complete body: a placeholder can straddle any number of SSE frames, and a restore call per frame would cost a network round trip per token. So streaming decides what a streamed request gets while the response leg is active:
The request is refused with pii_streaming_unsupported (HTTP 400). This mirrors how guardrails.streaming_post_call behaves.
The stream is served with the response leg skipped: the caller gets placeholders, and provider-generated text is not sanitized.
The stream is restored as it goes, through restore_stream_url (below). Rolter still never holds the mapping: the sanitizer does the restoring. This covers restoration only. When direction is response or both, a streamed request is refused as under reject, because finding entities in a stream needs context the holdback does not give. restore needs restore_stream_url and a restoration other than never, or the configuration does not load.
A request-only setup (direction = "request", restoration = "never") has no response leg, so it streams normally in every mode, and so does a caller_authorized request that does not ask for restoration.
Restoring a stream#
Under streaming = "restore", each streamed response is restored over one full-duplex POST to restore_stream_url: Rolter streams the request body and reads the reply body at the same time, one JSON object per line (application/x-ndjson). This needs HTTP/2 (negotiated over https), or an HTTP/1.1 server that answers while it is still reading the request body.
The first line names the exchange, as a restore call does, and is not answered:
Then Rolter sends the text of each delta of the stream, decoded from its JSON, as it arrives:
A channel is one text field of the stream: a choice’s content, a tool call’s arguments, an Anthropic content block, a Responses API output part. Keep a separate holdback for each. "format": "json" marks a channel whose text is JSON source (tool-call arguments, Anthropic partial_json): write a value restored into it JSON-escaped, as it sits between quotes, so a " in a name does not break the arguments.
When a channel ends (finish_reason, content_block_stop, a Responses API done event, or the end of the stream), Rolter sends {"seq": 18, "close": "0/content"}. The last line is {"seq": 19, "end": true}, after which you may discard the exchange.
Answer every line after the first with exactly one line, in order:
text is the restored text to emit for that line, and held is how many characters (Unicode scalar values) of the channel’s input you are holding back because they could be the start of a placeholder. Held text comes out later, restored, in the reply to a later line of the same channel or in the reply to its close, which must report held: 0. held may never exceed 1,024; a reply that holds more is treated as a failure. Hold back only a suffix that is a prefix of one of this exchange’s placeholders, so other text, a < in code included, is not delayed. Rolter puts reply 17’s text into frame 17 and leaves the rest of the frame as the provider sent it; a frame whose text you held is sent once the held text is released.
Rolter keeps at most 256 lines (or 256 KiB of text) unanswered. When that window is full it stops reading from the provider, so a slow sanitizer slows the stream rather than filling a buffer. If the caller disconnects, the request is reset; discard the exchange’s holdback then too.
A failure is never fatal and never the unsafe way round. If the exchange cannot be opened (a refused connection, a non-2xx status, or a URL the egress policy refuses), the stream is served unrestored. You may send the response head together with your first reply: the clock starts when Rolter sends its first delta, not before, so a slow first token does not count against you. If it fails mid-stream (an error, a reset, a reply that breaks the rules above, or no reply for timeout_ms while a line is waiting for one), Rolter emits the text you were holding and every unanswered delta as the provider sent it, then carries on unrestored: nothing is duplicated or lost, and the caller sees placeholders from that point. Rolter does not retry, since a new exchange would need holdback state you have discarded. Both count in rolter_pii_restore_errors_total.
Restoration on a stream never restores more than the buffered path would: a placeholder the model mangled (PERSON_1, <Person_1>) stays as it is in both. A cached SSE response is restored the same way when it is replayed, with the replaying request’s own token. Usage and cost are read from the provider’s stream, and payload capture stores it before restoration. A streamed request that the provider answers without a stream (plain JSON) is delivered with its placeholders and counted in rolter_pii_restore_errors_total.
Presidio has no streaming endpoint, so it needs a small service in front that wraps deanonymize with the holdback.
Failure mode#
fail_open forwards the unsanitized body when the sanitizer is unreachable — availability over enforcement. fail_closed refuses the request with HTTP 502 pii_sanitizer_unavailable.
If the sanitizer is a compliance control rather than a best-effort improvement, use fail_closed. Under fail_open an outage silently sends untouched personal data to the provider.
The sanitizer cannot run on a realtime session. Under fail_closed, a /v1/realtime upgrade is refused with HTTP 400 sanitizer_unsupported_on_realtime, so no unsanitized session is opened. Under fail_open the session is admitted and its content reaches the provider unsanitized; protect those with a redact guardrail rule.
A failure on the response leg never fails the request, in either mode: the upstream call already happened and was already billed, so refusing to deliver would spend the caller’s money and return nothing. The rolter_pii_sanitizer_errors_total counter records it.
Egress policy#
url, restore_url and restore_stream_url are checked against the egress policy when the configuration loads and again before every call, and Rolter does not follow a redirect from either. In a control-plane snapshot a denied URL does not switch the sanitizer off: it is listed as a configuration problem and every call is refused, so the sanitizer’s failure mode decides whether the request is rejected or forwarded unchanged.
The service contract#
POST to url:
reversible tells the service whether Rolter will ask for a restore later, so a service can skip storing a mapping nobody will request. truncated: true is added when the body exceeded max_body_bytes and was sent as a preview.
The reply:
Omit restoration_token if the service cannot reverse this exchange. Rolter reads that as “do not restore”, not as an error.
POST to restore_url carries trace_id, tenant, the restoration_token, and the content to reverse; the reply is {"content": ..., "restored": <count>}. A streamed response under streaming = "restore" uses restore_stream_url instead, described in Restoring a stream.
A malformed reply is a failure, not a pass. Defaulting to “allow” here would forward content the gateway believes is sanitized and is not.
What to watch#
All six counters are cardinality-bounded and carry no content:
| Metric | Meaning |
|---|---|
rolter_pii_entities_redacted_total | entities replaced across both legs |
rolter_pii_sanitizer_errors_total | calls that failed (timeout, transport, non-2xx, malformed reply) |
rolter_pii_sanitizer_blocks_total | requests refused under fail_closed |
rolter_pii_restorations_total | successful restores; a restored stream counts once |
rolter_pii_restore_errors_total | failed restores — the caller kept placeholders, from the failure on a stream |
rolter_pii_stream_rejections_total | streamed requests refused under streaming = "reject", or under restore when the response is also sanitized |
A rising rolter_pii_sanitizer_errors_total under fail_open is the alert that matters: it means unsanitized traffic is reaching providers.