Plugins

Open Plugins to arrange desired request and response middleware across three stages:

• Pre-route — before Rolter selects a route.
• Pre-upstream — after routing and immediately before provider delivery.
• Post-response — after the provider returns and before client delivery.

Each webhook configuration has a stable slug, an organization or project scope, an order, a failure policy, an HTTP endpoint, an optional secret environment-variable reference, and a JSON object with plugin-specific settings.

An enabled row is live: the gateway consults it on every matching request at its configured stage, in position order alongside every other enabled plugin at that stage for the same organization/project.

Scope#

An instance with no project_id applies to every project in its organization. A project-scoped instance applies only to that project. If both an org-level and a project-level instance are enabled at the same stage, both run, interleaved by position — the org tier does not always run first or last.

Dispatch#

At each stage the gateway sends the request or response body under inspection to the plugin’s endpoint as JSON, and the plugin replies with a decision:

• allow — forward unchanged
• transform — replace the content with the plugin’s response before continuing to the next plugin (or upstream/the client)
• block — reject the request with an OpenAI-compatible error, ending the chain
• annotate — forward unchanged, annotations are recorded but not applied

The request body#

The gateway POSTs one JSON object to the endpoint. Its fields:

Contract version#

contract_version names the shape of the body above. Rolter sends exactly one version, the current one, in every call; it does not negotiate and it does not send two shapes at once.

The rule for changing it, which is also the rule your endpoint can rely on:

• The number stays the same for an additive change: a new optional field, or a new value of a field whose documentation already says new values may appear. Write your endpoint to ignore fields it does not know, and to treat a field that is absent as absent.
• The number goes up for anything that can break an endpoint written against the old body: removing or renaming a field, changing a field’s type, or changing what a field means.
• A bump is announced in the release notes of the release that makes it, and the previous number stops being sent in that release.

An endpoint that wants to refuse a version it was not written for can answer with a non-2xx status; the plugin’s failure policy then decides whether the request continues. The decision a plugin returns (allow, transform, block, annotate) is not versioned by this number.

Streamed responses#

A post_response plugin judges a whole response body, and a streamed response is never a whole body until it has already reached the caller. What happens to a streamed request ("stream": true) depends on the failure mode of the post_response plugins that apply to it:

• Any applicable plugin is fail closed: the request is refused before it reaches the model, with HTTP 400 and code plugin_streaming_unsupported (param: "stream"). The message names the plugin. A fail-closed plugin has to approve every response, so a stream it cannot see is never delivered. The same request without "stream": true runs the chain as usual.
• Every applicable plugin is fail open: the stream is delivered without running the post_response chain. Fail open already means the response is delivered when the plugin cannot give a verdict, which is the only thing a stream allows.

To keep streaming available under a post_response plugin, set that plugin to fail open. Refusals are counted in rolter_plugin_stream_rejections_total. Output guardrails (guardrails.streaming_post_call) and the PII sanitizer (pii_sanitizer.streaming) make the same choice for their own response checks; the sanitizer can also restore a stream instead.

The dashboard shows which of the two applies before you save. Once Post-response is picked in the plugin dialog, a note under the pickers follows the chosen Failure policy: with fail closed it says streamed requests in the plugin’s scope are refused, with fail open it says streams skip the plugin unchecked. On the Plugins screen, every post-response plugin set to fail closed carries a one-line warning on its card that it refuses streamed requests while enabled. A paused plugin applies to no request, streamed or not.

The response cache does not change which responses the chain sees. Every non-streaming delivery runs it, including the first response on a cached route and every exact or semantic cache hit. The cache stores the provider’s response before the chain runs. A transform is applied again on each delivery rather than saved, and a plugin added or changed by a hot reload applies to entries that were cached before it existed.

Failure policy#

• Fail open favors availability: an unreachable, timed-out, or erroring plugin is treated as allow.
• Fail closed favors enforcement: the same failure blocks the request with an OpenAI-compatible error.

A block from any plugin in the chain stops it immediately — later plugins in the same stage never run.

A post_response block, or a transform that rewrites the usage object, does not change what the request is billed. The provider has already generated the answer, so its reported usage counts toward budgets and is logged with withheld = 1 when delivery is refused. See Withheld responses.

What each plugin did#

Every plugin call leaves an entry on the request’s row in LLM Logs (plugin_events): allowed, blocked, transformed or annotated, followed by the plugin’s slug, or error:<slug>:<kind> when the call produced no verdict. The kind is one of refused (the egress policy turned the endpoint away), timeout, connect, status_<code> (a non-2xx answer) or body (an unreadable response). Slugs and that fixed list only: nothing the plugin answered, and nothing from the request, reaches a row. A request refused by a plugin is logged like any other refusal, with the entries of the plugins that had already run.

A call that failed is an error whatever the failure policy then did. A fail-closed plugin whose endpoint is down refuses the request, and the row says error:<slug>:connect, not blocked, because the plugin never gave a verdict. Counts of blocked are verdicts only.

Each card on Plugins states what its plugin did over the last seven days: Calls, Blocked, Transformed and Errors (and Annotated once there is one), and the newest failure with its cause and how long ago it happened. A plugin that is enabled and failed every call in the window is flagged Failing every call, with what its failure policy is doing because of it: fail open says requests continue unchecked, fail closed says the requests it covers are being refused. A plugin with no calls in the window says No calls in the last 7 days, which is not the same as healthy: a plugin outside the route’s scope is never called. A paused plugin keeps its counts but is never flagged.

The counts are read from ClickHouse, the same way as the guardrail counts. Without CLICKHOUSE_URL the screen says so once, and with ClickHouse but without clickhouse/019_plugin_events.sql applied it says the counts are not being kept (see Upgrading); neither is shown as “no calls”. Counts are scaled by the log’s sampling weight, so at a sample rate below 1 they are estimates and the last error is the newest one among the sampled rows. Plugin calls made on a Realtime session are counted here too: they are written onto the session’s next log row, and onto one more row when the session closes if no response finished after them. The same rollup is GET /api/v1/analytics/plugins (see Dashboard analytics). A slug is only unique within an organization, so the rollup reports org_id beside it; an org-wide plugin and a project plugin of one organization that share a slug are counted together.

Egress policy#

A plugin’s endpoint is checked against the egress policy when it is saved (a 400 names the field) and again before every call. Rolter does not follow a redirect from a plugin endpoint.

Secrets#

secret_env contains only an environment variable name such as ROLTER_PLUGIN_TOKEN. Put the value in the gateway environment. The control-plane API never accepts or returns the secret itself.

API#

Reads require viewer access at the organization. Mutations require admin access at the configuration’s organization or project scope, and every mutation is written to the audit log.