Feature flags are deployment-wide switches for the gateway subsystems that are safe to turn on and off at runtime. They are stored in the control plane, edited from Settings → Feature Flags in the dashboard, and applied to every gateway replica through the same snapshot mechanism as hot reload — no restart, no dropped requests.
The flags#
| Flag | What it controls |
|---|---|
response_cache | Serving repeated completions from the shared Redis cache instead of forwarding them upstream. |
cache_aware_routing | Scoring targets by KV-cache affinity, so a conversation keeps landing on the replica that already holds its prefix. |
circuit_breaker | Tripping a failing target out of the pool and probing it before restoring traffic. |
active_health_checks | Polling targets in the background so an unhealthy one is parked before a live request finds it. |
complexity_routing | Estimating prompt complexity and routing cheap requests to smaller models. |
guardrails | Running the configured pre- and post-call guardrail rules. |
The set is a fixed allowlist. There is no free-form flag namespace: each flag maps to a subsystem the gateway knows how to enable at a safe configuration boundary.
Unavailable flags#
A switch that does nothing is worse than no switch. Some subsystems need more than a flag — the response cache needs Redis, and cache-aware routing needs a provider that publishes KV-cache events or LMCache metrics. When a deployment cannot run one, the API reports it as unavailable with a reason, the dashboard renders it greyed out and labelled rather than as a live toggle, and an attempt to enable it is rejected:
Availability is computed from deployment facts, not from the stored row, so a flag left enabled by an earlier deployment still reports as unavailable after the subsystem goes away.
Only turning an unavailable flag on is refused. One that was already on when its subsystem went away — the providers that published KV-cache events were removed, say — does not block saving the other flags, and its switch stays live so you can turn it off. Once off, it cannot be turned back on until the subsystem is available again.
Switches you have flipped are a draft until you save them. Leaving the screen with a draft — through another nav item, the back button, or by closing or reloading the tab — asks you to confirm before the draft is discarded.
API#
Both endpoints are for a superadmin or an operator (a deployment role), and every update writes a feature_flags.update entry to the audit log.
A successful update bumps config_version, so gateways pick the new policy up on their next snapshot poll.