The bundled Compose file runs the gateway, the control plane and all three backing stores. It comes in two shapes, and the one you start decides who can reach your data.
| Local stack | Team stack | |
|---|---|---|
| For | One person, on a machine nobody else can reach | A team sharing one host |
| Start it with | docker/docker-compose.yml | The same file plus docker/docker-compose.team.yml and an env file |
| Management plane | Open. There is no admin token, so everyone who reaches port 4001 is a superadmin | Closed behind ROLTER_ADMIN_TOKEN, published on 127.0.0.1 |
| Datastores | Published on every interface. Postgres has the example rolter/rolter login, Redis has no password, ClickHouse has a passwordless user | Each has a password from the env file. Only Postgres is published, on 127.0.0.1 |
| Secrets | None | KEK, admin and internal tokens, key and session peppers, three datastore passwords |
| Gateway | Follows the control plane | Follows the control plane, with the internal token and the key pepper |
The local stack is open on purpose. Do not start it on a shared host, a server or a cloud VM. Adding ROLTER_ADMIN_TOKEN to it by hand closes the management API only: Postgres, Redis and ClickHouse stay reachable from the network, so anyone there can write an account into Postgres, reset budget counters in Redis or read every request log from ClickHouse around the control plane. Use the team stack.
Both shapes build the image from your checkout on the first start, which takes several minutes. The build turns on the postgres feature. Without it the control plane ignores the database URL, keeps everything in memory and has no sign-in and none of the dashboard’s create, edit and delete screens. The published ghcr.io/rolter-ai/rolter image is built without that feature (#2405), so it cannot replace the build here yet.
Both shapes need Docker Compose 2.24 or later.
Local stack#
Start the stack
Bring up all five services in the background:
Docker Compose starts PostgreSQL, Redis and ClickHouse first, then the control plane, then the gateway. On first boot the control plane applies the database schema itself, so no manual SQL is required. The gateway polls the control plane for its configuration: a provider, route or virtual key you create in the dashboard reaches it within a few seconds, without a restart.
ClickHouse asks for 262144 open files. If the ClickHouse container fails to start with error setting rlimit type 7: operation not permitted, the container runtime (often rootless Docker or Podman) cannot grant that many. Set a lower limit in docker/.env and start again: run echo CLICKHOUSE_NOFILE=20000 >> docker/.env, then docker compose -f docker/docker-compose.yml up -d. Compose reads docker/.env because it sits beside the first -f file; a .env at the repository root does not set this value.
The stack turns off the diagnostic tables ClickHouse writes about itself, which keep a small host with a slow disk busy while rolter is idle. A volume created by an earlier version still holds them, and a small host should cap ClickHouse’s caches too: see ClickHouse on a small host.
Give it provider keys (optional)
The bundled rolter.toml reads OPENAI_API_KEY and ANTHROPIC_API_KEY from the gateway’s environment. Export them before up, or put them in docker/.env. A .env at the repository root is read only if you pass --env-file .env.
To add providers in the dashboard instead, export a key-encryption key before up. Without one the dashboard refuses to store a provider key.
This key seals provider keys in the database. Keep it: a new one strands every key stored under the old one.
Watch the gateway logs
Confirm the gateway started cleanly by tailing its logs:
You should see the gateway announce the port it is listening on and that it applied a config snapshot from the control plane. Press Ctrl+C to stop following.
Verify the services are running
Once all containers are up, the stack exposes the following default ports:
| Service | Address | Notes |
|---|---|---|
| Gateway | http://localhost:4000 | OpenAI/Anthropic-compatible API |
| Control plane + UI | http://localhost:4001 | Dashboard and config API |
| PostgreSQL | localhost:5432 | Source of truth for all config |
| Redis | localhost:6379 | Config pub/sub and hot-reload events |
| ClickHouse (HTTP) | localhost:8123 | Request logging and analytics |
| ClickHouse (native) | localhost:9000 | ClickHouse native protocol |
Open http://localhost:4001 in your browser to access the dashboard.
Send a test request
Confirm the gateway is healthy and accepting traffic:
A 200 OK from /healthz and a JSON model list from /v1/models confirm the gateway is fully operational.
Team stack#
The team stack layers docker/docker-compose.team.yml over the same file. It closes the management plane, gives every datastore a password and reads every secret from an env file, so a value you forgot stops docker compose up with a message instead of starting something half-configured.
Generate the secrets
rolter init writes the five values the control plane and gateway share (install it with uv tool install rolter, see Installation). Add --compose and it also writes the three datastore passwords and the connection URLs built from them. This is the one command a fresh clone needs before docker compose up:
| Variable | What it does |
|---|---|
ROLTER_KEK | Seals provider keys in the database |
ROLTER_ADMIN_TOKEN | Closes the management API and turns on role-based access control |
ROLTER_INTERNAL_TOKEN | What the gateway presents to fetch its configuration |
ROLTER_KEY_PEPPER | Peppers virtual-key digests. The control plane and the gateway must hold the same value |
ROLTER_SESSION_PEPPER | Peppers the digests of sign-in sessions, invitations, SCIM tokens and recovery codes |
ROLTER_PG_PASSWORD, ROLTER_REDIS_PASSWORD, ROLTER_CLICKHOUSE_PASSWORD | Datastore credentials, generated as hex so they sit inside URLs unescaped |
ROLTER_COMPOSE_DATABASE_URL, ROLTER_COMPOSE_REDIS_URL, ROLTER_COMPOSE_CLICKHOUSE_URL | The in-network connection URLs with those passwords in them. The compose file reads them whole, so it carries no credential of its own |
--compose writes the user rolter into the URLs, which is also the default of ROLTER_PG_USER and ROLTER_CLICKHOUSE_USER. If you set either to another name, change the user in the matching URL too. Without --compose you write all six values yourself. ROLTER_CONTROL_HOST is the address the control plane is published on, 127.0.0.1 in the file rolter init writes.
Keep .env out of version control and somewhere it outlives a restart. A new KEK strands every stored provider key, and a new pepper invalidates the keys or sessions issued under the old one. The datastore passwords are set when a volume is first created, so changing one in .env later does not change it inside an existing database.
Check the environment
The preflight service runs rolter check --strict against the same values the control plane will get, with the binary that will run it:
It prints all pre-boot checks passed. --env-file goes before run: the subcommand has no such flag. A warning fails the check, for example when ROLTER_CONTROL_HOST is 0.0.0.0 and ROLTER_ADMIN_TOKEN is unset, which would publish an unauthenticated management plane on every interface. With the token set, an all-interfaces bind passes, because every management route is authenticated.
Start the stack
The team stack has its own project name, rolter-team, so it never reuses the volumes of a local stack you ran before, which hold the example credentials. Every later command (logs, ps, down) needs the same two -f flags and --env-file.
Create the first administrator and sign in
rolter seed ships in the image, so run it in the control container; it already has the database URL:
Postgres is also published on 127.0.0.1, so the same command works from a host that has the rolter CLI (pip install rolter) with ROLTER_DATABASE_URL="postgres://rolter:<ROLTER_PG_PASSWORD>@127.0.0.1:5432/rolter" set.
Open http://127.0.0.1:4001 and sign in. The provider keys you add in the dashboard are sealed under ROLTER_KEK, and the gateway picks up providers, routes and virtual keys within seconds.
Put TLS in front of it
The gateway is published on every interface at port 4000, and it authenticates every request with a virtual key. The control plane is published on 127.0.0.1 only. For people who are not on this host, run a reverse proxy that terminates TLS and forward it to port 4001 for the dashboard and to port 4000 for the API. /internal/*, which carries decrypted provider keys, is on a separate port, 4002, that is not published, so a proxy cannot forward it.
What the team stack publishes:
| Service | Address | Notes |
|---|---|---|
| Gateway | 0.0.0.0:4000 | OpenAI/Anthropic-compatible API, virtual keys required |
| Control plane + UI | 127.0.0.1:4001 | Dashboard and management API, admin token or sign-in required |
| PostgreSQL | 127.0.0.1:5432 | Set ROLTER_PG_HOST_PORT to move it |
| Redis and ClickHouse | not published | Reach them with docker compose ... exec redis redis-cli ping and exec clickhouse clickhouse-client |
The team stack does not read the rolter.toml that rolter init writes. It mounts docker/rolter.team.toml, which defines no providers, routes or virtual keys, in its place: the image’s bundled example would otherwise reach the gateway through the control plane with the public sk-rolter-dev key in it. Everything is created in the dashboard. To keep some providers in a file, mount your own over docker/rolter.team.toml and leave [[virtual_keys]] out of it.
The control plane and the gateway print the Redis and ClickHouse URLs they connect to at startup, and any error that quotes a datastore URL, with the password replaced by *** (#2406). The container logs do not carry the datastore passwords; the env file does, so keep it readable by its owner only.
Run the single image#
The published image runs rolter easy-up by default: the gateway, the control plane and the dashboard in one container, answering on the built-in fake-llm model with no config file and no database. Release images are signed and carry an SBOM; Verify images shows how to check one before you run it.
The image’s default command runs rolter easy-up --host 0.0.0.0. Inside a container that is required: a published port forwards to the container’s network interface, never to its loopback, so a loopback bind answers nothing from the host.
Binding every interface with no ROLTER_ADMIN_TOKEN would serve the management API to anyone who can reach it, so the container refuses to start in that state until it is acknowledged:
| You pass | What happens |
|---|---|
| nothing | Refuses to start and says why. |
-e ROLTER_ALLOW_OPEN_MODE=1 | Starts with an unauthenticated control plane and a warning banner. Keep the ports on loopback with -p 127.0.0.1:…, as above. |
-e ROLTER_ADMIN_TOKEN | Starts with the management API closed behind the token. Use this for anything other hosts can reach. |
Generate the token on its own line and keep it, because the dashboard and the management API ask for it. Then pass it through by name:
The admin token closes the management API only. The gateway on port 4000 accepts the virtual keys in rolter.toml, and the one the image ships, sk-rolter-dev, is printed in these docs and allows every model. Before other hosts can reach port 4000, replace it in the [[virtual_keys]] section of your own rolter.toml with a key nobody else knows, and mount that file as above. Deleting the section instead leaves the gateway open to anyone.
Once an admin token is set, the control plane also stops handing sk-rolter-dev to gateways that poll its snapshot, and reports the omission under GET /api/v1/config/problems. A gateway that loads its own rolter.toml directly is not covered by that, so the file is still the place to remove it.
Where sk-rolter-dev can come from, and how each production path avoids it:
| Path | Where the key comes from | What avoids it |
|---|---|---|
| Single image, no mounted config | the example baked in as /app/rolter.toml | mount your own rolter.toml without [[virtual_keys]], or run open on loopback only |
| Team compose stack | none: it mounts docker/rolter.team.toml | nothing to do |
| Helm chart | none: control plane, gateways and preflight read config.file | keep [[virtual_keys]] out of config.file or your existingConfigMap |
rolter init --profile production | none: the generated rolter.toml has no [[virtual_keys]] | create keys in the dashboard or API |
rolter init --profile local | the example, on purpose | do not ship it |
rolter check fails with an error on any config file that still declares sk-rolter-dev.
Run just the gateway#
If you want to run only the gateway, without the control plane or any backing stores, override the command and mount a rolter.toml. This is useful for quick tests or for edge deployments where you manage external databases separately.
Mount your local rolter.toml into the container at /app/rolter.toml. The gateway reads provider credentials from the environment variables you inject, keeping secrets out of the config file. It binds the [server] host from that file, and its clients authenticate with the virtual keys in it: replace the example’s sk-rolter-dev before the port is reachable from other hosts.
To run the control plane instead of the gateway, override the command with rolter-control. It binds loopback unless ROLTER_CONTROL_HOST says otherwise, and it follows the same rule as the single image, so give it an admin token along with the bind:
Build the image locally#
Both binaries and the compiled dashboard UI are packaged into a single slim Debian image. To build it from source:
The build includes the control plane’s database support, sign-in and CRUD API (CARGO_FEATURES=postgres, the Dockerfile default), the same as the published image. Pass --build-arg CARGO_FEATURES= to build an in-memory-only control plane without them.
ClickHouse on a small host#
ClickHouse’s default server config writes its own diagnostics into system.text_log, metric_log, asynchronous_metric_log, trace_log and the rest of the system.*_log family, with an insert every few seconds into each. On a small host with a slow disk (a Raspberry Pi on an SD card was the measured case) the merges those inserts cause never catch up, and an idle ClickHouse measured at 190% to 320% CPU there. Rolter reads none of those tables, so docker/clickhouse/system-logs.xml switches them off for the local stack and, because the team stack layers over it, the team stack too. The SigNoz overlay’s own ClickHouse (signoz-clickhouse) mounts the same file, since SigNoz reads none of them either. system.query_log stays on. Nothing in the file needs editing.
Drop the tables an existing volume already has. The file only stops the tables being created; ClickHouse does not drop a table because its logging was disabled. Run this once against a volume created by an earlier version. The statement list is generated, so the numbered copies (text_log_0) ClickHouse leaves behind after an upgrade go too. In the team stack add --user "$ROLTER_CLICKHOUSE_USER" --password "$ROLTER_CLICKHOUSE_PASSWORD" to both clickhouse-client calls, and the team -f and --env-file flags to docker compose:
A SigNoz overlay volume created before the overlay mounted the file has the tables too: run the same two lines with -f docker/docker-compose.signoz.yml added and signoz-clickhouse in place of clickhouse.
Cap the caches. ClickHouse sizes its mark and uncompressed caches for a large server (5 GiB and 8 GiB by default). Rolter does not pick a smaller figure for you, because the right one depends on the host. To set one, put a file in docker/clickhouse/caches.xml:
Mount it with a second Compose file, docker/docker-compose.caches.yml, and start the stack with both:
select name, value from system.server_settings where name like '%cache_size' shows the values the running server picked up.
Database schemas#
Schema migrations are applied automatically on first container start — no manual SQL or migration commands needed.
The PostgreSQL schema is applied by rolter itself: both the control plane and the seed tool run the migration set on startup and record what they applied. ClickHouse is initialised by the Compose setup before the application containers start.
Subsequent starts skip already-applied migrations, so it is safe to bring the stack up and down repeatedly.
Do not mount crates/rolter-store/migrations into the Postgres container’s
docker-entrypoint-initdb.d. PostgreSQL would run those files itself on first
init, outside rolter’s migration runner, so rolter would see an untracked
database and try to reapply every migration to it — failing on the first
statement that cannot run twice.