This guide takes a deployment from nothing to “our people and our applications call our models through Rolter, inside limits we chose, and we can see what they did.” Each step links the page that covers it in depth; follow the links only when you need more than the step says.
Just want to see it work first? rolter easy-up starts everything on your own machine with no database and no provider key, and the built-in fake-llm model answers straight away. See the quickstart. Come back here when you are ready to run it for a team.
Generate the deployment secrets
Do this before the control plane is reachable from anywhere but your own machine. rolter init writes a production rolter.toml and a .env holding five generated values:
ROLTER_ADMIN_TOKENturns on role-based access control and protects the management API. Without it the control plane runs open: everyone who reaches it is a superadmin, whatever accounts exist in the database.ROLTER_INTERNAL_TOKENis what the gateways present to fetch their configuration from the control plane.ROLTER_KEKseals provider keys at rest.ROLTER_KEY_PEPPERpeppers the stored digests of virtual keys.ROLTER_SESSION_PEPPERpeppers the stored digests of sign-in sessions, invitation links, SCIM tokens and MFA recovery codes.
Keep .env out of version control and somewhere it outlives a restart: a new KEK strands every stored provider key, and a new pepper invalidates every virtual key or session issued under the old one. Then run rolter check with that environment loaded. It fails on a missing KEK, admin token or database URL, and warns on a missing key pepper, a missing session pepper or a control plane bound to every interface. See Production, Environment variables and Which key do I need?.
Install the stack
A team deployment runs the gateway, the control plane and dashboard, Postgres, Redis and ClickHouse. Postgres is what makes the dashboard’s create, edit and delete screens exist; Redis keeps budgets and rate limits accurate across gateway replicas; ClickHouse holds the request logs.
For a team, install the Helm chart (charts/rolter) with the values from the previous step as its secret environment. It points every gateway at the control plane and runs rolter check in an init container before each pod starts. The control plane’s init container runs every check, and the gateway’s runs only the ones that apply to a gateway. Supply the admin token, KEK, database URL and peppers through secretEnv, and the check passes with the chart’s 0.0.0.0 bind, since that bind is only a warning while the management API is open. See Production and Cluster deployment.
For one shared host, Docker Compose runs the same stack. Layer docker/docker-compose.team.yml over the compose file and pass the .env from the previous step. It reads every value rolter init wrote, takes three datastore passwords and their connection URLs, closes the management plane behind the admin token, and points the gateway at the control plane with the internal token and the key pepper. Postgres is published on 127.0.0.1 only, and Redis and ClickHouse are not published at all.
Generate the file for the compose stack with rolter init --env-only --compose, which adds those passwords and URLs to the five values above. Then:
The preflight service runs rolter check --strict on the values the stack will use, and up refuses to start while one of them is missing. The dashboard is at http://127.0.0.1:4001, so put a reverse proxy that terminates TLS in front of it for everyone else. The Docker page has the details.
docker/docker-compose.yml on its own is the local stack, and it is open on purpose. It binds the control plane to every interface, acknowledges open mode (ROLTER_ALLOW_OPEN_MODE=1), publishes port 4001 and sets no ROLTER_ADMIN_TOKEN, so anyone who can reach port 4001 is a superadmin. It also publishes every datastore on every interface: Postgres on 5432 with the example rolter/rolter login, Redis on 6379 with no password, and ClickHouse on 8123 and 9000 with a passwordless default user. Whoever reaches those ports can write an account straight into Postgres, reset budget counters in Redis, or read every tenant’s request logs and captured bodies from ClickHouse, whatever the control plane enforces. Adding ROLTER_ADMIN_TOKEN to it by hand does not make it safe to share. Run the local stack only on a machine nobody else can reach, to try Rolter before you set up the team stack.
See Installation.
Create the first administrator and sign in
rolter seed creates the account. It ships in the image and in the Python package, and connects to the deployment’s database through ROLTER_DATABASE_URL:
The password is read from standard input (--admin-password-stdin, the first line) or from a file (--admin-password-file <path>). --admin-password <value> still works, but a value on the command line is visible to every user on the host through ps and lands in shell history, so the three are mutually exclusive and the first two are the ones to use. Write password.txt with chmod 600 (or umask 077) before you put the password in it, and delete it as soon as rolter seed has run.
It is safe to run again: it creates the org, team, project and administrator once and never touches an existing account. rolter seed --import rolter.toml applies a config file as desired state (Config export).
Open the dashboard (port 4001) and sign in. The account belongs to no organization, so it is not in any organization’s people list until you ask for it: a superadmin sees such accounts under Governance → Users. Change the password you seeded under Settings → Account & keys → Password: it asks for the current password and the new one twice, and signs the account out of its other sessions while keeping the one you are in (Change your password). Then protect the account with a second factor on the same screen, under Two-factor authentication. See Two-factor authentication.
Follow the Getting started card
A fresh deployment opens on a Getting started card with the four steps between an empty control plane and a client calling Rolter, each ticked from the deployment itself:
- Make a call with no credentials — the Playground answers with
fake-llm. It mints its own short-lived key, which for a project with no routes yet reachesfake-llmalone, so this works before steps 2 and 3. It runs inside a project, so on a deployment with none the card asks you to create one first: its Create project button opens the same dialog as the + beside Project in the scope switcher. - Connect a provider — Models → Model Providers → + Add provider. The key is sealed and never shown again; Test connection proves it before anything depends on it.
- Add a route — Models → Routing Rules → + Add route, or + Add route on Model Catalog, which opens the same sheet: the public model name your clients will send, its strategy, and the provider it lands on. With more than one provider, pick it yourself; the sheet does not choose for you.
- Mint a virtual key — Governance → Virtual Keys. Applications authenticate with it, never with your provider key.
Connecting more than one model, or a whole self-hosted fleet, is its own guide: Connecting models.
Lay out teams and projects
Most screens work on one organization → team → project chain, chosen in the scope switcher at the top of the rail; the ⋯ menu on its Team and Project rows creates a team or a project. Put the boundaries where access and money should split — a project per product or workstream is a good default. See Dashboard.
Bring people in
Pick the way your company signs in:
- No identity provider: Governance → Users → Invite user at a role and scope, for the whole organization, a team or a project. You get a one-time link to send to the person; they choose their own password. Someone who already has a rolter account keeps it: the link adds the role, and they sign in as usual. Links not yet accepted are listed under the users table, where you can revoke one sent to the wrong address. See Inviting people.
- An OIDC identity provider (Okta, Entra ID, Google, Keycloak): register it under Governance → Single Sign-On, map its groups to roles at scopes, and decide whether passwords stay on. See Single Sign-On.
- Automatic joiners and leavers: add SCIM provisioning under Governance → User Provisioning. See User Provisioning.
Make engineers members of their projects: members mint their own keys, and read their own project’s request logs and captured bodies. Viewers read without changing anything, and see request logs without the captured bodies unless a project admin allows it. See RBAC.
Price your models and cap spend
Give every model that costs money a price (Models → Model Prices) so spend is never counted as zero by accident, then add budgets and rate limits under Models → Budgets & Limits at the org, team, project or key. A request past a budget gets HTTP 402; past a rate limit, HTTP 429. The whole topic is its own guide: Tracking and capping spend.
Watch the traffic
Observability → Dashboard shows traffic, spend, latency and errors; LLM Logs shows every request with its status, target, tokens and cost. Every change anyone made is under Governance → Audit Logs. Sign-ins and second-factor changes are recorded there too. See Logs & Cost and What to look at.