Rolter ships in three forms: a Docker Compose stack that starts the gateway alongside its supporting services, a native Rust binary you can install with Cargo, and a Python-packaged wheel you can install with uv or pip. Pick the method that fits your environment — Docker Compose is the easiest way to get everything running, while the binary options are better suited for minimal setups and embedding Rolter in existing infrastructure.
Prerequisites#
Before you install, make sure you have at least one of the following depending on your chosen method:
- Docker Compose: Docker Desktop (macOS/Windows) or Docker Engine + the Compose plugin (Linux)
- Cargo: Rust toolchain (stable, 1.94 or later — the project MSRV)
- uv / pip: uv or Python 3.9+ with pip
You’ll also need at least one upstream provider API key ready to export:
Rolter never stores provider API keys in its config file. Instead, it reads them from the environment variables you reference with api_key_env in rolter.toml. Set those variables before starting the gateway.
Docker Compose (recommended)#
Docker Compose brings up the complete Rolter stack — gateway, control plane, Postgres, Redis, and ClickHouse — with a single command. Database schemas are applied automatically on first start. The steps below start the local stack, which is open on purpose and meant for one machine nobody else can reach. For a team on a shared host, use the closed team stack described on the Docker page.
Clone the repository
Set your provider keys
Compose reads variables from the shell and from docker/.env, not from a .env at the repository root. Export your provider API keys before starting the stack:
Start the stack
Once the containers are running, the gateway and control plane are available at:
| Service | URL |
|---|---|
| Gateway (data plane) | http://localhost:4000 |
| Control plane + UI | http://localhost:4001 |
Follow gateway logs in real time with docker compose -f docker/docker-compose.yml logs -f gateway to confirm your config loaded cleanly.
Other install methods#
The unified rolter binary exposes three subcommands: rolter gateway (data
plane), rolter control (control plane + UI), and rolter easy-up (both, plus
the UI, in one supervised process). easy-up creates rolter.toml from a
bundled example on first run and answers on the built-in fake-llm model with
no keys and no database; pass --database-url to run migrations, seed a default
org/team/project, and serve config from Postgres.
The cargo install and uv/pip methods install the rolter binary only — they do not start Postgres, Redis, or ClickHouse. Budget enforcement and request logging require those services. For a local-only setup without persistence, the gateway works fine without them; budgets will fail open and request logs will be skipped.
Default ports#
| Port | Service |
|---|---|
4000 | Gateway — handles all /v1/* AI requests from your clients |
4001 | Control plane — hosts the dashboard and configuration API |
5432 | PostgreSQL (Docker Compose only) |
6379 | Redis (Docker Compose only) |
8123 / 9000 | ClickHouse HTTP / native (Docker Compose only) |
Do not expose port 4001 (control plane) to the public internet. Place the gateway (port 4000) behind a TLS-terminating load balancer or ingress, and keep the control plane on an internal network.
Configure your first route#
After installing, create a rolter.toml from the example config:
The example file is fully annotated and ready to use. It defines routes for gpt-4o (OpenAI), claude (Anthropic), and a sample vLLM pool — edit the provider entries to match your setup, then start the gateway.
Head to the Quickstart to send your first request.