Rolter Quickstart: Send Your First Request in 5 Min

The fastest way to try Rolter is the published Docker image — it starts the gateway, dashboard, and built-in fake-llm without a provider account. If you prefer a binary install, see the Installation guide. By the end of this guide you’ll have a running gateway and a verified response from your first AI request.

Install Rolter

Make sure you have Docker installed, then pull the image.

docker pull ghcr.io/rolter-ai/rolter:latest

Tip

Prefer a local binary? The Installation page covers cargo install and uv tool install rolter as alternatives.

Start Rolter

Start the zero-config local stack.

docker run --rm -p 127.0.0.1:4000:4000 -p 127.0.0.1:4001:4001 \  -e ROLTER_ALLOW_OPEN_MODE=1 \  ghcr.io/rolter-ai/rolter:latest

The default single-container command is rolter easy-up: gateway on port 4000 and the dashboard on port 4001. It also includes the deterministic fake-llm route, so no provider credential or config file is needed for the first request.

Note

The image binds every interface inside the container so that the published ports reach it. With no ROLTER_ADMIN_TOKEN the dashboard and management API accept every request as superadmin, so the container refuses to start until -e ROLTER_ALLOW_OPEN_MODE=1 acknowledges that. The 127.0.0.1: prefix on each -p keeps both ports on your own machine.

To serve other hosts, drop the acknowledgement and close the management API with an admin token you keep a copy of: run export ROLTER_ADMIN_TOKEN=$(openssl rand -hex 32) and echo "$ROLTER_ADMIN_TOKEN", then pass -e ROLTER_ADMIN_TOKEN. The token does not protect the gateway. Also replace the bundled sk-rolter-dev virtual key in your rolter.toml with a key of your own: it is public and allows every model. Deleting it without adding another leaves the gateway open. Run the single image has the full command.

Use a real provider (optional)

Mount an annotated config file and provide a provider key when you are ready to route real traffic:

cp rolter.example.toml rolter.tomldocker run --rm -p 127.0.0.1:4000:4000 -p 127.0.0.1:4001:4001 \  -e ROLTER_ALLOW_OPEN_MODE=1 \  -e OPENAI_API_KEY \  -v "$PWD/rolter.toml:/app/rolter.toml" \  ghcr.io/rolter-ai/rolter:latest

Once the containers are running:

Gateway (data plane): http://localhost:4000
Control plane + dashboard: http://localhost:4001

Tip

Need Postgres, Redis, ClickHouse, and separate gateway/control processes? Use the full Compose deployment: docker compose -f docker/docker-compose.yml up -d. That is the open local stack; for a shared host see the team stack.

Send a chat completions request

With the gateway up, call the built-in fake-llm model. It needs no provider account and is intended for smoke tests and local development.

It does need a rolter virtual key: the config easy-up writes on first run ships one, sk-rolter-dev. Every command below presents it. If you edited rolter.toml, use the key from your own [[virtual_keys]] section — easy-up prints a ready-to-run command with it at startup.

curl -s http://localhost:4000/v1/chat/completions \  -H "Authorization: Bearer sk-rolter-dev" \  -H "Content-Type: application/json" \  -d '{"model":"fake-llm","messages":[{"role":"user","content":"hello"}]}'

from openai import OpenAIclient = OpenAI(    base_url="http://localhost:4000/v1",    api_key="sk-rolter-dev",)response = client.chat.completions.create(    model="fake-llm",    messages=[{"role": "user", "content": "Hello!"}],)print(response.choices[0].message.content)

curl -s http://localhost:4000/v1/messages \  -H "Authorization: Bearer sk-rolter-dev" \  -H "Content-Type: application/json" \  -d '{"model":"fake-llm","max_tokens":256,"messages":[{"role":"user","content":"hello"}]}'

You should receive a standard OpenAI or Anthropic response object — exactly what your existing code already expects.

Info

Streaming works too. Add "stream": true to your request body and Rolter will forward the SSE stream from the upstream provider directly to your client.

Verify with the models endpoint

Confirm the gateway is healthy and listing its built-in route:

curl -s http://localhost:4000/v1/models -H "Authorization: Bearer sk-rolter-dev"

The response includes fake-llm. When you mount rolter.toml, it will also list the public model names you configure.

You can also hit the liveness endpoint at any time without a key:

curl -s http://localhost:4000/healthz

Note

A Prometheus-format metrics endpoint is available at GET /metrics for integration with Grafana, Datadog, or any other scraper.

What’s next?#

Now that requests are flowing through the gateway, you can explore more of what Rolter offers:

Configure routes & strategies

Set up weighted routing, cache-aware balancing for vLLM pools, A/B variants, and request parameter policies.

Manage virtual keys

Create scoped keys for each team or service, set expiry dates, and revoke them without restarting the gateway.

Set budgets & rate limits

Add monthly spend caps and per-minute token limits per org, team, project, or individual key.

API reference

Browse the full list of supported endpoints, request fields, and authentication schemes.