System One: Decision Models Through Rolter

Decision models do not write text. They answer typed questions about an input (pick one of several options, score on a rubric, yes or no) and return probabilities and a confidence. They are used for routing, triage, classification and guard checks, where a generative model is slow and its answer has to be parsed.

The servers below all speak one protocol, the System One API (POST /v1/systemone), and Rolter fronts all of them with one provider kind, system_one:

Callers send decisions to Rolter’s POST /v1/systemone with a Rolter virtual key. Virtual keys, budgets, rate limits, route caps, balancing, retries, request logs and cost all apply, as they do for chat.

Configuration#

# TypeSafe's hosted Jev[[providers]]name = "typesafe"kind = "system_one"api_base = "https://api.typesafe.ai"api_key_env = "TYPESAFE_API_KEY"# a self-hosted vLLM Semantic Router engine serving vllm-sr/d3-edge[[providers]]name = "vllm-sr"kind = "system_one"api_base = "http://vllm-sr:8899"[[routes]]model = "jev"strategy = "round_robin"[[routes.targets]]provider = "typesafe"model = "jev-latest"

api_base is the server root, without /v1: Rolter appends /v1/systemone itself. The key, when the server takes one, is sent as Authorization: Bearer. An open vLLM Semantic Router listener and a local Ollama take no key, so leave it out for those.

In the dashboard, pick System One (decisions) as the kind. The preview under API base shows the decision endpoint the provider will be called at.

Health checks#

The free probe is GET {api_base}/v1/models, which TypeSafe and Ollama both answer with a model list. A vLLM Semantic Router engine publishes its list at /v1/systemone/models instead. If it answers /v1/models with a 404, the provider still reads as healthy, since any answer below 500 does, but its models are not discovered, so name them on the route targets. The opt-in LLM-call check asks one yes/no question and proves a decision works end to end.

Pricing#

Rolter ships no price list. Jev bills input tokens only, at $0.042 per million tokens as of October 2026 (TypeSafe models); output is free. That is a price row with a zero output rate:

[[model_prices]]model = "jev"input_per_mtok = 0.042output_per_mtok = 0

The cost of each decision is read from the usage.input_tokens and usage.output_tokens the server reports, so request logs never mark a decision as having unknown usage. A self-hosted server’s tokens cost whatever you price them at, including nothing.

Which endpoints a decision model answers#

A decision model cannot generate text, embed or transcribe, and a hosted chat API has no decision endpoint. Rolter checks this per route, before anything is forwarded:

• A route whose targets are all system_one answers only POST /v1/systemone. A chat, embeddings or audio request for it gets 400 with code unsupported_endpoint and a message naming /v1/systemone.
• POST /v1/systemone needs a route with at least one target of kind system_one, openai_compatible or ollama. The last two are allowed because a self-hosted server or another gateway behind them may serve the endpoint beside chat. A decision for any other route gets the same 400.

Keep decision models in routes of their own. A route that mixes decision and chat targets is not refused, and requests of either kind will reach targets that cannot answer them. It is reported, though: rolter check, the gateway at startup and the dashboard’s config problems all name such a route and the targets on each side.

Guardrails and privacy#

Guardrails scan every string in the decision’s state and in each question’s instructions and criteria, so a redaction rule masks an email inside a ticket the same way it does inside a chat message, and a block rule refuses the request. Question ids, option keys and type are never rewritten. Images and videos are not scanned.

The external PII sanitizer and the guardrail webhook receive the whole request body, as they do for every endpoint.

Limits#

Rolter does not cap the number of questions; the servers do (Ollama takes up to 64, Jev a 64k-token context per request). The request body is bounded by server.max_body_bytes. vLLM Semantic Router’s d3 models accept images and video in the body, so a deployment sending video may need to raise that limit.

Without a key or a GPU, use the built-in fake-llm model: it answers choice, score and noul questions deterministically.