POST /v1/messages: Anthropic Messages API for Rolter

The /v1/messages endpoint is compatible with the Anthropic Messages API. Any Anthropic SDK or HTTP client works with Rolter without code changes — set base_url to your gateway address and supply a Rolter virtual key. Rolter accepts both x-api-key (the Anthropic convention) and Authorization: Bearer (the OpenAI convention) so you can use either depending on which SDK you are working with. Rolter handles routing, load balancing, and retries before the request reaches your upstream.

Request#

POST /v1/messages

Headers#

x-api-keystring#

Your Rolter virtual key. This is the header expected by Anthropic SDKs and is the recommended approach when targeting /v1/messages.

Authorizationstring#

Bearer <virtual-key> — accepted as an alternative to x-api-key. Both headers are equivalent within Rolter.

Content-Typestringrequired#

Must be application/json.

x-session-idstring#

Optional session identifier. Pass a stable value to enable affinity-based routing and improve KV-cache reuse across turns in a multi-turn conversation.

Body#

modelstringrequired#

The route name defined in your rolter.toml, for example claude. Rolter looks up this route and rewrites model to the upstream model ID (e.g. claude-sonnet-4-20250514) before forwarding. Returns 404 if no matching route exists.

messagesarrayrequired#

Array of message objects forming the conversation.

Show Message object fields

rolestringrequired#

The speaker role: user or assistant.

contentstring | arrayrequired#

The message content. Pass a plain string for simple text, or an array of content blocks for multi-modal input.

max_tokensintegerrequired#

The maximum number of tokens to generate. The Anthropic API requires this field — requests without it are rejected by the upstream with a validation error.

streamboolean#

Set to true to receive the response as a Server-Sent Events stream. Rolter proxies the upstream SSE stream directly to your client.

Default: "false"

systemstring#

An optional system prompt prepended to the conversation context. Passed through to the upstream unchanged.

temperaturenumber#

Sampling temperature. Passed through to the upstream and subject to the route’s param_policy.

top_pnumber#

Nucleus sampling probability mass. An alternative to temperature.

stop_sequencesarray#

Array of strings at which the model stops generating further tokens.

Warning

max_tokens is required by the Anthropic API. Omitting it produces a validation error from the upstream. Rolter forwards the request body as-is and does not inject max_tokens automatically — set it in [routes.params] if you want a default applied gateway-side.

Examples#

curl http://localhost:4000/v1/messages \  -H "x-api-key: sk-rolter-dev" \  -H "Content-Type: application/json" \  -d '{    "model": "claude",    "max_tokens": 256,    "messages": [      {"role": "user", "content": "Hello!"}    ]  }'

curl -N http://localhost:4000/v1/messages \  -H "x-api-key: sk-rolter-dev" \  -H "Content-Type: application/json" \  -d '{    "model": "claude",    "max_tokens": 256,    "stream": true,    "messages": [      {"role": "user", "content": "Hello!"}    ]  }'

import anthropicclient = anthropic.Anthropic(    api_key="sk-rolter-dev",    base_url="http://localhost:4000",)message = client.messages.create(    model="claude",    max_tokens=256,    messages=[{"role": "user", "content": "Hello!"}],)print(message.content[0].text)

import anthropicclient = anthropic.Anthropic(    api_key="sk-rolter-dev",    base_url="http://localhost:4000",)with client.messages.stream(    model="claude",    max_tokens=256,    messages=[{"role": "user", "content": "Hello!"}],) as stream:    for text in stream.text_stream:        print(text, end="", flush=True)

Self-hosted vLLM with session affinity#

If you run a pool of vLLM instances configured with an OpenAI-compatible provider and your pool serves a model that accepts Anthropic-format requests, you can use x-session-id to pin multi-turn traffic to the same replica.

curl (session affinity)

curl http://localhost:4000/v1/messages \  -H "x-api-key: sk-rolter-dev" \  -H "Content-Type: application/json" \  -H "x-session-id: conversation-xyz" \  -d '{    "model": "llama",    "max_tokens": 512,    "messages": [      {"role": "user", "content": "Summarise the Rust ownership model."}    ]  }'

Response#

A successful non-streaming response follows the Anthropic Messages shape:

{  "id": "msg_abc123",  "type": "message",  "role": "assistant",  "content": [    {      "type": "text",      "text": "Hello! How can I help you today?"    }  ],  "model": "claude-sonnet-4-20250514",  "stop_reason": "end_turn",  "usage": {    "input_tokens": 10,    "output_tokens": 12  }}

idstring#

Unique message identifier, prefixed with msg_.

typestring#

Always "message" for a completed response.

contentarray#

Array of content blocks. Each block has a type field ("text" for standard text) and a text field.

stop_reasonstring#

Why generation stopped: end_turn, max_tokens, or stop_sequence.

usageobject#

Token counts for the request.

Show Usage fields

input_tokensinteger#

Tokens in the input messages.

output_tokensinteger#

Tokens generated in the response.

cache_read_input_tokensinteger#

Input tokens served from the provider’s prompt cache. Present when the provider reports it, including when a Chat Completions or Gemini provider answers a Messages request, and not part of input_tokens.

cache_creation_input_tokensinteger#

Input tokens written to the provider’s prompt cache. Present when the provider reports it, and not part of input_tokens.