Model settings are the inference parameters Rolter supplies when a request does not. They exist so a fleet of callers you do not control — a notebook, a third-party SDK, someone’s shell alias — still lands on a sane temperature and a bounded completion length.
Edit them from Settings → Model Settings in the dashboard, or through the management API.
Defaults only ever fill a gap#
A parameter the client sent is never overwritten. That includes an explicit null, which is a deliberate “use the provider’s own default” and is left alone.
This is what makes the feature safe to turn on mid-flight: enabling it cannot change the meaning of a request that was already explicit. It can only change requests that said nothing.
Model settings are applied last. A route’s own params, with their override policy, are resolved first, so when a route and these defaults both set temperature the route’s value is the one sent; a default here only fills a parameter that is still unset after that.
The whole feature is off by default. With enabled false — or with every field empty — request bodies reach the provider untouched.
On the dashboard the fields are disabled while the switch is off, and they are not validated then: an out-of-range value already stored never blocks Save, and saving keeps it unchanged. Turning the switch back on checks every field again.
Sampling#
| Field | Range | Notes |
|---|---|---|
| Temperature | 0 – 2 | |
| Top-p | 0 – 1 | |
| Max tokens | 1 – 1000000 | Completion-length cap |
These apply to chat-shaped traffic only: /v1/chat/completions, /v1/messages and /v1/responses. Embeddings, audio and image endpoints take no sampling parameters and are left alone.
Rolter writes the key the upstream dialect actually reads, after any translation — max_tokens for OpenAI chat and Anthropic Messages, max_output_tokens for the Responses API. A request that already set max_completion_tokens on OpenAI chat has stated its cap, so no default is added.
Gemini nests these parameters under generationConfig and does not take defaults from this screen.
Default model#
Used when a request arrives without a model field, or with an empty one. It can name a routed model or a group-slug/model address.
Leave it empty to keep rejecting such requests. That is usually what you want: silently picking a model for a caller who did not name one makes their bill and their latency depend on a setting they cannot see.
What lives elsewhere#
Upstream timeouts and the retry budget are deployment-wide and are configured under Settings → Performance Tuning — see Performance tuning. They are not duplicated here, so there is one source of truth for each.
API#
Both endpoints are superadmin only: no deployment role reaches them. Every update is audited.
Any field may be null, meaning “no default for this parameter”.