> ## Documentation index
> Fetch the complete documentation index at: https://rolter-ai-rolter.docs7.io/llms.txt
> Ask it with GET https://rolter-ai-rolter.docs7.io/ask?q=<keywords or question>. Short keywords rank best.
> Use these to discover all available pages before exploring further.

# Model Settings: Deployment-Wide Inference Defaults

> Sampling parameters and a fallback model Rolter fills in when a client omits them — never overwriting a value the caller sent.

Model settings are the inference parameters Rolter supplies when a request does not. They exist so a fleet of callers you do not control — a notebook, a third-party SDK, someone's shell alias — still lands on a sane temperature and a bounded completion length.

Edit them from **Settings → Model Settings** in the dashboard, or through the management API.

---

## Defaults only ever fill a gap

A parameter the client sent is never overwritten. That includes an explicit `null`, which is a deliberate "use the provider's own default" and is left alone.

This is what makes the feature safe to turn on mid-flight: enabling it cannot change the meaning of a request that was already explicit. It can only change requests that said nothing.

Model settings are applied last. A route's own params, with their override policy, are resolved first, so when a route and these defaults both set `temperature` the route's value is the one sent; a default here only fills a parameter that is still unset after that.

The whole feature is off by default. With `enabled` false — or with every field empty — request bodies reach the provider untouched.

On the dashboard the fields are disabled while the switch is off, and they are not validated then: an out-of-range value already stored never blocks **Save**, and saving keeps it unchanged. Turning the switch back on checks every field again.

---

## Sampling

| Field       | Range       | Notes                 |
| ----------- | ----------- | --------------------- |
| Temperature | 0 – 2       |                       |
| Top-p       | 0 – 1       |                       |
| Max tokens  | 1 – 1000000 | Completion-length cap |

These apply to chat-shaped traffic only: `/v1/chat/completions`, `/v1/messages` and `/v1/responses`. Embeddings, audio and image endpoints take no sampling parameters and are left alone.

Rolter writes the key the **upstream** dialect actually reads, after any translation — `max_tokens` for OpenAI chat and Anthropic Messages, `max_output_tokens` for the Responses API. A request that already set `max_completion_tokens` on OpenAI chat has stated its cap, so no default is added.

Gemini nests these parameters under `generationConfig` and does not take defaults from this screen.

---

## Default model

Used when a request arrives without a `model` field, or with an empty one. It can name a routed model or a `group-slug/model` address.

Leave it empty to keep rejecting such requests. That is usually what you want: silently picking a model for a caller who did not name one makes their bill and their latency depend on a setting they cannot see.

---

## What lives elsewhere

Upstream timeouts and the retry budget are deployment-wide and are configured under **Settings → Performance Tuning** — see [Performance tuning](/reliability/performance-tuning). They are not duplicated here, so there is one source of truth for each.

---

## API

Both endpoints are **superadmin only**: no [deployment role](/security/rbac#deployment-roles) reaches them. Every update is audited.

```bash
curl -H "Authorization: Bearer $ROLTER_ADMIN_TOKEN" \
  https://control.example.com/api/v1/model-defaults

curl -X PUT -H "Authorization: Bearer $ROLTER_ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
        "enabled": true,
        "default_model": "gpt-4o-mini",
        "default_temperature": 0.7,
        "default_top_p": null,
        "default_max_tokens": 2048
      }' \
  https://control.example.com/api/v1/model-defaults
```

Any field may be `null`, meaning "no default for this parameter".