Candidate list

With model:"auto", the router doesn't just pick one model — it returns a ranked list (e.g. gpt-4o-mini → claude-3-5-haiku → …), and the gateway walks it in order on failure. With an explicit model id, the candidate list starts as just that one slug, capped at 6 candidates overall once extra models are added.

What triggers a fallback

Only transient / capacity upstream statuses fail over to the next candidate — a real client-side mistake (bad request, auth) does not, since retrying it elsewhere wouldn't help:

Up to 3 attempts are made against a given deployment before moving to the next candidate model.

Extending it: models[]

Pass a top-level models array — the same parameter OpenRouter uses for model-layer fallback — to append your own ordered list of backup slugs after whatever the primary pick (explicit or auto-routed) already has:

curl https://api.llmrouter.sh/v1/chat/completions \
  -H "Authorization: Bearer llmr_sk_live_..." \
  -H "content-type: application/json" \
  -d '{
    "model": "openai/gpt-4o",
    "models": ["anthropic/claude-3-5-sonnet", "openai/gpt-4o-mini"],
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Streaming caveat

Fallback only happens before the first token. Once bytes have started streaming to you, a mid-stream upstream error can't be transparently swapped for another model — it surfaces as an SSE error event carrying the request's trace id instead. Non-streaming requests can fail over across the full response. See Streaming for the full event shape.

Unhealthy-model demotion

A per-model circuit breaker tracks failure rates across all traffic (not just yours) and pre-demotes a model out of the routable set once it's tripped, rather than making every caller discover the outage independently. This runs at the model level, complementary to the deployment-level cooldown described in Provider selection.

Seeing it happen

The response headers x-llmrouter-fallbacks (the ordered candidate list) and x-llmrouter-primary-model (present only when a fallback actually served) tell you in-band whether this happened. Every request's full trace — fallback_used, fallbacks_tried, fallback_latency_ms — is browsable at /logs.