Cookbook

Copy-paste recipes for the things people actually build with LLMRouter. Each one links to the full guide for the details — headers, edge cases, error shapes.

Never go down when one provider does

List fallback models in priority order. If the primary model's provider errors or times out before the first token, the gateway walks the list automatically — you get one response, from whichever model actually served it.

resp = client.chat.completions.create(
    model="anthropic/claude-opus-4.8",
    extra_body={
        "models": ["openai/gpt-5", "google/gemini-2.5-pro"],
    },
    messages=[{"role": "user", "content": "Summarize this contract..."}],
)

# which model actually served it
print(resp.model)

Full guide: model fallbacks →

Stream tokens as they arrive

Standard SSE, identical shape to the OpenAI API — stream: true and iterate the chunks.

stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Write a haiku about routing"}],
    stream=True,
)
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Full guide: streaming →

Force valid JSON out of any model

response_format is normalized across providers — the same JSON-schema body works whether the request lands on OpenAI, Anthropic, or Gemini.

resp = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Extract the invoice total and due date"}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "invoice",
            "schema": {
                "type": "object",
                "properties": {
                    "total_usd": {"type": "number"},
                    "due_date": {"type": "string"},
                },
                "required": ["total_usd", "due_date"],
            },
        },
    },
)

Full guide: tool calling & structured outputs →

Ask a question about an image

Pass an image_url content block like the OpenAI vision API — works with any vision-capable model in the catalog.

resp = client.chat.completions.create(
    model="auto",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What's in this chart?"},
            {"type": "image_url", "image_url": {"url": "https://example.com/chart.png"}},
        ],
    }],
)

Full guide: image understanding →

Cut costs on work that isn't latency-sensitive

Upload a JSONL file of requests and get a discounted batch price back within the commitment window you choose — same OpenAI-compatible Files/Batches API shape.

batch_file = client.files.create(
    file=open("requests.jsonl", "rb"),
    purpose="batch",
)
batch = client.batches.create(
    input_file_id=batch_file.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
)

Full guide: Batch API →

Never pay for a response you didn't get

If a request errors out after the provider already generated (and would normally bill for) tokens, that spend is refunded automatically — no support ticket required.

Full guide: zero-completion insurance →