Batch API

Big jobs, up to 50% off

Submit up to 50,000 requests as a single JSONL file and get results back within the window you choose — at up to 50% off the per-token price. The surface is a drop-in for the OpenAI Batch API (/v1/files + /v1/batches), so any OpenAI Batch SDK client works with a one-line base_url change.

Batch runs off a separate queue from your interactive traffic, so a 50k-line job never eats your real-time rate limits. You choose how long you're willing to wait — the longer the window, the deeper the discount we can honestly source.

Using the API

Four steps: upload an input file, create the batch, poll until it's done, download the results. Base URL https://api.llmrouter.sh/v1, Bearer key llmr_sk_live_….

1 · Upload your input file

One JSON object per line. custom_id is your own id used to match results back; body is a normal chat-completions request. You may mix different models in one file.

{"custom_id":"req-1","method":"POST","url":"/v1/chat/completions","body":{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Summarize: ..."}]}}
{"custom_id":"req-2","method":"POST","url":"/v1/chat/completions","body":{"model":"fireworks-deepseek-v4-pro","messages":[{"role":"user","content":"Classify: ..."}],"max_tokens":256}}

curl https://api.llmrouter.sh/v1/files \
  -H "Authorization: Bearer llmr_sk_live_..." \
  -F purpose=batch \
  -F file=@requests.jsonl
# → {"id":"file_abc...","object":"file","bytes":812,"purpose":"batch"}

2 · Create the batch

completion_window picks your tier: "24h" (default, deepest discount), "1h", or "10m". At submit we hold an estimate of the job's cost against your credit balance (settled to the actual as lines finish).

curl https://api.llmrouter.sh/v1/batches \
  -H "Authorization: Bearer llmr_sk_live_..." \
  -H "content-type: application/json" \
  -d '{
    "input_file_id": "file_abc...",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'
# → {"id":"batch_...","status":"in_progress","request_counts":{"total":2,"completed":0,"failed":0}}

3 · Poll for completion

Status moves validating → in_progress → completed. When it's terminal, output_file_id (and, if any line failed, error_file_id) are populated.

curl https://api.llmrouter.sh/v1/batches/batch_... \
  -H "Authorization: Bearer llmr_sk_live_..."
# cancel any time: POST /v1/batches/batch_.../cancel

4 · Download the results

One JSON object per line (order not guaranteed — match on custom_id). A successful line carries the full chat.completion under response.body.

curl https://api.llmrouter.sh/v1/files/file_out.../content \
  -H "Authorization: Bearer llmr_sk_live_..."

# each line:
{"custom_id":"req-1","response":{"status_code":200,"body":{ /* chat.completion */ }},"error":null}

Or with the OpenAI SDK

from openai import OpenAI
client = OpenAI(base_url="https://api.llmrouter.sh/v1", api_key="llmr_sk_live_...")

f = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")
batch = client.batches.create(input_file_id=f.id,
                              endpoint="/v1/chat/completions",
                              completion_window="24h")
# poll client.batches.retrieve(batch.id) until status == "completed"
results = client.files.content(batch.output_file_id).text

Just one request? You may not need a file at all

POST /v1/batches also accepts an inline requests array in place of input_file_id — no /v1/files upload, and small jobs get their result embedded right on the batch object too. But if you're already calling /v1/chat/completions for a single request that's OK to wait a few minutes, you don't need to learn the Batch API's shape at all — see Flex Mode for a normal blocking call at the same discount, or Async Jobs for the same job-id-and-poll-or-webhook mechanism as a single-request convenience over POST /v1/jobs.

Time commitments

The window you accept is the discount knob — a longer wait lets us route to genuinely cheaper capacity. We never advertise a discount we don't actually buy: the guaranteed ~50% is on the Day tier for models with a real provider batch endpoint. Waiting only 15 minutes for a single request? That's Flex Mode on /v1/chat/completions — same discount sources, no file. Need 1 or 6 hours instead? That's Async Jobs — same discount sources, delivered by poll or webhook, still no file.

Supported models

Every model in the catalog can be submitted as a batch — you always get the async convenience and the separate rate-limit pool. These families get a real ~50% token discount:

A batch may mix models freely; each line is priced on its own model and tier. Use GET /v1/models for the full live catalog.

Pricing

Batch is billed exactly like a normal request — the model's per-token price × the tier discount, plus the flat 1% platform fee. There is no separate batch surcharge and no Stripe fee baked in. For example, gpt-4o-mini at standard $0.15 / $0.60 per Mtok (in / out) runs on the Day tier at roughly $0.075 / $0.30 + 1%.

• Reservations. On submit we hold an estimate of the job's cost against your credit balance (invisible to your interactive traffic), then settle the actual cost — always ≤ the reserved amount — as each line finishes.
• Zero-completion insurance. Lines that fail or expire deliver nothing and are never billed — their reservation is released.
• Per-line pricing. Each line is charged on its own model and the discount its tier actually earned; the discount is itemized in your logs.

Limits & notes

• Up to 50,000 requests and 200 MB per input file.
• Each batch's estimated cost is capped at $100; larger jobs must be split across multiple batches.
• Your available credit balance is checked before a batch is created — if it can't cover the batch's estimated cost, submission fails with a 402 and nothing is enqueued or billed.
• completion_window ∈ 10m · 1h · 24h. A stock OpenAI client that only sends "24h" lands on the Day tier unchanged.
• Endpoint: /v1/chat/completions. Each line's url must match the batch endpoint; streaming is not used inside a batch.
• A malformed line or an unknown model fails the whole submit with a 400 naming the offending custom_id.