Async Jobs

Submit once, get it back by poll or webhook

A single chat request that's OK to wait 1 or 6 hours for a deeper discount. POST /v1/jobs returns a job id immediately (202) — no blocking connection, no file upload. Get the result back however suits you: poll GET /v1/jobs/{id}, register a webhook_url to get pushed a signed notification, or both — a missed webhook delivery never strands a job, since polling always works too.

Need an answer right now? See Flex Mode — a normal blocking call, up to 15 minutes, same discount sources. Submitting more than a handful of requests at once? See the file-based Batch API — up to 50,000 requests, up to 24 hours.

How it works

Neither the 1-hour nor the 6-hour window has a synchronous provider equivalent to be wire-compatible with — no provider or router exposes an hour-scale service_tier, so blocking an HTTP connection for that long isn't a reasonable request/response shape for anyone. A Job is a background request instead: your submission becomes a one-line job on the same queue and driver that power the Batch API, pooling with other requests for up to your chosen window before it runs.

When the resolved model is OpenAI or Gemini (AI-Studio or Vertex) and currently believed to support flex processing, we forward a real service_tier=flex upstream and bill you the discounted price those providers actually charge us — the same real discount Flex Mode gets, just with more headroom before we serve at standard price. Any other model still serves the request at the standard price rather than failing.

Using the API

Base URL https://api.llmrouter.sh/v1, Bearer key llmr_sk_live_….

Submit — POST /v1/jobs

A normal chat-completion-shaped body, plus completion_window ("1h" or "6h" — required, no default) and an optional webhook_url (must be https://). Returns 202 immediately — the request has been accepted, not completed.

curl https://api.llmrouter.sh/v1/jobs \
  -H "Authorization: Bearer llmr_sk_live_..." \
  -H "content-type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Summarize this thread..."}],
    "completion_window": "6h",
    "webhook_url": "https://example.com/hooks/llmrouter-jobs"
  }'
# → 202 {"id":"job_...","object":"job","status":"queued","completion_window":"6h",
#        "webhook_url":"https://example.com/hooks/llmrouter-jobs","estimated_completion":1731451600}

Poll — GET /v1/jobs/{id}

status moves queued → in_progress → completed (or failed/expired). Once terminal, result holds a normal chat.completion object — the same thing a synchronous call would have returned, no batch-shaped wrapper to unwrap.

curl https://api.llmrouter.sh/v1/jobs/job_... \
  -H "Authorization: Bearer llmr_sk_live_..."

# once status == "completed":
{"id":"job_...","status":"completed","completion_window":"6h",
 "result":{"id":"chatcmpl-...","choices":[{"message":{"role":"assistant","content":"..."}}], ...}}

Webhook — pushed instead of polled

If you passed webhook_url at submit, the identical job object GET /v1/jobs/{id} would return is POSTed to that URL once, on the terminal transition (completed/failed/expired/cancelled) — never on queued/in_progress. The payload carries an X-LLMRouter-Signature header — an HMAC-SHA256 over the raw body with a secret unique to your org — so you can verify the request actually came from us before trusting it. Delivery is retried with backoff on a non-2xx response or timeout, up to a bounded number of attempts; if it never succeeds, the job's own state is completely unaffected — GET /v1/jobs/{id} always has the answer, whether or not the webhook ever landed.

import hashlib, hmac

def verify(raw_body: bytes, signature: str, secret: str) -> bool:
    expected = hmac.new(secret.encode(), raw_body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, signature)

# in your webhook handler:
# verify(request.body, request.headers["X-LLMRouter-Signature"], YOUR_WEBHOOK_SECRET)

Find YOUR_WEBHOOK_SECRET — rotatable if it ever leaks — and save a default webhook_url so you don't have to pass one on every submission, under Async Jobs in your dashboard, which also lists recent jobs and their delivery status.

Choosing a window

A longer window means a bigger pool of requests waiting together, which reaches deeper savings on models without a native flex tier. Both windows carry the same real OpenAI/Gemini discount on models that support it, live from the start — the window mainly buys headroom before we fall back to standard price.

Need an answer in seconds or a blocking call up to 15 minutes? See Flex Mode. Submitting a bulk file of up to 50,000 requests, or need the full 24h window? See the Batch API — POST /v1/jobs is a single-request convenience front door onto the same underlying 1h/6h rungs; submitting a bulk batch at those windows via POST /v1/batches works identically.

Which models get the discount

Async Jobs isn't one discount — it stacks a few independent levers, and a job takes whichever ones its model actually qualifies for. Nothing here is ever faked: a model that doesn't qualify for a lever just skips it and completes at the next one down, all the way to standard price. You never have to hardcode a model list to use Async Jobs safely.

• Provider-native flex — OpenAI and Gemini. The same real service_tier=flex ~50% discount this platform forwards for Flex Mode — see that page for the current model list. Live on both 1h and 6h windows, billed at the discounted price the provider actually charges us.
• Best-price-within-window — every model. A job doesn't route the instant it's submitted; it rests as a limit order for its full window. If a cheaper direct deployment of the model opens up before the deadline, the job waits for it and bills off that cheaper price instead of falling back to a pricier catch-all route. A 6h window catches materially more of these recoveries than 1h.
• Native batch-API capture — 6h only. A model with no flex tier but its own real batch API — and a wholesale batch discount on it (OpenAI Batch, Azure, Vertex, Anthropic, Fireworks today) — gets speculatively pooled into a real provider batch submission to capture that discount (typically ~50% off). If the pooled job doesn't clear before the 6-hour deadline, it falls back to plain full price rather than failing or billing a discount that wasn't actually earned. 1h jobs don't attempt this — the window's too tight for a worthwhile clear rate.

A model on none of these lists, or one whose flex/batch attempt doesn't come through in time, still completes at the standard price rather than failing.

Errors

• 400 unsupported_completion_window — completion_window must be "1h" or "6h" — there's no default.
• 400 invalid_webhook_url — webhook_url was set but wasn't an https:// URL.
• 400 job_streaming_unsupported — stream: true is not supported; a job can't stream a response that doesn't exist yet.
• 402 insufficient_credits — checked before anything is enqueued; nothing is billed on a failed submit.
• 400 batch_cost_limit_exceeded — the same $100 per-line reservation cap every batch line carries.
• 404 job_not_found — polling an id that doesn't exist, or isn't yours.

Pricing

Billed exactly like every other request — the model's per-token price × whatever discount actually applied, plus the flat 1% platform fee. We never advertise a discount we haven't bought: whichever lever from above the line actually qualifies for — provider flex, a cheaper deployment caught within the window, or a cleared native batch submission — is billed at the real discounted price behind it. A model that qualifies for none of them, or whose attempt doesn't come through in time, runs at the standard price — the job still completes rather than failing, just without a price cut. Webhook delivery has no separate charge.