Batch API
The Batch API is an asynchronous inference endpoint that processes large volumes of requests at 50% of the standard API price. You submit a batch of requests, Anthropic processes them within 24 hours, and you retrieve the results. No streaming, no low-latency guarantees — just throughput at reduced cost.
When to Use It
The Batch API is purpose-built for workloads that tolerate delay:
- Evals — running a test suite against a model before a deploy
- Bulk annotation — labeling thousands of examples for a training dataset
- Offline summarization — processing a document corpus overnight
- Embedding generation — computing embeddings for a large knowledge base before indexing
- Report generation — nightly batch jobs that synthesize data
The cost reduction is significant at scale. A task that costs $1,000 at standard rates costs $500 via the Batch API.
API Mechanics
Batches are submitted as a JSONL file or array of request objects. Each request is a standard Messages API call with an additional custom_id for result correlation.
import anthropic
client = anthropic.Anthropic()
# Create a batch
batch = client.messages.batches.create(
requests=[
{
"custom_id": "doc-001",
"params": {
"model": "claude-opus-4-5",
"max_tokens": 1024,
"messages": [{"role": "user", "content": "Summarize: ..."}]
}
},
# ... up to 10,000 requests per batch
]
)
print(batch.id) # msgbatch_01...
print(batch.processing_status) # "in_progress"
Polling status:
import time
while True:
batch = client.messages.batches.retrieve(batch.id)
if batch.processing_status == "ended":
break
time.sleep(60)
Retrieving results:
for result in client.messages.batches.results(batch.id):
if result.result.type == "succeeded":
print(result.custom_id, result.result.message.content)
else:
print(f"Failed: {result.custom_id} — {result.result.error}")
Results are returned in the same JSONL format, matched to requests by custom_id. Failed requests appear in the results with error details rather than silently disappearing.
Limits
- Up to 10,000 requests per batch
- 24-hour processing window (usually faster, not guaranteed)
- Same model access as the standard API
- No streaming support
- Results available for 29 days after batch completion
When Not to Use It
Do not use the Batch API for:
- Any user-facing interaction where latency matters
- Interactive agents or chat
- Workflows where step N depends on the result of step N-1 — batch processing is not sequential
- Anything with a deadline shorter than a few hours
If you need cost reduction on interactive traffic, prompt-caching is the right lever.