Overview
Concrete tool for the Batch API resource provided by the openai-python SDK.
Description
The Batches Resource module provides Batches (synchronous) and AsyncBatches (asynchronous) resource classes for managing batch processing jobs via the OpenAI Batch API. The create() method submits a new batch from an uploaded JSONL file of requests, specifying the target endpoint (e.g., /v1/chat/completions, /v1/responses, /v1/embeddings) and a completion_window (currently only "24h"). The retrieve() method fetches the status and details of a specific batch by its ID. The list() method returns a paginated cursor-based list of all batches in the organization. The cancel() method cancels an in-progress batch, which transitions to cancelling status for up to 10 minutes before becoming cancelled with any partial results available. Both classes extend SyncAPIResource / AsyncAPIResource and provide with_raw_response and with_streaming_response property accessors for alternative response handling.
Usage
Use the Batches Resource when you need to submit large volumes of API requests for asynchronous processing at reduced cost. This is ideal for bulk operations such as processing thousands of chat completions, embeddings, or moderations where immediate responses are not required.
Code Reference
Source Location
Signature
Batches.create()
def create(
self,
*,
completion_window: Literal["24h"],
endpoint: Literal[
"/v1/responses",
"/v1/chat/completions",
"/v1/embeddings",
"/v1/completions",
"/v1/moderations",
"/v1/images/generations",
"/v1/images/edits",
],
input_file_id: str,
metadata: Optional[Metadata] | Omit = omit,
output_expires_after: batch_create_params.OutputExpiresAfter | Omit = omit,
extra_headers: Headers | None = None,
extra_query: Query | None = None,
extra_body: Body | None = None,
timeout: float | httpx.Timeout | None | NotGiven = not_given,
) -> Batch: ...
Batches.retrieve()
def retrieve(
self,
batch_id: str,
*,
extra_headers: Headers | None = None,
extra_query: Query | None = None,
extra_body: Body | None = None,
timeout: float | httpx.Timeout | None | NotGiven = not_given,
) -> Batch: ...
Batches.list()
def list(
self,
*,
after: str | Omit = omit,
limit: int | Omit = omit,
extra_headers: Headers | None = None,
extra_query: Query | None = None,
extra_body: Body | None = None,
timeout: float | httpx.Timeout | None | NotGiven = not_given,
) -> SyncCursorPage[Batch]: ...
Batches.cancel()
def cancel(
self,
batch_id: str,
*,
extra_headers: Headers | None = None,
extra_query: Query | None = None,
extra_body: Body | None = None,
timeout: float | httpx.Timeout | None | NotGiven = not_given,
) -> Batch: ...
Import
from openai.resources.batches import Batches, AsyncBatches
I/O Contract
Inputs (create)
| Name |
Type |
Required |
Description
|
| completion_window |
Literal["24h"] |
Yes |
The time frame within which the batch should be processed. Currently only "24h" is supported.
|
| endpoint |
Literal["/v1/responses", "/v1/chat/completions", ...] |
Yes |
The API endpoint to use for all requests in the batch.
|
| input_file_id |
str |
Yes |
The ID of an uploaded JSONL file containing the batch requests (uploaded with purpose "batch").
|
| metadata |
Optional[Metadata] |
No |
Up to 16 key-value pairs for storing additional information about the batch.
|
| output_expires_after |
OutputExpiresAfter |
No |
The expiration policy for the generated output and error files.
|
Inputs (retrieve / cancel)
| Name |
Type |
Required |
Description
|
| batch_id |
str |
Yes |
The unique identifier of the batch to retrieve or cancel.
|
Inputs (list)
| Name |
Type |
Required |
Description
|
| after |
str |
No |
A cursor (object ID) for pagination. Fetch results after this ID.
|
| limit |
int |
No |
Maximum number of objects to return (1-100, default 20).
|
Outputs
| Name |
Type |
Description
|
| return (create) |
Batch |
The created batch object with its ID, status, and metadata.
|
| return (retrieve) |
Batch |
The batch object with current status, progress, and file IDs.
|
| return (list) |
SyncCursorPage[Batch] / AsyncPaginator[Batch, AsyncCursorPage[Batch]] |
A paginated list of batch objects.
|
| return (cancel) |
Batch |
The batch object with updated cancellation status.
|
Usage Examples
Creating a Batch
from openai import OpenAI
client = OpenAI()
batch = client.batches.create(
input_file_id="file-abc123",
endpoint="/v1/chat/completions",
completion_window="24h",
metadata={"description": "nightly eval run"},
)
print(batch.id) # "batch_abc123"
Retrieving Batch Status
batch = client.batches.retrieve("batch_abc123")
print(batch.status) # "completed"
print(batch.output_file_id) # "file-xyz789"
Listing All Batches
page = client.batches.list(limit=10)
for batch in page.data:
print(f"{batch.id}: {batch.status}")
Cancelling a Batch
cancelled = client.batches.cancel("batch_abc123")
print(cancelled.status) # "cancelling"
Async Usage
from openai import AsyncOpenAI
client = AsyncOpenAI()
batch = await client.batches.create(
input_file_id="file-abc123",
endpoint="/v1/chat/completions",
completion_window="24h",
)
Related Pages
