Implementation:Togethercomputer Together python Batches Create Batch

Attribute	Value
Type	Implementation
Domains	Batch_Processing, Inference, API_Client
Repository	togethercomputer/together-python
Source	src/together/resources/batch.py:L18-40
Last Updated	2026-02-15 16:00 GMT

Overview

Implementation of batch job creation via the Batches.create_batch() method. Submits an uploaded input file for asynchronous batch inference processing with a fixed 24-hour completion window.

Code Reference

class Batches:
    def __init__(self, client: TogetherClient) -> None:
        self._client = client

    def create_batch(self, file_id: str, endpoint: str) -> BatchJob:
        requestor = api_requestor.APIRequestor(
            client=self._client,
        )

        parameter_payload = {
            "input_file_id": file_id,
            "endpoint": endpoint,
            "completion_window": "24h",
        }

        response, _, _ = requestor.request(
            options=TogetherRequest(
                method="POST",
                url=f"batches",
                params=parameter_payload,
            ),
            stream=False,
        )

        assert isinstance(response, TogetherResponse)
        response_body = response.data.get("job", {})
        return BatchJob(**response_body)

Source: src/together/resources/batch.py:L18-40

API Signature

Batches.create_batch(file_id: str, endpoint: str) -> BatchJob

Parameter	Type	Description
`file_id`	`str`	The ID of the uploaded input file (from `Files.upload()` with `purpose="batch-api"`)
`endpoint`	`str`	The target API endpoint string (e.g., `/v1/chat/completions` or `/v1/completions`)

I/O Contract

Input:

file_id -- A valid file identifier returned by a prior Files.upload() call.
endpoint -- One of the supported batch endpoint strings.

Output: A BatchJob object with the following fields:

Field	Type	Description
`id`	string	Unique batch job identifier
`user_id`	string	ID of the user who created the job
`input_file_id`	string	ID of the input file
`file_size_bytes`	int	Size of the input file in bytes
`status`	BatchJobStatus	Current job status (initially `VALIDATING`)
`job_deadline`	datetime	Deadline for job completion (created_at + 24h)
`created_at`	datetime	Job creation timestamp
`endpoint`	string	The target endpoint
`progress`	float	Progress percentage (0.0 to 1.0), defaults to 0.0
`model_id`	string or None	Model identifier (if applicable)
`output_file_id`	string or None	Output file ID (populated upon completion)
`error_file_id`	string or None	Error file ID (populated if errors occurred)
`error`	string or None	Error message (if the job failed)
`completed_at`	datetime or None	Completion timestamp

BatchJobStatus enum values:

VALIDATING -- Server is validating the input file
IN_PROGRESS -- Requests are being processed
COMPLETED -- All requests finished; output file available
FAILED -- Job encountered an unrecoverable error
EXPIRED -- Job did not complete within the 24-hour window
CANCELLED -- Job was cancelled by the user
CANCELING -- Cancellation is in progress

Key Behavior

Fixed completion window: The method internally hardcodes "completion_window": "24h" in the request payload. This is not configurable by the caller:

parameter_payload = {
    "input_file_id": file_id,
    "endpoint": endpoint,
    "completion_window": "24h",
}

Response extraction: The batch job data is extracted from the "job" key in the response body via response.data.get("job", {}), which is then unpacked into a BatchJob model.

Usage Examples

Creating a Batch Job

from together import Together

client = Together()

# Assumes file_id was obtained from a prior Files.upload() call
batch_job = client.batches.create_batch(
    file_id="file-abc123",
    endpoint="/v1/chat/completions",
)

print(f"Batch job ID: {batch_job.id}")
print(f"Status: {batch_job.status}")
print(f"Deadline: {batch_job.job_deadline}")

End-to-End: Upload and Create Batch

from together import Together

client = Together()

# Step 1: Upload the input file
file_response = client.files.upload(
    file="batch_input.jsonl",
    purpose="batch-api",
)

# Step 2: Create the batch job
batch_job = client.batches.create_batch(
    file_id=file_response.id,
    endpoint="/v1/chat/completions",
)

print(f"Batch job created: {batch_job.id}")

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment