Implementation:Togethercomputer Together python Files Upload

Attribute	Value
Implementation Name	Files_Upload
Type	API Method
Source	src/together/resources/files.py:L28-59
Domain	MLOps, Fine_Tuning, Data_Preparation
Repository	togethercomputer/together-python
Last Updated	2026-02-15 16:00 GMT

API Signature

class Files:
    def upload(
        self,
        file: Path | str,
        *,
        purpose: FilePurpose | str = FilePurpose.FineTune,
        check: bool = True,
    ) -> FileResponse:

Import

from together import Together

client = Together()
response = client.files.upload(...)

I/O Contract

Inputs

Parameter	Type	Default	Description
`file`	str	(required)	Path to the local file to upload.
`purpose`	str	`FilePurpose.FineTune`	The intended purpose of the file. Determines server-side processing. Values: `"fine-tune"`, `"eval"`.
`check`	`bool`	`True`	Whether to run `check_file()` validation before uploading. Only applied when `purpose` is `FilePurpose.FineTune`.

Output

Returns a FileResponse object with the following fields:

Field	Type	Description
`id`	`str`	Unique file identifier (e.g., `"file-abc123"`). Used as `training_file` in fine-tuning job creation.
`object`	`str`	Object type identifier.
`created_at`	`int`	Unix timestamp of creation.
`type`	`str`	File MIME type.
`purpose`	`str`	The file's purpose tag.
`filename`	`str`	Original filename.
`bytes`	`int`	File size in bytes.
`line_count`	`int`	Number of lines/samples in the file.

Errors

FileTypeError -- Raised when check=True and the file fails validation.

Code Reference

From src/together/resources/files.py:L28-59:

def upload(
    self,
    file: Path | str,
    *,
    purpose: FilePurpose | str = FilePurpose.FineTune,
    check: bool = True,
) -> FileResponse:

    if check and purpose == FilePurpose.FineTune:
        report_dict = check_file(file)
        if not report_dict["is_check_passed"]:
            raise FileTypeError(
                f"Invalid file supplied, failed to upload. Report:\n{pformat(report_dict)}"
            )

    if isinstance(file, str):
        file = Path(file)

    if isinstance(purpose, str):
        purpose = FilePurpose(purpose)

    assert isinstance(purpose, FilePurpose)

    file_size = os.stat(file).st_size
    file_size_gb = file_size / NUM_BYTES_IN_GB

    if file_size_gb > MULTIPART_THRESHOLD_GB:
        multipart_manager = MultipartUploadManager(self._client)
        return multipart_manager.upload("files", file, purpose)
    else:
        upload_manager = UploadManager(self._client)
        return upload_manager.upload("files", file, purpose=purpose, redirect=True)

Key implementation details:

The MULTIPART_THRESHOLD_GB is set to 5.0 in src/together/constants.py.
Files exceeding 5.0 GB are uploaded via MultipartUploadManager which splits the file into parts (target 250 MB, minimum 5 MB) and uploads up to 4 parts concurrently.
Smaller files use UploadManager with a single request and redirect handling.
Validation via check_file() is only performed when both check=True and purpose == FilePurpose.FineTune.

Usage Examples

Basic Upload

from together import Together

client = Together()

# Upload with automatic validation (default)
response = client.files.upload("training_data.jsonl")
print(f"File ID: {response.id}")
print(f"Size: {response.bytes} bytes")
print(f"Samples: {response.line_count}")

Upload Without Validation

from together import Together

client = Together()

# Skip validation for faster upload (when you have already validated)
response = client.files.upload("training_data.jsonl", check=False)
print(f"File ID: {response.id}")

Upload a Parquet File

from together import Together

client = Together()

# Upload a pre-tokenized Parquet file
response = client.files.upload("processed_dataset.parquet")
print(f"File ID: {response.id}")

Upload for Evaluation

from together import Together
from together.types import FilePurpose

client = Together()

# Upload a CSV file for evaluation (not fine-tuning)
response = client.files.upload("eval_data.csv", purpose=FilePurpose.Eval, check=False)
print(f"File ID: {response.id}")

End-to-End: Upload then Fine-Tune

from together import Together

client = Together()

# Step 1: Upload the training file
file_response = client.files.upload("training_data.jsonl")
file_id = file_response.id

# Step 2: Create a fine-tuning job using the uploaded file ID
job = client.fine_tuning.create(
    training_file=file_id,
    model="meta-llama/Meta-Llama-3.1-8B-Instruct",
    n_epochs=3,
    lora=True,
)
print(f"Fine-tuning job: {job.id}")

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment