Implementation:Ollama Ollama Imagegen Transfer Upload

Knowledge Sources	Ollama
Domains	Image Generation, Transfer
Last Updated	2025-02-15 00:00 GMT

Overview

Implements parallel blob uploading with two-phase existence checking and cross-repository blob mounting for tensor model publishing.

Description

The upload.go file provides the uploader struct and upload logic for pushing model blobs to an OCI registry. It uses a two-phase approach: Phase 1 performs fast parallel HEAD checks (128 concurrent) to determine which blobs already exist on the registry, then Phase 2 uploads only missing blobs with controlled concurrency (default 32). Uploads use the standard OCI blob upload flow (POST to initiate, PUT with data). Cross-repository blob mounting is supported via the Blob.From field, which adds ?mount=<digest>&from=<repo> to avoid re-uploading blobs that exist in another repository. After all blobs complete, the manifest can be optionally pushed via pushManifest.

Usage

Used when pushing tensor-based image generation models to a registry, optimizing for the many-small-blob pattern.

Code Reference

Source Location

• Repository: Ollama
• File: x/imagegen/transfer/upload.go
• Lines: 1-346

Signature

type uploader struct {
	client     *http.Client
	baseURL    string
	srcDir     string
	repository string
	token      *string
	getToken   func(context.Context, AuthChallenge) (string, error)
	userAgent  string
	progress   *progressTracker
	logger     *slog.Logger
}

func upload(ctx context.Context, opts UploadOptions) error
func (u *uploader) upload(ctx context.Context, blob Blob) error
func (u *uploader) uploadOnce(ctx context.Context, blob Blob) (int64, error)
func (u *uploader) exists(ctx context.Context, blob Blob) (bool, error)
func (u *uploader) pushManifest(ctx context.Context, repo, ref string, manifest []byte) error

Import

import "github.com/ollama/ollama/x/imagegen/transfer"

I/O Contract

Inputs

Name	Type	Required	Description
ctx	context.Context	Yes	Context for cancellation
opts	UploadOptions	Yes	Upload configuration with blobs, URLs, auth, optional manifest

Outputs

Name	Type	Description
error	error	Error if any blob upload fails after max retries

Usage Examples

err := transfer.Upload(ctx, transfer.UploadOptions{
    Blobs:       blobs,
    BaseURL:     "https://registry.ollama.ai",
    SrcDir:      "/home/user/.ollama/blobs",
    Repository:  "myorg/my-model",
    Concurrency: 32,
    Manifest:    manifestJSON,
    ManifestRef: "latest",
})

Related Pages

• Principle:Ollama_Ollama_ImageGeneration