Implementation:Ollama Ollama Imagegen Transfer Download
| Knowledge Sources | |
|---|---|
| Domains | Image Generation, Transfer |
| Last Updated | 2025-02-15 00:00 GMT |
Overview
Implements parallel blob downloading with streaming SHA256 verification, stall detection, and adaptive retry logic for tensor model transfer.
Description
The download.go file provides the downloader struct and download logic for fetching model blobs from an OCI registry. Downloads are whole-blob (no chunking) with inline SHA256 hash verification during streaming. The system detects download stalls (no data for stallTimeout) and slow transfers (below 10% of median speed), with separate retry logic for each: stall retries do not count against the limit, slow retries require 3 occurrences before counting. Already-downloaded blobs are detected by matching file size and skipped. The progress tracker reports completed bytes including pre-existing blobs for accurate resume display. Concurrency is controlled via semaphore (default 64 parallel downloads).
Usage
Used when pulling tensor-based image generation models that consist of many small blobs rather than few large ones.
Code Reference
Source Location
- Repository: Ollama
- File: x/imagegen/transfer/download.go
- Lines: 1-329
Signature
type downloader struct {
client *http.Client
baseURL string
destDir string
repository string
token *string
getToken func(context.Context, AuthChallenge) (string, error)
userAgent string
stallTimeout time.Duration
progress *progressTracker
speeds *speedTracker
}
func download(ctx context.Context, opts DownloadOptions) error
func (d *downloader) download(ctx context.Context, blob Blob) error
func (d *downloader) downloadOnce(ctx context.Context, blob Blob) (int64, error)
Import
import "github.com/ollama/ollama/x/imagegen/transfer"
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| ctx | context.Context | Yes | Context for cancellation |
| opts | DownloadOptions | Yes | Download configuration with blobs, URLs, auth |
Outputs
| Name | Type | Description |
|---|---|---|
| error | error | Error if any blob fails after max retries |
Usage Examples
err := transfer.Download(ctx, transfer.DownloadOptions{
Blobs: blobs,
BaseURL: "https://registry.ollama.ai",
DestDir: "/home/user/.ollama/blobs",
Repository: "library/flux2-klein",
Concurrency: 64,
Progress: func(completed, total int64) {
fmt.Printf("%.1f%%\n", float64(completed)/float64(total)*100)
},
})