Implementation:Ollama Ollama Imagegen Server

Knowledge Sources	Ollama
Domains	Image Generation, Server
Last Updated	2025-02-15 00:00 GMT

Overview

Implements the Server struct that wraps an MLX runner subprocess, conforming to Ollama's llm.LlamaServer interface for scheduler integration.

Description

The server.go file defines the Server type that spawns and manages an MLX runner subprocess for both LLM and image generation models. NewServer validates platform support, allocates a free TCP port, spawns the subprocess (ollama runner --imagegen-engine --model --port), configures LD_LIBRARY_PATH for MLX shared libraries on Linux, and waits for the /health endpoint to respond. The Server implements llm.LlamaServer methods for Ollama's scheduler: Ping, Completion (forwarding HTTP requests to subprocess), Tokenize, Detokenize, Close, and EstimatedVRAM. VRAM is estimated from the model's manifest tensor sizes with an 8GB fallback. Subprocess stdout/stderr are forwarded to slog for unified logging.

Usage

Used by Ollama's scheduler to load, manage, and communicate with MLX model runners as subprocess servers.

Code Reference

Source Location

• Repository: Ollama
• File: x/imagegen/server.go
• Lines: 1-472

Signature

type Server struct {
	mu          sync.Mutex
	cmd         *exec.Cmd
	port        int
	modelName   string
	mode        ModelMode
	vramSize    uint64
	done        chan error
	client      *http.Client
	lastErr     string
	lastErrLock sync.Mutex
}

func NewServer(modelName string, mode ModelMode) (*Server, error)
func (s *Server) Ping(ctx context.Context) error
func (s *Server) Completion(ctx context.Context, req llm.CompletionRequest, fn func(llm.CompletionResponse)) error
func (s *Server) Close() error
func (s *Server) EstimatedVRAM() uint64

Import

import "github.com/ollama/ollama/x/imagegen"

I/O Contract

Inputs

Name	Type	Required	Description
modelName	string	Yes	Model name for manifest resolution
mode	ModelMode	Yes	ModeLLM or ModeImageGen

Outputs

Name	Type	Description
*Server	*Server	Running server wrapping MLX subprocess
error	error	Error if platform unsupported or subprocess fails to start

Usage Examples

server, err := imagegen.NewServer("flux2-klein:latest", imagegen.ModeImageGen)
if err != nil {
    return err
}
defer server.Close()

// Server conforms to llm.LlamaServer for scheduler integration
err = server.Completion(ctx, req, func(resp llm.CompletionResponse) {
    // Handle streaming response
})

Related Pages

• Principle:Ollama_Ollama_ImageGeneration