Principle:Ollama Ollama GGUF Model Conversion Gemma3

Knowledge Sources	Ollama
Domains	Model Conversion, Gemma
Last Updated	2025-02-15 00:00 GMT

Overview

Gemma 3 model conversion extends the Gemma 1 converter with support for sliding window attention patterns, per-layer normalization variants, vision encoder integration, multi-modal projector handling, and YaRN RoPE scaling -- transforming both text-only and vision-language Gemma 3 variants into GGUF format.

Core Concepts

Tensor Name Mapping

The converter applies the following HuggingFace-to-GGUF tensor name replacements:

• lm_head -> output
• model.embed_tokens -> token_embd
• model.norm -> output_norm
• vision_tower.vision_model.embeddings / vision_model.vision_model.embeddings -> v
• vision_tower.vision_model / vision_model.vision_model -> v
• language_model. -> (stripped)
• encoder.layers -> blk
• self_attn.q_norm -> attn_q_norm
• self_attn.k_norm -> attn_k_norm
• post_attention_layernorm -> post_attention_norm
• pre_feedforward_layernorm -> ffn_norm
• post_feedforward_layernorm -> post_ffw_norm
• multi_modal_projector -> mm

Architecture-Specific Hyperparameters

The GGUF metadata is written under the gemma3.* namespace:

• gemma3.block_count -- number of text model layers
• gemma3.attention.head_count / head_count_kv -- inferred from model size (4B/12B/27B have preset values)
• gemma3.attention.sliding_window -- sliding window size
• gemma3.attention.sliding_window_pattern -- per-layer boolean array indicating local vs global attention
• gemma3.final_logit_softcapping -- logit soft-capping factor (if > 0)
• gemma3.rope.local.freq_base -- local attention RoPE theta (default 10000)
• gemma3.rope.freq_base -- global attention RoPE theta (default 1000000)
• gemma3.rope.scaling.* -- YaRN scaling parameters (factor, original_context_length, extrapolation_factor, beta_fast, beta_slow)
• gemma3.mm.tokens_per_image -- multi-modal tokens per image

Vision (multimodal variant):

• gemma3.vision.block_count, embedding_length, feed_forward_length
• gemma3.vision.image_size, patch_size, num_channels
• gemma3.vision.attention.head_count, layer_norm_epsilon

Special Handling

Sliding Window Pattern

Gemma 3 uses a hybrid attention pattern where some layers use sliding window (local) attention and others use full (global) attention. The pattern is determined from either the sliding_window_pattern integer (where every Nth layer is global) or the layer_types string array. The resulting boolean array is stored in GGUF metadata.

Size-Based Head Count Inference

For known model sizes (34 layers = 4B, 48 layers = 12B, 62 layers = 27B), the converter uses preset head counts rather than relying on config values, ensuring correct GQA ratios.

Causal vs Multimodal Variants

The converter supports both Gemma3ForCausalLM (text-only) and multimodal architectures. The text-only variant reads parameters from top-level config fields, while the multimodal variant reads from nested text_config and vision_config structures.

Inherited Norm Offset

The addOne normalization weight adjustment from the base Gemma converter is inherited, adding 1.0 to all RMSNorm weights.

Implementation Notes

The conversion is implemented in convert/convert_gemma3.go via the gemma3Model struct which embeds gemmaModel. This struct dispatches KV generation based on the Architecture field to handle both causal and multimodal model configurations. The sliding window pattern logic uses Go iterators to construct the per-layer boolean array.

Related Pages

• Implementation:Ollama_Ollama_Convert_Gemma3