Implementation:Ollama Ollama Convert NomicBert

Knowledge Sources	Ollama
Domains	Model Conversion, GGUF Format
Last Updated	2025-02-15 00:00 GMT

Overview

Implements the GGUF model converter for the Nomic-BERT embedding model architecture, supporting both standard and MoE variants with RoPE, combined QKV projections, and sentence-transformer pooling configuration.

Description

The nomicbertModel struct implements both ModelConverter and moreParser interfaces. It dynamically selects the architecture name (nomic-bert or nomic-bert-moe) based on whether MoE parameters are present. The parseMore method reads modules.json for pooling and normalization configuration, identical to the BERT converter. KV emits metadata including RoPE frequency base, optional MoE parameters (expert count, experts used per token, MoE every N layers), and phantom-space tokenizer conversion. The Tensors method filters out embeddings.position_ids and pooler.dense tensors. Replacements handle combined attention.self.qkv projections and MoE expert routing/projections.

Usage

Invoked automatically when the model's architecture matches Nomic-BERT variants.

Code Reference

Source Location

• Repository: Ollama
• File: convert/convert_nomicbert.go
• Lines: 1-213

Signature

type nomicbertModel struct {
    ModelParameters
    NumHiddenLayers uint32  `json:"num_hidden_layers"`
    HiddenSize      uint32  `json:"hidden_size"`
    RopeFreqBase    float32 `json:"rope_theta"`
    NumExperts      uint32  `json:"num_local_experts"`
    NumExpertsUsed  uint32  `json:"num_experts_per_tok"`
    MoEEveryNLayers uint32  `json:"moe_every_n_layers"`
    PoolingType     uint32
}

func (p *nomicbertModel) parseMore(fsys fs.FS) error
func (p *nomicbertModel) KV(t *Tokenizer) KV
func (p *nomicbertModel) Tensors(ts []Tensor) []*ggml.Tensor
func (nomicbertModel) Replacements() []string

Import

import "github.com/ollama/ollama/convert"

I/O Contract

Inputs

Name	Type	Required	Description
t	*Tokenizer	Yes	Tokenizer with tokens to convert to phantom-space format
ts	[]Tensor	Yes	Source tensors to convert and filter
fsys	fs.FS	Yes	Filesystem for reading modules.json

Outputs

Name	Type	Description
KV	KV	GGUF metadata with nomic-bert (or nomic-bert-moe) keys
[]*ggml.Tensor	slice	Filtered tensors excluding position_ids and pooler.dense

Usage Examples

// Converter registered for Nomic-BERT architectures
// Architecture is "nomic-bert" for dense, "nomic-bert-moe" for MoE variant
// QKV projections are combined: attention.self.qkv -> attn_qkv

Related Pages

• Principle:Ollama_Ollama_GGUF_Model_Conversion_NomicBert