Implementation:Ggml org Llama cpp Common Embd Normalize

Field	Value
Implementation Name	Common Embd Normalize
Doc Type	API Doc
Domain	Vector Mathematics, Embedding Post-Processing
Description	common_embd_normalize(inp, out, n, embd_norm) for vector normalization and common_embd_similarity_cos(embd1, embd2, n) for cosine similarity computation
Related Workflow	Embedding_Extraction

Overview

Description

The Common Embd Normalize implementation provides two utility functions for embedding post-processing: common_embd_normalize() normalizes an embedding vector using a configurable p-norm, and common_embd_similarity_cos() computes the cosine similarity between two embedding vectors. These functions are used as the final step in the embedding extraction pipeline, transforming raw model outputs into normalized vectors suitable for similarity comparison and storage.

Usage

#include "common.h"

float raw_embedding[768];   // from llama_get_embeddings_ith/seq
float normalized[768];

// L2 normalize (most common for cosine similarity)
common_embd_normalize(raw_embedding, normalized, 768, 2);

// Compute cosine similarity between two embeddings
float similarity = common_embd_similarity_cos(embd_a, embd_b, 768);

Code Reference

Field	Value
Source Location (header)	common/common.h:830-832
Source Location (implementation)	common/common.cpp:1541-1596
Signature (normalize)	void common_embd_normalize(const float * inp, float * out, int n, int embd_norm)
Signature (similarity)	float common_embd_similarity_cos(const float * embd1, const float * embd2, int n)
Import	#include "common.h"

common_embd_normalize implementation:

void common_embd_normalize(const float * inp, float * out, int n, int embd_norm) {
    double sum = 0.0;

    switch (embd_norm) {
        case -1: // no normalisation
            sum = 1.0;
            break;
        case 0: // max absolute
            for (int i = 0; i < n; i++) {
                if (sum < std::abs(inp[i])) {
                    sum = std::abs(inp[i]);
                }
            }
            sum /= 32760.0; // make an int16 range
            break;
        case 2: // euclidean
            for (int i = 0; i < n; i++) {
                sum += inp[i] * inp[i];
            }
            sum = std::sqrt(sum);
            break;
        default: // p-norm (euclidean is p-norm p=2)
            for (int i = 0; i < n; i++) {
                sum += std::pow(std::abs(inp[i]), embd_norm);
            }
            sum = std::pow(sum, 1.0 / embd_norm);
            break;
    }

    const float norm = sum > 0.0 ? 1.0 / sum : 0.0f;

    for (int i = 0; i < n; i++) {
        out[i] = inp[i] * norm;
    }
}

common_embd_similarity_cos implementation:

float common_embd_similarity_cos(const float * embd1, const float * embd2, int n){
    double sum  = 0.0;
    double sum1 = 0.0;
    double sum2 = 0.0;

    for (int i = 0; i < n; i++) {
        sum  += embd1[i] * embd2[i];
        sum1 += embd1[i] * embd1[i];
        sum2 += embd2[i] * embd2[i];
    }

    // Handle the case where one or both vectors are zero vectors
    if (sum1 == 0.0 || sum2 == 0.0) {
        if (sum1 == 0.0 && sum2 == 0.0) {
            return 1.0f; // two zero vectors are similar
        }
        return 0.0f;
    }

    return sum / (sqrt(sum1) * sqrt(sum2));
}

I/O Contract

common_embd_normalize:

Direction	Description
Input	inp: source embedding vector (const float *); n: vector dimension; embd_norm: normalization mode
Output	out: normalized embedding vector (float *), may alias inp for in-place normalization
Preconditions	out must have space for at least n floats; inp must be at least n elements
Edge Cases	If norm computes to 0.0 (zero vector), all output elements are set to 0.0

Normalization modes:

embd_norm Value	Mode	Formula	Use Case
-1	No normalization	out[i] = inp[i]	Raw embeddings, custom normalization downstream
0	Max-absolute	inp|) / 32760)	Integer quantization (int16 range)
2	Euclidean (L2)	out[i] = inp[i] / sqrt(sum(inp[i]^2))	Cosine similarity, unit hypersphere
p (other)	P-norm	inp[i]|^p))^(1/p)	Generalized normalization

common_embd_similarity_cos:

Direction	Description
Input	Two embedding vectors embd1, embd2 (each const float * of length n)
Output	float cosine similarity in range [-1.0, 1.0]
Edge Cases	Returns 1.0 if both vectors are zero; returns 0.0 if exactly one vector is zero
Precision	Uses double accumulators for dot products and norms to minimize floating-point error

Usage Examples

L2 normalization and cosine similarity matrix (from embedding example):

// Normalize embeddings during batch_decode
for (int i = 0; i < batch.n_tokens; i++) {
    if (!batch.logits[i]) continue;

    const float * embd = llama_get_embeddings_seq(ctx, batch.seq_id[i][0]);
    float * out = output + batch.seq_id[i][0] * n_embd_out;
    common_embd_normalize(embd, out, n_embd_out, 2);  // L2 normalize
}

// Print cosine similarity matrix
for (int i = 0; i < n_prompts; i++) {
    for (int j = 0; j < n_prompts; j++) {
        float sim = common_embd_similarity_cos(
            emb + i * n_embd_out,
            emb + j * n_embd_out,
            n_embd_out);
        printf("%6.2f ", sim);
    }
    printf("\n");
}

Max-absolute normalization for quantized storage:

float raw[768];
float quantized[768];

// Normalize to int16 range
common_embd_normalize(raw, quantized, 768, 0);

// Now values are in approx [-32760, 32760] range
// Can be safely cast to int16_t for compact storage

No normalization (pass-through):

float raw[768];
float same[768];

// Copy without normalization
common_embd_normalize(raw, same, 768, -1);
// same[i] == raw[i] for all i

Similarity with pre-normalized vectors (optimization):

// If vectors are already L2-normalized, cosine similarity equals dot product
// But common_embd_similarity_cos computes it correctly regardless of normalization
float sim = common_embd_similarity_cos(norm_a, norm_b, 768);

Related Pages

• Principle:Ggml_org_Llama_cpp_Embedding_Normalization