Implementation:Ggml org Llama cpp Ngram Mod Header

Knowledge Sources	Ggml_org_Llama_cpp
Domains	Speculative_Decoding, Hashing
Last Updated	2026-02-15 00:00 GMT

Overview

Declares the `common_ngram_mod` struct, a basic n-gram hash table for mapping token n-grams to predicted next tokens.

Description

Defines a struct with a configurable n-gram size and a flat vector of int32_t entries acting as a hash table. Provides `idx()` for hash computation, `add()` for insertion, `get()` for lookup (returns -1 if not found), `reset()` for clearing, and size/memory reporting methods. Uses EMPTY = -1 as the sentinel value for unoccupied entries.

Usage

Include this header to declare `common_ngram_mod` instances. This provides a minimal, self-contained n-gram hash structure for speculative decoding, designed for simplicity and low overhead in memory and computation.

Code Reference

Source Location

• Repository: Ggml_org_Llama_cpp
• File: common/ngram-mod.h
• Lines: 1-38

Signature

struct common_ngram_mod {
    using entry_t = int32_t;
    static constexpr entry_t EMPTY = -1;

    common_ngram_mod(uint16_t n, size_t size);

    size_t  idx(const entry_t * tokens) const;
    void    add(const entry_t * tokens);
    entry_t get(const entry_t * tokens) const;

    void reset();

    size_t get_n()    const;
    size_t get_used() const;
    size_t size()       const;
    size_t size_bytes() const;

private:
    size_t n;
    size_t used;
    std::vector<entry_t> entries;
};

Import

#include "ngram-mod.h"

I/O Contract

Inputs

Name	Type	Required	Description
n	uint16_t	Yes	N-gram size (number of tokens used as the hash key)
size	size_t	Yes	Number of entries in the hash table (determines memory usage)
tokens	const entry_t *	Yes	Pointer to token array for add/get/idx operations

Outputs

Name	Type	Description
get return	entry_t (int32_t)	Predicted next token, or EMPTY (-1) if not found
idx return	size_t	Computed hash index for the given n-gram tokens
size return	size_t	Total number of hash table entries
size_bytes return	size_t	Memory usage of the entries vector in bytes

Usage Examples

#include "ngram-mod.h"

// Declare a 3-gram hash table with 500K entries
common_ngram_mod ngram(3, 500000);

// Check table statistics
size_t n      = ngram.get_n();      // 3
size_t used   = ngram.get_used();   // 0
size_t total  = ngram.size();       // 500000
size_t bytes  = ngram.size_bytes(); // 500000 * sizeof(int32_t)

Related Pages

• Principle:Ggml_org_Llama_cpp_Speculative_Decoding