Implementation:Ollama Ollama Llama Memory Recurrent Types

Knowledge Sources	Ollama
Domains	LLM Inference, Memory Management
Last Updated	2025-02-15 00:00 GMT

Overview

Header declaring the recurrent state memory class and context for SSM-based model architectures.

Description

Declares llama_memory_recurrent inheriting from llama_memory_i, with per-layer R (recurrent state) and S (short convolution cache) tensor vectors. Uses mem_cell structs for tracking per-slot metadata: position, sequence IDs, source index, and tail. llama_memory_recurrent_context manages per-batch context with methods for graph input preparation, state application, and ubatch iteration.

Usage

Include this header when implementing or extending recurrent model support, or when building compute graphs that need access to recurrent state tensors.

Code Reference

Source Location

• Repository: Ollama
• File: llama/llama.cpp/src/llama-memory-recurrent.h
• Lines: 1-182

Signature

class llama_memory_recurrent : public llama_memory_i {
public:
    struct mem_cell {
        llama_pos pos  = -1;
        int32_t   src  = -1;
        int32_t   tail = -1;
        std::set<llama_seq_id> seq_id;
        bool has_seq_id(const llama_seq_id & id) const;
        bool is_empty() const;
    };

    uint32_t head = 0;
    uint32_t size = 0;
    uint32_t used = 0;

    std::vector<mem_cell> cells;
    std::vector<ggml_tensor *> r_l;
    std::vector<ggml_tensor *> s_l;
};

class llama_memory_recurrent_context : public llama_memory_context_i {
public:
    uint32_t get_n_rs() const;
    uint32_t get_head() const;
    ggml_tensor * get_r_l(int32_t il) const;
    ggml_tensor * get_s_l(int32_t il) const;
    int32_t s_copy(int i) const;
};

Import

#include "llama-memory-recurrent.h"

I/O Contract

Inputs

Name	Type	Required	Description
model	const llama_model &	Yes	Model reference
type_r	ggml_type	Yes	Recurrent state data type
type_s	ggml_type	Yes	Convolution state data type

Outputs

Name	Type	Description
r_l	std::vector<ggml_tensor*>	Per-layer recurrent state tensors
s_l	std::vector<ggml_tensor*>	Per-layer convolution state tensors
cells	std::vector<mem_cell>	Cell metadata for state management

Usage Examples

#include "llama-memory-recurrent.h"

// Access through context during graph building:
const auto * mctx = static_cast<const llama_memory_recurrent_context *>(mctx);
ggml_tensor * r = mctx->get_r_l(il);
ggml_tensor * s = mctx->get_s_l(il);
uint32_t head = mctx->get_head();

Related Pages

• Principle:Ollama_Ollama_LLM_Memory_Architecture