Implementation:Ollama Ollama Llama KV Cache ISWA Types

Knowledge Sources	Ollama
Domains	LLM Inference, Memory Management
Last Updated	2025-02-15 00:00 GMT

Overview

Header declaring the ISWA (Interleaved Sliding Window Attention) KV cache class and its context type for managing two separate caches for SWA and non-SWA layers.

Description

Declares llama_kv_cache_iswa inheriting from llama_memory_i, with constructor parameters for key/value types, offloading, SWA-specific settings, and layer filter callbacks. Also declares llama_kv_cache_iswa_context which manages the combined context state during batch processing, holding contexts for both the base and SWA caches.

Usage

Include this header when working with the ISWA KV cache type or when the code needs to access the separate base/SWA cache instances through the ISWA interface.

Code Reference

Source Location

• Repository: Ollama
• File: llama/llama.cpp/src/llama-kv-cache-iswa.h
• Lines: 1-137

Signature

class llama_kv_cache_iswa : public llama_memory_i {
public:
    llama_kv_cache_iswa(
            const llama_model & model,
                    ggml_type   type_k,
                    ggml_type   type_v,
                         bool   v_trans,
                         bool   offload,
                         bool   swa_full,
                         bool   unified,
                     uint32_t   kv_size,
                     uint32_t   n_seq_max,
                     uint32_t   n_ubatch,
                     uint32_t   n_pad,
        const layer_filter_cb & filter,
        const  layer_reuse_cb & reuse);

    llama_kv_cache * get_base() const;
    llama_kv_cache * get_swa () const;
};

class llama_kv_cache_iswa_context : public llama_memory_context_i {
    // ...
};

Import

#include "llama-kv-cache-iswa.h"

I/O Contract

Inputs

Name	Type	Required	Description
model	const llama_model &	Yes	Model reference for hyperparameters and layer info
type_k	ggml_type	Yes	Key tensor data type
type_v	ggml_type	Yes	Value tensor data type

Outputs

Name	Type	Description
get_base()	llama_kv_cache*	Access to the non-SWA cache
get_swa()	llama_kv_cache*	Access to the SWA cache

Usage Examples

#include "llama-kv-cache-iswa.h"

// Created internally by llama_model::init_memory()
// The ISWA cache wraps two llama_kv_cache instances
auto iswa = std::make_unique<llama_kv_cache_iswa>(
    model, type_k, type_v, v_trans, offload, swa_full, unified,
    kv_size, n_seq_max, n_ubatch, n_pad, filter, reuse);

Related Pages

• Principle:Ollama_Ollama_LLM_Memory_Architecture