Principle:Ggml org Llama cpp File Memory Mapping
| Knowledge Sources | |
|---|---|
| Domains | File_IO, Memory_Mapping |
| Last Updated | 2026-02-15 00:00 GMT |
Overview
File Memory Mapping is the principle of using operating system memory mapping to efficiently access model file data without loading it entirely into application memory.
Description
This principle covers the use of memory-mapped file I/O (mmap) for loading GGUF model files. Instead of reading the entire model file into a heap-allocated buffer, mmap maps the file directly into the process's virtual address space. The operating system then manages loading pages on demand, allowing the system to work with models larger than available RAM through transparent paging.
Usage
Apply this principle when loading large model files where the total file size may approach or exceed available system memory, or when fast startup time is desired since mmap avoids an upfront read of the entire file.
Theoretical Basis
Memory-mapped file I/O leverages the operating system's virtual memory subsystem to treat file contents as if they were in memory. When a page is first accessed, the OS loads it from disk on demand (demand paging). Pages that are not actively used can be evicted and re-loaded transparently. This provides several benefits: zero-copy access to file data, automatic memory management by the OS page cache, the ability to share mapped memory between processes, and the ability to work with files larger than physical RAM. The implementation handles cross-platform differences between POSIX mmap and Windows MapViewOfFile, memory locking (mlock) for latency-sensitive scenarios, and proper alignment for GPU buffer transfers.