Implementation:Ggml org Ggml Cann common

Metadata

Field	Value
Page Type	Implementation (Infrastructure Header)
Knowledge Sources	GGML
Domains	ML_Infrastructure, Tensor_Computing, NPU_Computing
Last Updated	2026-02-10 12:00 GMT

Overview

Common header defining shared types, error handling macros, device context structures, memory pools, and graph caching for the CANN backend.

Description

common.h (641 lines) is the foundational header for the entire CANN backend. All other CANN source files include it. It defines:

Constants:

MATRIX_ROW_PADDING (512) -- alignment requirement for matrix rows
GGML_CANN_MAX_STREAMS (8) -- maximum concurrent ACL streams

Error handling:

ACL_CHECK(stmt) macro -- validates ACL return codes and calls ggml_cann_error() on failure with file, line, and function context
ggml_cann_error() -- prints diagnostic information including the current device ID and recent ACL error message, then aborts

Device information:

ggml_cann_device_info -- device count and per-device capabilities (compute capability, shared memory, VMM support, VRAM)
ggml_cann_info() -- returns a cached reference to device information
ggml_cann_set_device() -- thread-safe device context switching

Memory management:

ggml_cann_pool -- abstract interface for device memory allocation/deallocation
ggml_cann_pool_alloc -- RAII wrapper that automatically returns memory to the pool on scope exit

Graph caching:

ggml_cann_graph -- compiled computation graph representation
ggml_cann_graph_lru_cache -- LRU cache for compiled graphs, enabling reuse across inference invocations

Backend context:

ggml_backend_cann_context -- holds device ID, description, ACL stream, memory pool, graph cache, RoPE cache, and tensor cache. This is the central context struct passed to all CANN operation functions.

Usage

Include this header in any CANN backend source file that needs access to the backend context, memory pool, error handling, or device information.

Code Reference

Source Location

GGML repo, file: src/ggml-cann/common.h, 641 lines.

Signature

// Error handling
[[noreturn]] void ggml_cann_error(const char * stmt, const char * func,
                                   const char * file, int line, const char * msg);
#define ACL_CHECK(stmt) ACL_CHECK_GEN(stmt, 0, aclGetRecentErrMsg)

// Device management
const ggml_cann_device_info & ggml_cann_info();
void ggml_cann_set_device(int32_t device);

// Environment helpers
std::optional<std::string> get_env_as_lowercase(const std::string & name);
bool parse_bool(const std::string & value);
int  parse_integer(const std::string & value);

// Memory pool interface
struct ggml_cann_pool {
    virtual ~ggml_cann_pool() = default;
    virtual void * alloc(size_t size, size_t * actual_size) = 0;
    virtual void free(void * ptr, size_t size) = 0;
};

Import

#include "common.h"

Dependencies

ggml-impl.h -- internal GGML utilities
ggml-cann.h -- public CANN API header
ggml.h -- core GGML types
acl/acl.h -- Huawei ACL runtime

I/O Contract

Inputs

Parameter	Type	Required	Description
`device`	`int32_t`	Yes (for `ggml_cann_set_device`)	Target Ascend NPU device index.
`size`	`size_t`	Yes (for pool alloc)	Requested memory allocation size in bytes.

Outputs

Output	Type	Description
Device info	`const ggml_cann_device_info &`	Cached device capabilities including device count, VRAM, and VMM support.
Pool memory	`void *`	Pointer to allocated device memory from the CANN memory pool.

Usage Examples

Error-Checked ACL Calls

#include "common.h"

// ACL_CHECK macro validates return code automatically
ACL_CHECK(aclrtSetDevice(0));
ACL_CHECK(aclrtCreateStream(&stream));

RAII Memory Pool Allocation

#include "common.h"

void some_operation(ggml_backend_cann_context & ctx) {
    // Automatically freed when alloc goes out of scope
    ggml_cann_pool_alloc workspace(ctx.pool(), workspace_size);
    void * ws_ptr = workspace.ptr;
    // ... use workspace for ACLNN operator ...
}

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment