Implementation:Ggml org Ggml Cpu simd mappings

Metadata

Field	Value
Page Type	Implementation (SIMD Abstraction)
Knowledge Sources	GGML
Domains	ML_Infrastructure, Tensor_Computing, CPU_Backend, SIMD
Last Updated	2025-05-15 12:00 GMT

Overview

Defines platform-abstracted SIMD macros for FP16/FP32 conversion, vector load/store/FMA/reduce operations across ARM NEON, SVE, x86 SSE/AVX/AVX-512, PowerPC VSX, RISC-V V, LoongArch LSX/LASX, s390x VXE, and Wasm SIMD.

Description

simd-mappings.h is the core SIMD abstraction layer that enables portable high-performance code. All vectorized operations in vec.h, vec.cpp, and quants.c depend on these mappings. Key components include:

1. FP16/FP32 conversion macros:
   • GGML_CPU_FP16_TO_FP32(x) -- Converts FP16 to FP32 using the best available method per architecture.
   • GGML_CPU_FP32_TO_FP16(x) -- Converts FP32 to FP16.
   • ARM NEON: direct __fp16 cast; x86: F16C _cvtsh_ss/_cvtss_sh; PowerPC: inline assembly xscvhpdp/xscvdphp; RISC-V: _Float16 cast; Fallback: precomputed ggml_table_f32_f16 lookup table (256 KB).
2. E8M0 conversion: GGML_CPU_E8M0_TO_FP32_HALF(x) for MX (Microscaling) formats, with lookup table on x86.
3. Generic SIMD type macros: GGML_F32_VEC, GGML_F16_VEC, GGML_F32_VEC_LOAD, GGML_F32_VEC_STORE, GGML_F32_VEC_FMA, GGML_F32_VEC_REDUCE that map to the best available instruction set.
4. Architecture constants: GGML_F32_EPR (elements per register), GGML_F32_ARR (array unroll factor), GGML_F32_STEP (loop step size), and the GGML_SIMD flag indicating SIMD availability.
5. Supported architectures: ARM NEON/SVE, x86 SSE/AVX/AVX2/AVX-512, PowerPC VSX/POWER9, RISC-V V, LoongArch LSX/LASX, s390x VXE/VXE2, WebAssembly SIMD128.

Usage

Include this header in any CPU backend source file that needs portable SIMD operations. The macros automatically resolve to the best available instruction set at compile time.

Code Reference

Source Location

GGML repo, file: src/ggml-cpu/simd-mappings.h (1253 lines).

Signature

// FP16/FP32 conversion
#define GGML_CPU_FP16_TO_FP32(x)  /* architecture-specific */
#define GGML_CPU_FP32_TO_FP16(x)  /* architecture-specific */

// Generic SIMD vector operations
#define GGML_F32_VEC              /* native SIMD vector type */
#define GGML_F32_VEC_ZERO         /* zero vector */
#define GGML_F32_VEC_LOAD(p)      /* load from memory */
#define GGML_F32_VEC_STORE(p, v)  /* store to memory */
#define GGML_F32_VEC_FMA(a, b, c) /* fused multiply-add: a + b * c */
#define GGML_F32_VEC_REDUCE(s, v) /* horizontal sum reduction */

// Architecture detection
#define GGML_SIMD                 /* defined when SIMD is available */
#define GGML_F32_EPR              /* elements per register (e.g., 8 for AVX) */
#define GGML_F32_STEP             /* elements per loop iteration */

// Lookup tables (defined in ggml-cpu.c)
extern float ggml_table_f32_f16[1 << 16];
extern float ggml_table_f32_e8m0_half[1 << 8];

Import

#include "simd-mappings.h"

I/O Contract

Inputs

Macro	Input Type	Description
GGML_CPU_FP16_TO_FP32	ggml_fp16_t	A 16-bit floating point value to convert.
GGML_F32_VEC_LOAD	const float *	Pointer to aligned/unaligned float data.
GGML_F32_VEC_FMA	SIMD vector types	Three vector operands for fused multiply-add.

Outputs

Macro	Output Type	Description
GGML_CPU_FP16_TO_FP32	float	The converted 32-bit float value.
GGML_F32_VEC_LOAD	SIMD vector	Loaded vector register.
GGML_F32_VEC_REDUCE	float	Scalar sum of all vector lanes.

Usage Examples

SIMD-Accelerated Dot Product Loop

#include "simd-mappings.h"

float dot_product(const float * x, const float * y, int n) {
    float sumf = 0.0f;

#if defined(GGML_SIMD)
    const int np = (n & ~(GGML_F32_STEP - 1));
    GGML_F32_VEC sum[GGML_F32_ARR] = { GGML_F32_VEC_ZERO };
    GGML_F32_VEC ax[GGML_F32_ARR], ay[GGML_F32_ARR];

    for (int i = 0; i < np; i += GGML_F32_STEP) {
        for (int j = 0; j < GGML_F32_ARR; j++) {
            ax[j] = GGML_F32_VEC_LOAD(x + i + j * GGML_F32_EPR);
            ay[j] = GGML_F32_VEC_LOAD(y + i + j * GGML_F32_EPR);
            sum[j] = GGML_F32_VEC_FMA(sum[j], ax[j], ay[j]);
        }
    }
    GGML_F32_VEC_REDUCE(sumf, sum);

    for (int i = np; i < n; ++i) {
        sumf += x[i] * y[i];
    }
#else
    for (int i = 0; i < n; ++i) {
        sumf += x[i] * y[i];
    }
#endif

    return sumf;
}

Related Pages

• Ggml_org_Ggml_Cpu_vec_ops -- Vector operations that use these SIMD mappings.
• Ggml_org_Ggml_Cpu_vec_api -- Header declaring vectorized functions built on these mappings.
• Ggml_org_Ggml_Cpu_impl -- Internal header providing MSVC compatibility macros complementing this file.
• Ggml_org_Ggml_Cpu_quantization -- Quantization implementations that use these SIMD primitives.