Implementation:Ggml org Ggml Cpu sgemm ppc

Metadata

Field	Value
Page Type	Implementation (PowerPC SGEMM)
Knowledge Sources	GGML
Domains	ML_Infrastructure, Tensor_Computing, CPU_Backend, Quantized_Matrix_Multiplication
Last Updated	2025-05-15 12:00 GMT

Overview

Implements PowerPC-specific tinyBLAS quantized matrix multiplication using POWER10 MMA (Matrix-Multiply Assist) instructions.

Description

llamafile/sgemm-ppc.h provides a specialized tinyBLAS implementation that leverages IBM POWER10's dedicated Matrix-Multiply Assist hardware. Key components include:

1. Template class: tinyBLAS_Q0_PPC<TA> is templated on the A-matrix quantization type (either block_q4_0 or block_q8_0). The B-matrix is always block_q8_0.
2. MMA integration: Uses __vector_quad (128-bit quad accumulator) and __builtin_mma_disassemble_acc to extract results from MMA tile operations. The outer product accumulation uses POWER10's specialized MMA instructions.
3. Tiled execution: matmul_tiled_q0 divides the M-N dimensions into tiles (mc x nc), packs data into SIMD-friendly layouts, then calls KERNEL_Q0 for the inner computation.
4. Data packing: packNormalInt4_large unpacks 4-bit quantized values with nibble extraction and compensation array tracking. packNormal_large packs int8 data with optional sign-flip for B-matrix compatibility.
5. Micro-kernels: KERNEL_4x8, KERNEL_8x4, KERNEL_8x8 implement register-blocked inner kernels with different tile shapes.
6. Scale handling: compute_scale pre-computes the product of A and B block scales for accumulator scaling.
7. Vector permutation: vector_permute_store performs 4-way vector interleaving using vec_perm for optimal data layout.

Usage

This header is included by sgemm.cpp when the __MMA__ (Matrix-Multiply Assist) compiler flag is defined. It is only active on POWER10+ hardware.

Code Reference

Source Location

GGML repo, file: src/ggml-cpu/llamafile/sgemm-ppc.h (333 lines).

Signature

template <typename TA>
class tinyBLAS_Q0_PPC {
  public:
    tinyBLAS_Q0_PPC(int64_t k,
                    const TA *A, int64_t lda,
                    const block_q8_0 *B, int64_t ldb,
                    float *C, int64_t ldc,
                    int ith, int nth);

    void matmul(int64_t m, int64_t n);
    void matmul_tiled_q0(int64_t m, int64_t n,
        int64_t mc, int64_t nc, int64_t kc);
};

Import

// Included automatically by sgemm.cpp when __MMA__ is defined
#include "sgemm-ppc.h"

I/O Contract

Inputs

Parameter	Type	Required	Description
k	int64_t	Yes	Inner dimension (number of elements per row in quantization blocks).
A	const TA *	Yes	Pointer to A-matrix in quantized block format (q4_0 or q8_0).
B	const block_q8_0 *	Yes	Pointer to B-matrix in q8_0 block format.
lda, ldb, ldc	int64_t	Yes	Leading dimensions (number of blocks per row).
ith, nth	int	Yes	Thread index and total thread count for work distribution.

Outputs

Output	Type	Description
C	float *	Result matrix in f32 format, computed as C = A^T * B.

Usage Examples

PowerPC MMA Matrix Multiply (Internal)

// Called from sgemm.cpp's llamafile_sgemm function:
#if defined(__MMA__)
if (Atype == GGML_TYPE_Q4_0 && Btype == GGML_TYPE_Q8_0) {
    tinyBLAS_Q0_PPC<block_q4_0> tb(
        k, (const block_q4_0 *)A, lda / QK4_0,
        (const block_q8_0 *)B, ldb / QK8_0,
        C, ldc, ith, nth);
    tb.matmul(m, n);
    return true;
}
#endif

Related Pages

• Ggml_org_Ggml_Cpu_sgemm -- The main SGEMM dispatcher that includes this PowerPC specialization.
• Ggml_org_Ggml_Cpu_quantization -- Quantization block types (q4_0, q8_0) consumed by these kernels.
• Ggml_org_Ggml_Cpu_amx_mmq -- Intel AMX: analogous accelerated matmul for x86.