Implementation:InternLM Lmdeploy Gemm KernelImpl

Knowledge Sources	InternLM_Lmdeploy
Domains	GPU_Kernels, GEMM
Last Updated	2026-02-07 15:00 GMT

Overview

Concrete kernel implementation for SM70/SM75/SM80 architectures that populates kernel descriptors from the GemmUniversal template parameters and implements the Launch method for dispatching GEMM kernels.

Description

KernelImpl<Gemm> bridges the compile-time GemmUniversal type and the runtime dispatch system. The constructor extracts all properties from the template parameter -- operand types, orders, striding modes, packing, quantization group sizes, CTA/MMA tile sizes, cache eviction policies, pipeline stages, and architecture -- and populates the base class KernelDesc and KernelInfo.

The Launch method:

1. Constructs a scheduler with the problem shape, swizzle, and splits
2. Prepares GemmParam and EpilogueParam from the input matrix descriptors
3. Launches the CUDA kernel with the appropriate grid/block dimensions and dynamic shared memory

Also provides GetMaxSplits (calculates maximum split-K given workspace constraints) and GetMaxSwizzle (queries the scheduler for optimal swizzle factor).

Usage

Created by the Registry for each configured kernel variant and stored as Kernel* pointers for dispatch.

Code Reference

Source Location

• Repository: InternLM_Lmdeploy
• File: src/turbomind/kernels/gemm/kernel_impl.h

Signature

template<class Gemm>
class KernelImpl : public Kernel {
public:
    static constexpr int CTA_M = Gemm::CTA_M;
    static constexpr int CTA_N = Gemm::CTA_N;
    static constexpr int CTA_K = Gemm::CTA_K;

    KernelImpl();  // populates desc_ and info_

    int Launch(const Operation&, float alpha,
               const void* A, ..., cudaStream_t stream) override;
    int GetMaxSplits(...) const override;
    int GetMaxSwizzle(const int4& shape) const override;
};

Import

#include "src/turbomind/kernels/gemm/kernel_impl.h"

I/O Contract

Inputs

Name	Type	Required	Description
Gemm (template)	type	Yes	A GemmUniversal instantiation defining all kernel parameters
operation	Operation	Yes	Dispatch policy, epilogue flags, quantization info

Outputs

Name	Type	Description
(Launch)	int	0 on success, launches the CUDA GEMM kernel
desc_	KernelDesc	Populated kernel descriptor for dispatch matching

Usage Examples

// In the Registry
auto kernel = std::make_unique<KernelImpl<typename Config::Kernel>>();
// Later used for dispatch
kernel->Launch(op, alpha, A, Adesc, U, Udesc, B, Bdesc, V, Vdesc,
               beta, C, Cdesc, D, Ddesc, swizzle, splits, workspace, stream);

Related Pages

• Environment:InternLM_Lmdeploy_CUDA_GPU_Runtime