Implementation:InternLM Lmdeploy Gemm Registry

Knowledge Sources	InternLM_Lmdeploy
Domains	GPU_Kernels, GEMM
Last Updated	2026-02-07 15:00 GMT

Overview

The kernel registry that discovers, instantiates, and stores all available GEMM kernel variants for the current GPU architecture, providing filtered access for dispatch.

Description

The Registry class manages the collection of GEMM kernels available on the current device:

• Construction: Takes a shared_ptr<cudaDeviceProp> to determine the GPU architecture and selectively registers applicable kernels
• Architecture-specific registration: Private methods register kernels by architecture and tile configuration:
  • sm90_16816_*, sm90_64n32_8: SM90 GMMA-based kernels
  • sm80_16816_*: SM80 Ampere MMA kernels
  • sm75_16816_*: SM75 Turing MMA kernels
  • sm70_884_*: SM70 Volta MMA kernels
  • cublas_float: cuBLAS fallback for FP32
• Add<Config>: Template method to instantiate and register a KernelImpl from a configuration type
• kernels(): Returns raw pointers to all registered kernels for use by Context::Filter

Usage

Created once per device during Gemm initialization. The registry's kernel list is used by the dispatch system to find feasible kernels.

Code Reference

Source Location

• Repository: InternLM_Lmdeploy
• File: src/turbomind/kernels/gemm/registry.h

Signature

class Registry {
public:
    explicit Registry(std::shared_ptr<cudaDeviceProp> device_prop);

    template<class Config>
    bool Add();

    const std::vector<Kernel*>& kernels() const;

private:
    bool Add(std::unique_ptr<Kernel> kernel);
    void sm90_16816_4();  // and other arch-specific registration methods
    void sm80_16816_4();
    void sm70_884_4();
    // ...
};

Import

#include "src/turbomind/kernels/gemm/registry.h"

I/O Contract

Inputs

Name	Type	Required	Description
device_prop	shared_ptr<cudaDeviceProp>	Yes	GPU device properties for architecture detection

Outputs

Name	Type	Description
kernels	const vector<Kernel*>&	All registered kernel variants for the device

Usage Examples

auto prop = std::make_shared<cudaDeviceProp>();
cudaGetDeviceProperties(prop.get(), device_id);
Registry registry(prop);
auto& kernels = registry.kernels();  // All available kernels

Related Pages

• Environment:InternLM_Lmdeploy_CUDA_GPU_Runtime