Implementation:Sgl project Sglang Scalar Type

Knowledge Sources	Sgl_project_Sglang
Domains	Type_System, Quantization, Kernel_Infrastructure
Last Updated	2026-02-10 00:00 GMT

Overview

C++ class representing arbitrary scalar numeric types including sub-byte quantized formats not supported by torch.dtype, enabling type-safe kernel dispatch for diverse quantization schemes.

Description

The ScalarType class in the sglang namespace provides a compile-time and runtime representation for numeric types with arbitrary precision. It stores six fields:

• exponent (uint8_t) -- size of the exponent field (0 for integer types)
• mantissa (uint8_t) -- size of the mantissa field (or integer bits excluding sign)
• signed_ (bool) -- whether the type supports negative numbers
• bias (int32_t) -- stored values equal value + bias (for quantized types)
• finite_values_only (bool) -- whether +/-inf is excluded
• nan_repr (NanRepr enum) -- NaN representation strategy: NAN_NONE, NAN_IEEE_754, or NAN_EXTD_RANGE_MAX_MIN

Factory methods:

• ScalarType::int_(size_bits, bias) -- creates a signed integer type
• ScalarType::uint(size_bits, bias) -- creates an unsigned integer type
• ScalarType::float_IEEE754(exponent, mantissa) -- creates an IEEE 754 compliant float type
• ScalarType::float_(exponent, mantissa, finite_values_only, nan_repr) -- creates a non-standard float type

Compact ID encoding: The Id type (int64_t) provides a unique compile-time identifier computed by bit-packing all fields. The id() method encodes, and from_id() decodes. This enables C++17 template specialization since literal class objects cannot be used as template parameters before C++20.

Query methods: size_bits(), is_signed(), is_integer(), and is_floating_point() provide type introspection.

Pre-defined constants: The file defines commonly used quantization types like kU4B8 (4-bit unsigned with bias 8), kU8B128 (8-bit unsigned with bias 128), and floating-point types kFE4M3 (FP8 E4M3), kFE2M1 (FP4 E2M1), etc.

Usage

Include this header in any CUDA kernel or C++ code that needs to dispatch on quantization type. The ScalarTypeId type alias is used as a template parameter in Marlin dequantization and GEMM kernels for compile-time specialization on weight format.

Code Reference

Source Location

• Repository: Sgl_project_Sglang
• File: sgl-kernel/include/scalar_type.hpp
• Lines: 1-331

Signature

namespace sglang {

class ScalarType {
public:
  enum NanRepr : uint8_t {
    NAN_NONE = 0,
    NAN_IEEE_754 = 1,
    NAN_EXTD_RANGE_MAX_MIN = 2,
    NAN_REPR_ID_MAX
  };

  constexpr ScalarType(
      uint8_t exponent, uint8_t mantissa, bool signed_,
      int32_t bias, bool finite_values_only = false,
      NanRepr nan_repr = NAN_IEEE_754);

  // Factory methods
  static constexpr ScalarType int_(uint8_t size_bits, int32_t bias = 0);
  static constexpr ScalarType uint(uint8_t size_bits, int32_t bias = 0);
  static constexpr ScalarType float_IEEE754(uint8_t exponent, uint8_t mantissa);
  static constexpr ScalarType float_(uint8_t exponent, uint8_t mantissa,
      bool finite_values_only, NanRepr nan_repr);

  // Fields
  uint8_t const exponent;
  uint8_t const mantissa;
  bool const signed_;
  int32_t const bias;
  bool const finite_values_only;
  NanRepr const nan_repr;

  // Compact ID for template specialization
  using Id = int64_t;
  constexpr Id id() const;
  static constexpr ScalarType from_id(Id id);

  // Query methods
  constexpr int64_t size_bits() const;
  constexpr bool is_signed() const;
  constexpr bool is_integer() const;
  constexpr bool is_floating_point() const;
};

using ScalarTypeId = ScalarType::Id;

// Pre-defined quantization types
static constexpr auto kU4B8 = ScalarType::uint(4, 8);
static constexpr auto kU8B128 = ScalarType::uint(8, 128);
static constexpr auto kFE4M3 = ScalarType::float_IEEE754(4, 3);
static constexpr auto kFE2M1 = ScalarType::float_(2, 1, true, ...);

} // namespace sglang

Import

#include "scalar_type.hpp"

// Underlying dependency:
#include <torch/library.h>  // for TORCH_CHECK

I/O Contract

Inputs

Name	Type	Required	Description
exponent	uint8_t	Yes	Number of exponent bits (0 for integer types)
mantissa	uint8_t	Yes	Number of mantissa bits (or integer bits minus sign)
signed_	bool	Yes	Whether the type supports negative values
bias	int32_t	No	Quantization bias: stored_value = value + bias (default 0)
finite_values_only	bool	No	Whether +/-inf values are excluded (default false)
nan_repr	NanRepr	No	NaN representation mode (default NAN_IEEE_754)

Outputs

Name	Type	Description
ScalarType instance	ScalarType	Immutable type descriptor with all fields set
id()	int64_t	Compact bit-packed identifier suitable for template parameters
size_bits()	int64_t	Total bit width: mantissa + exponent + sign

Usage Examples

// Define a custom 4-bit unsigned integer with bias 8
constexpr auto my_type = sglang::ScalarType::uint(4, 8);

// Get the compact ID for template specialization
constexpr sglang::ScalarTypeId type_id = my_type.id();

// Use in Marlin dequantization kernel dispatch
dequant<half2, sglang::kU4B8.id()>(packed_val, frag_b);

// Reconstruct type from ID
constexpr auto recovered = sglang::ScalarType::from_id(type_id);
static_assert(recovered.size_bits() == 4);
static_assert(!recovered.is_signed());
static_assert(recovered.is_integer());

Related Pages

• Environment:Sgl_project_Sglang_CUDA_GPU_Runtime