Implementation:Mit han lab Llm awq Split and repack

Overview

Concrete tools for splitting and converting AWQ quantized checkpoints for efficient deployment provided by the llm-awq library.

Source

• Repository: llm-awq (https://github.com/mit-han-lab/llm-awq)
• File: tinychat/split_ckpt.py (split), tinychat/offline-weight-repacker.py (repacker)
• Lines: 7-19 (split function), 111-156 (offline_repacker)

Function Signature

def split(ckpt_path: str, out_folder_path: str):

Import

These are CLI scripts, not imported as modules. Run via:

python tinychat/split_ckpt.py --input_path model.pt --output_path ./shards/
python tinychat/offline-weight-repacker.py --input v1_model.pt --output v2_model.pt

I/O

split

• Inputs:
  • ckpt_path (str) - path to .pt checkpoint
  • out_folder_path (str) - output directory
• Output: per-tensor .pt shard files (one per state_dict key)

offline_repacker

• Inputs:
  • --input - v1 checkpoint path
  • --output - v2 checkpoint path
• Output: v2-format checkpoint with interleaved qweight layout and precomputed scaled_zeros

Related Pages

• Principle:Mit_han_lab_Llm_awq_Checkpoint_Preparation

Metadata

• Knowledge Sources: Repo|llm-awq|https://github.com/mit-han-lab/llm-awq
• Domains: Deployment, Model_Compression