Implementation:Mit han lab Llm awq Apply awq

Overview

Concrete tool for applying precomputed AWQ transforms to a model provided by the llm-awq library.

Source

File: awq/quantize/pre_quant.py, Lines: 252-254

Signature

def apply_awq(model, awq_results):
    apply_scale(model, awq_results["scale"])
    apply_clip(model, awq_results["clip"])

Import

from awq.quantize.pre_quant import apply_awq

I/O

Inputs:

• model (nn.Module) - FP16 model
• awq_results (dict) - dictionary with "scale" and "clip" keys, loaded via torch.load

Output:

• None (model modified in-place)

Related Pages

• Principle:Mit_han_lab_Llm_awq_AWQ_Transform_Application
• Environment:Mit_han_lab_Llm_awq_Python_Runtime_Environment
• Environment:Mit_han_lab_Llm_awq_VILA_Multimodal_Environment

Knowledge Sources

• Repo|llm-awq|https://github.com/mit-han-lab/llm-awq

Domains

• Quantization
• NLP