Implementation:NVIDIA NeMo Curator QwenLM

Knowledge Sources	NVIDIA NeMo Curator
Domains	Machine Learning, NLP, Text Generation
Last Updated	2026-02-14 00:00 GMT

Overview

Wraps the Qwen2.5-14B-Instruct text-only language model for caption enhancement and text generation via vLLM.

Description

The QwenLM class implements ModelInterface and provides a high-throughput text generation wrapper around the Qwen/Qwen2.5-14B-Instruct model (revision cf98f3b) using the vLLM inference engine.

On setup(), it initializes a vLLM LLM instance with the Qwen model weights, optional FP8 quantization (controlled by the fp8 parameter), and eager mode disabled for optimized execution. It configures SamplingParams with conservative generation settings: temperature=0.1, top_p=0.001, and repetition_penalty=1.05, with a configurable max_tokens. An AutoTokenizer is also loaded for chat template formatting.

The generate method takes a list of chat-format message dictionaries, applies the Qwen tokenizer's chat template (with add_generation_prompt=True), runs batch generation through vLLM, and returns the generated text strings.

If vLLM is not installed, the module defines dummy LLM and SamplingParams classes for type-hint compatibility, and setup() raises an ImportError at runtime.

Usage

Use QwenLM in the video curation pipeline for caption enhancement, where it takes initial captions (e.g., from QwenVL) and refines or enhances them using the text-only language model. It provides high-quality text generation for data annotation tasks.

Code Reference

Source Location

• Repository: NeMo-Curator
• File: nemo_curator/models/qwen_lm.py
• Lines: 1-93

Signature

class QwenLM(ModelInterface):
    def __init__(
        self,
        model_dir: str,
        caption_batch_size: int,
        fp8: bool,
        max_output_tokens: int,
    ): ...
    def model_id_names(self) -> list[str]: ...
    def setup(self) -> None: ...
    def generate(self, inputs: list[dict[str, Any]]) -> list[str]: ...
    @classmethod
    def download_weights_on_node(cls, model_dir: str) -> None: ...

Import

from nemo_curator.models.qwen_lm import QwenLM

I/O Contract

Inputs (Constructor)

Name	Type	Required	Description
model_dir	str	Yes	Path to the directory where model weights are stored or will be downloaded
caption_batch_size	int	Yes	Batch size for caption generation
fp8	bool	Yes	Whether to use FP8 quantization for reduced memory usage
max_output_tokens	int	Yes	Maximum number of tokens to generate per input

Inputs (generate)

Name	Type	Required	Description
inputs	list[dict[str, Any]]	Yes	List of chat-format message dictionaries (e.g., [{"role": "user", "content": "..."}])

Outputs

Name	Type	Description
results	list[str]	List of generated text strings, one per input

Model Configuration

Parameter	Value
Model ID	Qwen/Qwen2.5-14B-Instruct
Revision	cf98f3b
Temperature	0.1
top_p	0.001
Repetition penalty	1.05
Quantization	Optional FP8
Eager mode	Disabled (enforce_eager=False)

Usage Examples

Basic Usage

from nemo_curator.models.qwen_lm import QwenLM

# Download weights first
QwenLM.download_weights_on_node("/path/to/models")

# Initialize and setup
model = QwenLM(
    model_dir="/path/to/models",
    caption_batch_size=16,
    fp8=True,
    max_output_tokens=512,
)
model.setup()

# Generate text from chat messages
inputs = [
    [{"role": "user", "content": "Enhance this caption: A dog running in a park."}],
]
results = model.generate(inputs)
print(results[0])

Related Pages

• Environment:NVIDIA_NeMo_Curator_Python_Linux_Base
• NVIDIA_NeMo_Curator_ModelInterface -- Base class that QwenLM implements
• NVIDIA_NeMo_Curator_QwenVL -- Vision-language model that pairs with QwenLM for caption enhancement