Implementation:Princeton nlp SimPO Transformers Pipeline

Knowledge Sources	SimPO HF Pipelines
Domains	NLP, Inference
Last Updated	2026-02-08 04:30 GMT

Overview

Wrapper documentation for HuggingFace's transformers.pipeline as used for SimPO model inference.

Description

The generate.py script demonstrates inference with a SimPO-trained model using HuggingFace's pipeline API. The pipeline is created with task="text-generation" and a model ID pointing to a SimPO-trained model on HuggingFace Hub. It accepts OpenAI-format chat messages (list of role/content dicts) and returns the full conversation including the model's generated response. The pipeline handles chat template application, tokenization, generation, and decoding internally.

Usage

Use for quick inference testing with SimPO-trained models. Supports both greedy (do_sample=False) and sampling-based generation.

Code Reference

Source Location

• Repository: SimPO
• File: generate.py (Lines 1-13)

Signature

# Pipeline creation:
generator = transformers.pipeline(
    task: str = "text-generation",
    model: str,
    model_kwargs: dict = {"torch_dtype": torch.bfloat16},
    device: str = "cuda",
) -> Pipeline

# Pipeline invocation:
outputs = generator(
    messages: List[Dict[str, str]],
    do_sample: bool = False,
    max_new_tokens: int = 200,
) -> List[Dict]

Import

import torch
from transformers import pipeline

I/O Contract

Inputs

Name	Type	Required	Description
task	str	Yes	Must be "text-generation"
model	str	Yes	HuggingFace model ID (e.g., "princeton-nlp/gemma-2-9b-it-SimPO")
model_kwargs	dict	No	Model loading kwargs (e.g., {"torch_dtype": torch.bfloat16})
device	str	No	Device for inference (default: "cuda")
messages	List[Dict[str, str]]	Yes	OpenAI-format chat messages: [{"role": "user", "content": "..."}]
do_sample	bool	No	Whether to use sampling (default: False for greedy)
max_new_tokens	int	No	Maximum tokens to generate (default: 200)

Outputs

Name	Type	Description
outputs	List[Dict]	outputs[0]["generated_text"] contains full conversation with model response appended

Usage Examples

Basic Greedy Inference

import torch
from transformers import pipeline

# Load SimPO-trained model
model_id = "princeton-nlp/gemma-2-9b-it-SimPO"
generator = pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device="cuda",
)

# Generate response (greedy decoding)
messages = [{"role": "user", "content": "What's the difference between llamas and alpacas?"}]
outputs = generator(messages, do_sample=False, max_new_tokens=200)

# Extract the model's response
print(outputs[0]["generated_text"])
# Prints the full conversation including the assistant's response

Sampling-Based Generation

# Use sampling for more diverse responses
outputs = generator(
    [{"role": "user", "content": "Write a haiku about programming."}],
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    max_new_tokens=100,
)
print(outputs[0]["generated_text"])

Multi-Turn Conversation

messages = [
    {"role": "user", "content": "What is SimPO?"},
    {"role": "assistant", "content": "SimPO is a preference optimization method..."},
    {"role": "user", "content": "How does it differ from DPO?"},
]
outputs = generator(messages, do_sample=False, max_new_tokens=300)
print(outputs[0]["generated_text"][-1]["content"])

Related Pages

Implements Principle

• Principle:Princeton_nlp_SimPO_Pipeline_Inference