Implementation:Langchain ai Langchain HuggingFacePipeline

Knowledge Sources	Langchain_ai_Langchain
Domains	LLM, HuggingFace, Local Inference
Last Updated	2026-02-11 00:00 GMT

Overview

HuggingFacePipeline is a LangChain LLM wrapper that enables running HuggingFace Transformers pipelines locally for text generation, summarization, and translation tasks.

Description

The HuggingFacePipeline class, defined in the langchain-huggingface partner package, extends BaseLLM from langchain-core. It wraps a HuggingFace Transformers pipeline object to provide local inference for text-generation, text2text-generation, image-text-to-text, summarization, and translation tasks. The class supports multiple hardware backends including default (CPU/GPU), OpenVINO, and Intel IPEX for optimized inference. It processes prompts in configurable batches and supports both synchronous generation and streaming output via a background thread with TextIteratorStreamer.

Usage

Import this class when you need to run HuggingFace models locally without relying on a remote API. It is particularly useful for on-device inference, air-gapped environments, or when using specialized hardware acceleration backends like OpenVINO or IPEX.

Code Reference

Source Location

• Repository: Langchain_ai_Langchain
• File: libs/partners/huggingface/langchain_huggingface/llms/huggingface_pipeline.py
• Lines: 1-422

Signature

class HuggingFacePipeline(BaseLLM):
    pipeline: Any = None
    model_id: str | None = None
    model_kwargs: dict | None = None
    pipeline_kwargs: dict | None = None
    batch_size: int = DEFAULT_BATCH_SIZE  # default: 4

    @classmethod
    def from_model_id(
        cls,
        model_id: str,
        task: str,
        backend: str = "default",
        device: int | None = None,
        device_map: str | None = None,
        model_kwargs: dict | None = None,
        pipeline_kwargs: dict | None = None,
        batch_size: int = DEFAULT_BATCH_SIZE,
        **kwargs: Any,
    ) -> HuggingFacePipeline: ...

    def _generate(
        self,
        prompts: list[str],
        stop: list[str] | None = None,
        run_manager: CallbackManagerForLLMRun | None = None,
        **kwargs: Any,
    ) -> LLMResult: ...

    def _stream(
        self,
        prompt: str,
        stop: list[str] | None = None,
        run_manager: CallbackManagerForLLMRun | None = None,
        **kwargs: Any,
    ) -> Iterator[GenerationChunk]: ...

Import

from langchain_huggingface import HuggingFacePipeline

I/O Contract

Inputs

Name	Type	Required	Description
pipeline	Any	No	A pre-constructed HuggingFace Transformers pipeline object. If not provided, use from_model_id to construct one.
model_id	str or None	No	The HuggingFace model identifier (e.g. "gpt2"). Inferred from pipeline or defaults to "gpt2".
model_kwargs	dict or None	No	Keyword arguments passed to the model constructor (e.g. device_map, trust_remote_code).
pipeline_kwargs	dict or None	No	Keyword arguments passed to the HuggingFace pipeline at construction time.
batch_size	int	No	Number of prompts to process per batch. Defaults to 4.

Outputs

Name	Type	Description
LLMResult	LLMResult	Contains a list of Generation objects, one per prompt, each holding the generated text.
GenerationChunk	Iterator[GenerationChunk]	When streaming, yields chunks of generated text incrementally.

Usage Examples

Basic Usage (from_model_id)

from langchain_huggingface import HuggingFacePipeline

hf = HuggingFacePipeline.from_model_id(
    model_id="gpt2",
    task="text-generation",
    pipeline_kwargs={"max_new_tokens": 10},
)

result = hf.invoke("The meaning of life is")
print(result)

Passing a Pipeline Directly

from langchain_huggingface import HuggingFacePipeline
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

model_id = "gpt2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    max_new_tokens=10,
)
hf = HuggingFacePipeline(pipeline=pipe)

result = hf.invoke("Hello, world!")
print(result)

Streaming

from langchain_huggingface import HuggingFacePipeline

hf = HuggingFacePipeline.from_model_id(
    model_id="gpt2",
    task="text-generation",
    pipeline_kwargs={"max_new_tokens": 50},
)

for chunk in hf.stream("Once upon a time"):
    print(chunk, end="")