Implementation:Vllm project Vllm Spec Decode Method Config

Knowledge Sources	vLLM vLLM Docs
Domains	LLM Inference, Speculative Decoding, Configuration
Last Updated	2026-02-08 13:00 GMT

Overview

Concrete tool for selecting the speculative decoding method via user configuration provided by vLLM.

Description

The speculation method is chosen via the --method CLI argument in the vLLM speculative decoding example, or equivalently via the method key in the speculative_config dictionary passed to the LLM constructor. The method determines which drafting strategy is used: EAGLE head prediction, EAGLE3 improved head prediction, n-gram prompt lookup (no extra model), MTP multi-token prediction heads (model must natively support them), or a separate draft model. This is the first decision point in any speculative decoding pipeline and directly determines which subsequent configuration parameters are relevant.

Usage

Use this configuration pattern when setting up speculative decoding in vLLM. The method choice drives all downstream decisions: which model weights to download, which config keys to set, and what performance characteristics to expect.

Code Reference

Source Location

• Repository: vllm
• File: examples/offline_inference/spec_decode.py (CLI arg definition), vllm/engine/arg_utils.py:L512 (engine-level config), vllm/config/speculative.py:L46-53 (SpeculativeMethod type)

Signature

# CLI argument definition
parser.add_argument(
    "--method",
    type=str,
    default="eagle",
    choices=["ngram", "eagle", "eagle3", "mtp", "draft_model"],
)

# Type definition from vllm/config/speculative.py
SpeculativeMethod = Literal[
    "ngram",
    "medusa",
    "mlp_speculator",
    "draft_model",
    "suffix",
    EagleModelTypes,  # includes "eagle", "eagle3", and MTP model types
]

Import

# No special import needed for the method string; it is a plain string
# passed as part of the speculative_config dictionary:
from vllm import LLM

llm = LLM(model="meta-llama/Llama-3.1-8B-Instruct",
           speculative_config={"method": "eagle", ...})

I/O Contract

Inputs

Name	Type	Required	Description
method	Literal["eagle", "eagle3", "ngram", "mtp", "draft_model"]	Yes	The speculative decoding strategy to use. Determines which draft mechanism generates candidate tokens.

Outputs

Name	Type	Description
speculative_config (partial)	dict[str, Any]	A dictionary with the "method" key set, ready to receive method-specific parameters before being passed to the LLM constructor.

Usage Examples

Selecting EAGLE Method

speculative_config = {
    "method": "eagle",
    "model": "yuhuili/EAGLE-LLaMA3.1-Instruct-8B",
    "num_speculative_tokens": 3,
}

Selecting EAGLE3 Method

speculative_config = {
    "method": "eagle3",
    "model": "yuhuili/EAGLE3-LLaMA3.1-Instruct-8B",
    "num_speculative_tokens": 3,
}

Selecting N-gram Method

speculative_config = {
    "method": "ngram",
    "num_speculative_tokens": 3,
    "prompt_lookup_max": 5,
    "prompt_lookup_min": 2,
}

Selecting MTP Method

speculative_config = {
    "method": "mtp",
    "num_speculative_tokens": 2,
}

Selecting Draft Model Method

speculative_config = {
    "method": "draft_model",
    "model": "meta-llama/Llama-3.2-1B-Instruct",
    "num_speculative_tokens": 3,
}

Related Pages

Implements Principle

• Principle:Vllm_project_Vllm_Speculative_Method_Selection