Principle:Volcengine Verl VLM Actor Forward
| Knowledge Sources | |
|---|---|
| Domains | Vision_Language_Models, Training_Infrastructure, Deep_Learning |
| Last Updated | 2026-02-07 14:00 GMT |
Overview
The process of extracting multimodal inputs from data batches and passing them through the VLM actor model during training, ensuring correct handling of image tensors alongside text tokens.
Description
VLM Actor Forward handles the training-time forward pass for vision-language models. Unlike text-only models where the forward pass only receives token IDs, VLM models require additional multimodal inputs:
- pixel_values: Processed image tensors from the vision encoder
- image_grid_thw: Spatial dimensions of processed images (for 3D RoPE)
- Other model-specific inputs: Varies by VLM architecture
The extract_multi_modal_inputs utility function inspects the model's forward signature and extracts matching fields from the data batch, passing them as keyword arguments.
Usage
VLM actor forward is used internally during the RL training step whenever the actor model processes multimodal data. The extraction is automatic — if multimodal fields are present in the data batch and the model's forward method accepts them, they are passed through.
Theoretical Basis
VLM actor forward extends standard language model forward with multimodal inputs:
# Abstract VLM actor forward
# 1. Extract multimodal inputs from batch
mm_inputs = extract_multi_modal_inputs(model, batch)
# e.g., {"pixel_values": tensor, "image_grid_thw": tensor}
# 2. Forward pass with both text and vision inputs
logits = model(
input_ids=batch["input_ids"],
attention_mask=batch["attention_mask"],
position_ids=batch["position_ids"],
**mm_inputs # pixel_values, image_grid_thw, etc.
)