Principle:CARLA simulator Carla Sensor Data Streaming
| Knowledge Sources | |
|---|---|
| Domains | Simulation, Perception, Sensor_Fusion |
| Last Updated | 2026-02-15 00:00 GMT |
Overview
Sensor data streaming is the mechanism by which spawned sensors push observation data to the client through registered callback functions using the observer pattern.
Description
In CARLA, sensors are server-side actors that generate data (images, point clouds, radar detections, GPS fixes, inertial measurements) during each simulation tick. To receive this data on the client side, the user registers a callback function on each sensor using the Sensor.listen() method. When the sensor produces new data, the callback is invoked with a SensorData object containing the observation.
The data streaming pipeline works as follows:
- The sensor actor is spawned via World.spawn_actor() with a blueprint, transform, and optional attachment to a parent actor (typically a vehicle).
- The client calls Sensor.listen(callback) to register a callable that will receive each frame of sensor data.
- On each simulation tick (or according to the sensor's sensor_tick parameter), the server renders the sensor's observation, serializes it, and transmits it to the client.
- The client-side runtime deserializes the data and invokes the registered callback on a background thread.
The callback receives a typed SensorData subclass depending on the sensor modality: carla.Image for cameras, carla.LidarMeasurement for LiDAR, carla.RadarMeasurement for radar, carla.GnssMeasurement for GNSS, and carla.IMUMeasurement for IMU.
Usage
Sensor spawning and callback registration are used whenever setting up a sensor data collection rig. After configuring blueprints and transforms, the sensors must be spawned as attached children of the ego vehicle and then activated with listen() before the data collection loop begins.
Theoretical Basis
Observer Pattern: CARLA's sensor data streaming implements the observer (publish-subscribe) pattern. The sensor acts as the subject (publisher) that produces data events, and the callback function acts as the observer (subscriber) that reacts to each event. This decouples the data production (server-side rendering) from data consumption (client-side processing), allowing flexible and modular data handling.
Push-Based Data Flow: Unlike a poll-based model where the client would request data each frame, CARLA uses a push-based model where the server automatically delivers data to registered callbacks. This eliminates the need for the client to manage request timing and ensures that no data frames are missed (in synchronous mode).
Actor Attachment Model: When a sensor is spawned with attach_to=vehicle, the engine automatically updates the sensor's world transform each tick to maintain its relative pose to the vehicle. This uses CARLA's actor attachment system, which supports both rigid attachment (default, sensor moves exactly with parent) and spring arm attachment (for chase cameras with physics-based lag).
Thread Safety Considerations: Callbacks are invoked on background threads managed by CARLA's client runtime. This means the callback function must be thread-safe. Common patterns include: (1) writing to a thread-safe queue, (2) using locks to protect shared state, or (3) performing only simple copy/save operations within the callback.
SensorData Hierarchy: All sensor data types inherit from a base SensorData class that provides common attributes: frame (simulation frame number), timestamp (simulation time in seconds), and transform (sensor's world transform at the time of capture). These are essential for temporal alignment and spatial referencing.