Workflow:CARLA simulator Carla Sensor Data Collection
| Knowledge Sources | |
|---|---|
| Domains | Autonomous_Driving, Data_Collection, Sensor_Fusion |
| Last Updated | 2026-02-15 12:00 GMT |
Overview
End-to-end process for configuring, synchronizing, and collecting data from multiple CARLA sensors (cameras, LiDAR, radar, GNSS, IMU) attached to a vehicle, producing timestamped datasets for perception algorithm development.
Description
This workflow covers the complete sensor data collection pipeline in CARLA. It begins with selecting and configuring sensor blueprints with appropriate attributes (resolution, field of view, update rate), proceeds through spawning sensors attached to a vehicle at specific mounting positions, and establishes synchronized data capture using queues and frame-aligned collection. The workflow supports multiple sensor modalities simultaneously: RGB cameras from various angles, LiDAR point clouds (standard and semantic), radar detections, GNSS positioning, and IMU measurements. Data is collected frame-by-frame with guaranteed temporal alignment in synchronous mode.
Usage
Execute this workflow when you need to collect multi-modal sensor data for training perception models, testing sensor fusion algorithms, or validating detection pipelines. Prerequisites include a running CARLA server with a spawned ego vehicle. The workflow is particularly relevant for generating labeled datasets with ground truth from semantic sensors.
Execution Steps
Step 1: Configure Synchronous Mode
Enable synchronous mode on the world with a fixed delta time to ensure all sensors produce data at consistent intervals. This guarantees that each world.tick() advances the simulation by a fixed amount and all sensors capture data at the same simulation timestamp.
Key considerations:
- Synchronous mode is essential for temporally aligned multi-sensor data
- Fixed delta seconds (e.g., 0.05 for 20 Hz) determines the sensor capture rate
- Both the world and the Traffic Manager must be set to synchronous mode
- Always save original settings and restore them on cleanup
Step 2: Select and Configure Sensor Blueprints
For each desired sensor, find its blueprint in the library and configure its attributes. Camera sensors require resolution (image_size_x, image_size_y), field of view (fov), and optional post-processing. LiDAR requires channels, range, points_per_second, and rotation_frequency. Radar needs horizontal and vertical FOV, range, and points_per_second.
Key considerations:
- Higher resolution cameras produce more data but increase processing time
- LiDAR points_per_second and channels determine point cloud density
- sensor_tick=0.0 means the sensor fires every simulation tick; higher values reduce frequency
- Semantic sensors (semantic segmentation camera, semantic LiDAR) provide per-pixel/per-point labels automatically
Step 3: Define Sensor Mounting Positions
Create transforms that position each sensor relative to the ego vehicle. Camera transforms define the mounting point and viewing angle (e.g., front-facing at windshield height, rear-facing, side mirrors). LiDAR is typically mounted on the vehicle roof. IMU and GNSS are placed at the vehicle center.
Key considerations:
- Transforms use the vehicle's local coordinate system (x=forward, y=right, z=up)
- SpringArm attachment allows sensors to avoid clipping through the vehicle body
- Multiple cameras at different angles provide comprehensive visual coverage
- Sensor positions affect occlusion patterns and field-of-view coverage
Step 4: Spawn Sensors and Register Callbacks
Spawn each sensor as an actor attached to the ego vehicle using the configured blueprint and mounting transform. Register a listen callback on each sensor that places received data into a thread-safe queue tagged with the sensor name and frame number.
Key considerations:
- Sensors begin producing data immediately after spawning
- Callbacks execute in background threads; use thread-safe structures (Queue) for data collection
- Each callback should tag data with the frame number for synchronization verification
- Store sensor actor references for cleanup
Step 5: Synchronize Multi-Sensor Data
On each simulation tick, wait for all sensors to report data for the current frame. Use a blocking queue pattern with timeout to collect data from every sensor before proceeding. Verify frame alignment by checking that all sensor data carries the same frame number.
Key considerations:
- After world.tick(), wait for N sensor data items (one per sensor) from the queue
- Use a timeout (e.g., 1.0 seconds) to detect missing sensor data
- Frame number mismatches indicate synchronization issues
- In asynchronous mode, use world.wait_for_tick() instead but synchronization is not guaranteed
Step 6: Process and Save Sensor Data
For each collected frame, process the raw sensor data into usable formats. Camera images can be saved directly to disk using save_to_disk() with optional color conversion. LiDAR point clouds are extracted as NumPy arrays of (x, y, z, intensity) values. Radar detections provide velocity, azimuth, altitude, and depth per detection point.
Key considerations:
- Camera data supports multiple format conversions (Raw, Depth, LogarithmicDepth, CityScapesPalette)
- LiDAR raw data is a flat array of float32 values; reshape to (N, 4) for (x, y, z, intensity)
- Semantic LiDAR adds object index and semantic tag per point
- GNSS provides latitude, longitude, altitude; IMU provides accelerometer and gyroscope readings
- Save data with frame-number naming for post-processing alignment
Step 7: Cleanup Sensors
When data collection is complete, stop listening on all sensors, destroy sensor actors, and restore the original world settings. Proper cleanup prevents resource leaks and ensures the server returns to a usable state for the next session.
Key considerations:
- Call sensor.stop() before sensor.destroy() to cleanly shut down data streaming
- Restore original world settings (especially synchronous mode) before disconnecting
- Verify all sensor actors are destroyed to free GPU resources