Implementation:NVIDIA DALI Pascal TFRecord Creator
| Knowledge Sources | |
|---|---|
| Domains | Data_Conversion, Object_Detection |
| Last Updated | 2026-02-08 16:00 GMT |
Overview
Converts Pascal VOC dataset XML annotations and images into sharded TFRecord files for object detection training with EfficientDet.
Description
This script reads Pascal VOC-format XML annotation files and their corresponding JPEG images to produce TFRecord files compatible with TensorFlow detection pipelines. It parses bounding box coordinates from VOC XML, normalizes them relative to image dimensions, and encodes both the image and annotations into `tf.train.Example` protocol buffers. The script supports VOC2007, VOC2012, and merged datasets, and includes a built-in label map covering all 20 Pascal VOC categories plus background.
Each generated TFRecord example contains the JPEG-encoded image, normalized bounding boxes ([ymin, xmin, ymax, xmax]), class labels and names, truncation flags, pose annotations, difficulty flags, and object areas. The script also generates a companion COCO-format JSON annotation file alongside the TFRecord output, which can be used for evaluation with COCO evaluation tools. Global image and annotation IDs are maintained incrementally to ensure uniqueness across the dataset.
The converter supports configurable sharding (default 100 shards), optional filtering of difficult instances, and an optional limit on the number of images to process. It uses the companion `tfrecord_util` module for XML parsing and TFRecord feature serialization.
Usage
Use this script to prepare Pascal VOC training and validation data as TFRecord files before training EfficientDet models. Run it as a standalone command-line tool specifying the VOC data directory, year, split, and output path.
Code Reference
Source Location
- Repository: NVIDIA_DALI
- File: docs/examples/use_cases/tensorflow/efficientdet/dataset/create_pascal_tfrecord.py
- Lines: 1-326
Signature
def get_image_id(filename) -> int:
...
def get_ann_id() -> int:
...
def dict_to_tf_example(
data,
dataset_directory,
label_map_dict,
ignore_difficult_instances=False,
image_subdirectory="JPEGImages",
ann_json_dict=None,
) -> tf.train.Example:
...
def main(_):
...
Import
# Standalone script; run via command line:
# python create_pascal_tfrecord.py --data_dir=/tmp/VOCdevkit --year=VOC2012 --output_path=/tmp/pascal
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| data_dir | str (flag) | Yes | Root directory of the raw Pascal VOC dataset |
| set | str (flag) | No | Dataset split: 'train', 'val', 'trainval', or 'test' (default: 'train') |
| year | str (flag) | No | VOC challenge year: 'VOC2007', 'VOC2012', or 'merged' (default: 'VOC2007') |
| output_path | str (flag) | Yes | Output path prefix for TFRecord and JSON files |
| annotations_dir | str (flag) | No | Relative path to annotations directory (default: 'Annotations') |
| label_map_json_path | str (flag) | No | Path to custom label map JSON file |
| ignore_difficult_instances | bool (flag) | No | Whether to skip difficult instances (default: False) |
| num_shards | int (flag) | No | Number of output TFRecord shards (default: 100) |
| num_images | int (flag) | No | Maximum number of images to process (default: all) |
Outputs
| Name | Type | Description |
|---|---|---|
| TFRecord files | files | Sharded TFRecord files at `{output_path}-{shard_id}-of-{num_shards}.tfrecord` |
| JSON annotation file | file | COCO-format annotation JSON at `{output_path}.json` |
Usage Examples
Convert Pascal VOC 2012 Training Set
python create_pascal_tfrecord.py \
--data_dir=/data/VOCdevkit \
--year=VOC2012 \
--set=train \
--output_path=/data/tfrecords/pascal_train \
--num_shards=100