Implementation:LaurentMazare Tch rs Cifar Loader
| Knowledge Sources | |
|---|---|
| Domains | Dataset Loading, Computer Vision, CIFAR-10 |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
The cifar module loads the CIFAR-10 dataset from its binary file format into a Dataset struct containing training and test image tensors with labels.
Description
This module reads the CIFAR-10 binary format as distributed from the University of Toronto. The binary format stores images as flat byte arrays where each record is 3073 bytes: 1 byte for the label followed by 3072 bytes (32x32x3) of pixel data in channel-first order (red, green, blue planes).
Constants define the image dimensions: W = 32, H = 32, C = 3, BYTES_PER_IMAGE = 3073, and SAMPLES_PER_FILE = 10000.
The internal read_file_ function reads an entire binary file into a byte buffer, then iterates over each of the 10000 samples to extract labels (as Int64 tensors) and images (reshaped to [C, H, W] and cast to Float). Images are normalized to the range [0, 1] by dividing by 255.0.
The public load_dir function takes a directory path and loads:
- 1 test file: test_batch.bin (10,000 images)
- 5 training files: data_batch_1.bin through data_batch_5.bin (50,000 images total)
Training images and labels from the 5 files are concatenated using Tensor::cat. The result is returned as a Dataset struct with labels: 10 (the number of classes).
Usage
Use cifar::load_dir to load the CIFAR-10 dataset for image classification experiments. The dataset directory must contain the binary batch files downloaded from the official source.
Code Reference
Source Location
- Repository: LaurentMazare_Tch_rs
- File: src/vision/cifar.rs
Signature
pub fn load_dir<T: AsRef<std::path::Path>>(dir: T) -> Result<Dataset>;
Import
use tch::vision::cifar;
I/O Contract
| Parameter | Type | Description |
|---|---|---|
| dir | impl AsRef<Path> | Directory containing CIFAR-10 binary batch files |
| Required Files | Samples |
|---|---|
| data_batch_1.bin | 10,000 |
| data_batch_2.bin | 10,000 |
| data_batch_3.bin | 10,000 |
| data_batch_4.bin | 10,000 |
| data_batch_5.bin | 10,000 |
| test_batch.bin | 10,000 |
| Output Field | Type | Shape | Description |
|---|---|---|---|
| train_images | Tensor (Float) | [50000, 3, 32, 32] | Training images normalized to [0, 1] |
| train_labels | Tensor (Int64) | [50000] | Training labels (0-9) |
| test_images | Tensor (Float) | [10000, 3, 32, 32] | Test images normalized to [0, 1] |
| test_labels | Tensor (Int64) | [10000] | Test labels (0-9) |
| labels | i64 | N/A | 10 (number of classes) |
Usage Examples
use tch::vision::cifar;
// Load the CIFAR-10 dataset from a directory
let dataset = cifar::load_dir("/path/to/cifar-10-batches-bin").unwrap();
println!("Training images: {:?}", dataset.train_images.size());
// [50000, 3, 32, 32]
println!("Training labels: {:?}", dataset.train_labels.size());
// [50000]
println!("Test images: {:?}", dataset.test_images.size());
// [10000, 3, 32, 32]
println!("Number of classes: {}", dataset.labels);
// 10