Implementation:NVIDIA DALI FITS Parser

Knowledge Sources	NVIDIA_DALI
Domains	Utilities, File_IO
Last Updated	2026-02-08 16:00 GMT

Overview

Implements FITS (Flexible Image Transport System) file parsing and data extraction routines for reading astronomical image data into DALI tensors.

Description

The FITS parser implementation in dali/util/fits.cc provides the core logic for reading FITS files, a standard format widely used in astronomy and scientific computing. It leverages the CFITSIO library to open, interpret, and extract image data stored in FITS Header Data Units (HDUs). The implementation supports a wide range of numeric types including signed and unsigned integers from 8-bit to 64-bit, as well as single and double precision floating point values.

A central responsibility of this module is header parsing via ParseHeader, which reads the HDU type, image dimensions, equivalent image type, and compression metadata from a FITS file pointer. For compressed FITS images, additional metadata such as tile sizes, Rice compression parameters (bytepix, blocksize), and scaling values (bscale, bzero) are extracted to support GPU-accelerated decompression.

The module also provides ExtractUndecodedData, a function that reads the raw Rice-coded compressed tile data from a compressed image HDU without performing decompression. This is implemented using a recursive template function ExtractData that iterates over up to 6 dimensions of tile coordinates, reading the raw byte content from each compressed tile row. Thread safety for non-reentrant CFITSIO builds is handled by the FitsLock class, which conditionally acquires a mutex.

Usage

Use the FITS parser when loading FITS format astronomical or scientific image files into the DALI pipeline. Call ParseHeader to obtain metadata about the image (shape, data type, compression state), then either read pixel data directly for uncompressed images or use ExtractUndecodedData to obtain raw compressed tile data for subsequent GPU decompression.

Code Reference

Source Location

• Repository: NVIDIA_DALI
• File: dali/util/fits.cc
• Lines: 1-304

Signature

void ParseHeader(HeaderData& parsed_header, fitsfile* src);

int ExtractUndecodedData(fitsfile* fptr, std::vector<uint8_t>& data,
                         std::vector<int64_t>& tile_offset, std::vector<int64_t>& tile_size,
                         int64_t rows, int* status);

DALIDataType HeaderData::type() const;
size_t HeaderData::size() const;
size_t HeaderData::nbytes() const;

void HandleFitsError(int status);

FitsLock::FitsLock();

Import

#include "dali/util/fits.h"

I/O Contract

Inputs

Name	Type	Required	Description
src / fptr	fitsfile*	Yes	Open CFITSIO file pointer positioned at the desired HDU
parsed_header	HeaderData&	Yes	Output struct to receive parsed metadata (shape, type, compression info)
data	std::vector<uint8_t>&	Yes (ExtractUndecodedData)	Output buffer for raw compressed tile bytes
tile_offset	std::vector<int64_t>&	Yes (ExtractUndecodedData)	Output vector of byte offsets for each tile
tile_size	std::vector<int64_t>&	Yes (ExtractUndecodedData)	Output vector of element counts per tile
rows	int64_t	Yes (ExtractUndecodedData)	Number of compressed rows to extract
status	int*	Yes (ExtractUndecodedData)	CFITSIO status code (0 on success)

Outputs

Name	Type	Description
parsed_header	HeaderData	Populated struct containing shape, type info, HDU type, compression metadata
data	std::vector<uint8_t>	Concatenated raw compressed tile data bytes
tile_offset	std::vector<int64_t>	Cumulative byte offsets marking the start of each tile in the data buffer
tile_size	std::vector<int64_t>	Number of decompressed elements per tile
return value	int	CFITSIO status code (0 indicates success)

Usage Examples

Parsing a FITS Header

#include "dali/util/fits.h"

// Open a FITS file
auto handle = dali::fits::FitsHandle::OpenFile("image.fits", READONLY);

// Parse header metadata
dali::fits::HeaderData header;
dali::fits::ParseHeader(header, handle);

// Access parsed info
auto dtype = header.type();        // DALI data type
auto shape = header.shape;         // TensorShape
auto nbytes = header.nbytes();     // total bytes
bool compressed = header.compressed;

Extracting Raw Compressed Tile Data

#include "dali/util/fits.h"

auto handle = dali::fits::FitsHandle::OpenFile("compressed.fits", READONLY);

dali::fits::HeaderData header;
dali::fits::ParseHeader(header, handle);

if (header.compressed) {
  std::vector<uint8_t> raw_data;
  std::vector<int64_t> tile_offsets, tile_sizes;
  int status = 0;

  dali::fits::ExtractUndecodedData(handle, raw_data, tile_offsets,
                                   tile_sizes, header.rows, &status);
  // raw_data now contains compressed tile bytes for GPU decompression
}

Related Pages

• Environment:NVIDIA_DALI_CUDA_GPU_Environment