Implementation:NVIDIA DALI TF Helper
| Knowledge Sources | |
|---|---|
| Domains | TensorFlow_Integration, Data_Pipeline |
| Last Updated | 2026-02-08 16:00 GMT |
Overview
Provides type conversion utilities, shape helpers, batch management classes, and error handling infrastructure for bridging DALI and TensorFlow data types and tensor representations.
Description
This header file defines the shared utility layer used by both the DALIDataset op and the legacy Dali op within the dali_tf_impl namespace. It includes bidirectional type conversion functions (DaliToTfType and TfToDaliType) that map between DALI's daliDataType_t enumeration and TensorFlow's DataType enumeration, covering all supported numeric types including boolean, integer (8/16/32/64-bit signed and unsigned), floating-point (half, float, double), and sentinel invalid types.
The file also provides the DALIException class, which wraps DALI C API error codes into C++ exceptions with detailed error messages including the expression, file location, and line number where the error occurred. The DALI_RETHROW macro enables clean error propagation from DALI C API calls. Shape conversion utilities (ToTfShape, GetTfShape) convert between DALI's raw shape arrays and TensorFlow's TensorShape objects. The IsUniform function checks whether all samples in a tensor list share the same shape, which is required for dense tensor output in TensorFlow.
The Batch class is a central abstraction that encapsulates either per-sample tensors or a single batched tensor, providing a uniform interface for querying shape, dtype, batch size, and data pointers. It supports both sample-mode (vector of individual tensors) and batch-mode (single tensor with leading batch dimension) and includes verification that all samples have consistent dimensionality and data types. The ListOfBatches type alias represents a tuple of batches for one pipeline iteration across multiple inputs.
Usage
Include this header in any DALI TensorFlow plugin C++ source that needs to convert between DALI and TensorFlow types, manage batch data, handle DALI errors, or perform shape conversions. It is a required dependency for both dali_dataset_op.cc and daliop.cc.
Code Reference
Source Location
- Repository: NVIDIA_DALI
- File: dali_tf_plugin/dali_helper.h
- Lines: 1-335
Signature
namespace dali_tf_impl {
class DALIException : public std::runtime_error {
public:
explicit DALIException(daliResult_t result, std::string message,
const char *expression, const char *file, int line);
static daliResult_t Rethrow(daliResult_t result, const char *expression,
const char *file, int line);
static std::string MakeErrorString(daliResult_t result, const std::string &message,
const char *expression, const char *file, int line);
};
constexpr tensorflow::DataType DaliToTfType(daliDataType_t dali_type);
constexpr daliDataType_t TfToDaliType(tensorflow::DataType tf_type);
inline tensorflow::TensorShape ToTfShape(const int64_t *shape, int ndim);
inline tensorflow::TensorShape GetTfShape(const daliTensorDesc_t &desc);
inline tensorflow::TensorShape GetTfShape(daliTensor_h tensor);
inline bool IsUniform(int num_samples, int ndim, const int64_t *shape);
inline std::string ShapeToString(daliTensorList_h tl);
static const void* GetTensorData(const tensorflow::Tensor& t);
class Batch {
public:
Batch() = default;
explicit Batch(BatchStorage &&sample_tensors);
explicit Batch(tensorflow::Tensor &batch_tensor);
int64_t ndim() const;
daliDataType_t dtype() const;
int64_t batch_size() const;
void GetShapes(std::vector<int64_t> &shapes) const;
tensorflow::Status GetPtrs(std::vector<const void *> &ptrs) const;
tensorflow::Status GetPtr(const void *&ptr) const;
tensorflow::Status VerifyUniform(int input_idx);
void clear();
};
using TfExample = std::vector<tensorflow::Tensor>;
using BatchStorage = std::vector<tensorflow::Tensor>;
using ListOfBatches = std::vector<Batch>;
} // namespace dali_tf_impl
Import
#include "dali_tf_plugin/dali_helper.h"
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| dali_type | daliDataType_t | Yes | DALI data type enumeration value for DaliToTfType conversion |
| tf_type | tensorflow::DataType | Yes | TensorFlow data type enumeration value for TfToDaliType conversion |
| shape | const int64_t* | Yes | Raw shape array pointer for ToTfShape conversion |
| ndim | int | Yes | Number of dimensions for shape conversion |
| sample_tensors / batch_tensor | tensorflow::Tensor | Yes | TensorFlow tensor(s) for Batch construction |
Outputs
| Name | Type | Description |
|---|---|---|
| tensorflow::DataType | enum | Converted TF data type from DALI type (DT_INVALID if unsupported) |
| daliDataType_t | enum | Converted DALI data type from TF type (DALI_NO_TYPE if unsupported) |
| tensorflow::TensorShape | object | Converted TensorFlow shape from raw shape array |
| Batch | object | Unified batch representation with shape, dtype, and data access methods |
Usage Examples
Type Conversion
#include "dali_tf_plugin/dali_helper.h"
// Convert DALI type to TensorFlow type
tensorflow::DataType tf_type = dali_tf_impl::DaliToTfType(DALI_FLOAT);
// tf_type == tensorflow::DT_FLOAT
// Convert TensorFlow type to DALI type
daliDataType_t dali_type = dali_tf_impl::TfToDaliType(tensorflow::DT_UINT8);
// dali_type == DALI_UINT8
Batch Construction
#include "dali_tf_plugin/dali_helper.h"
using namespace dali_tf_impl;
// Per-sample mode
BatchStorage samples = {tensor_0, tensor_1, tensor_2};
Batch batch(std::move(samples));
int64_t bs = batch.batch_size(); // 3
daliDataType_t dt = batch.dtype();
// Batched mode
tensorflow::Tensor batched_tensor; // shape: [N, H, W, C]
Batch batch2(batched_tensor);
int64_t ndim = batch2.ndim(); // 3 (excludes batch dim)