Implementation:NVIDIA DALI C API V2 Pipeline Wrapper

Knowledge Sources	NVIDIA_DALI
Domains	Data_Pipeline, C_API
Last Updated	2026-02-08 16:00 GMT

Overview

The PipelineWrapper header defines the C++ wrapper class that adapts a DALI Pipeline for use through the C API v2 opaque handle system.

Description

This header file declares the PipelineWrapper class within the dali::c_api namespace, which serves as the internal C++ implementation behind the daliPipeline_h opaque handle type. It inherits from _DALIPipeline (the C handle base struct) and owns a std::unique_ptr<Pipeline> to the underlying DALI pipeline.

The class provides a clean C++ interface that the C API functions delegate to. It supports two construction modes: from a daliPipelineParams_t struct for new pipelines, and from serialized protobuf data with optional parameter overrides for deserialized pipelines. The wrapper exposes: Build(), Run(), and Prefetch() for pipeline execution; PopOutputs() for retrieving pipeline outputs as a PipelineOutputs object; FeedInput() for providing external data to input operators; GetInputCount()/GetInputDesc() and GetOutputCount()/GetOutputDesc() for introspecting pipeline I/O descriptors; and a full checkpoint API for fault-tolerant training.

The Unwrap() method provides direct access to the underlying dali::Pipeline pointer for internal use. The private FeedInputImpl template method handles the backend-specific dispatch when feeding tensor lists to the pipeline. A mutable input_names_ vector caches input operator names for index-based access.

Usage

This header is included by the C API v2 pipeline implementation file and any internal code that needs to access the PipelineWrapper type. It is not part of the public C API header surface; users interact with pipelines through the daliPipeline_h handle and the C functions declared in dali/dali.h.

Code Reference

Source Location

• Repository: NVIDIA_DALI
• File: dali/c_api_2/pipeline.h
• Lines: 1-100

Signature

namespace dali::c_api {

class PipelineWrapper : public _DALIPipeline {
 public:
  explicit PipelineWrapper(const daliPipelineParams_t &params);
  PipelineWrapper(const void *serialized, size_t length, const daliPipelineParams_t &params);
  ~PipelineWrapper();

  std::unique_ptr<PipelineOutputs> PopOutputs(AccessOrder order = AccessOrder::host());

  void Build();
  void Run();
  void Prefetch();

  int GetFeedCount(std::string_view input_name);
  void FeedInput(std::string_view input_name, const ITensorList *input_data,
                 std::optional<std::string_view> data_id,
                 daliFeedInputFlags_t options, AccessOrder order);

  int GetOutputCount() const;
  daliPipelineIODesc_t GetOutputDesc(int idx) const &;

  int GetInputCount() const;
  daliPipelineIODesc_t GetInputDesc(int idx) const &;
  daliPipelineIODesc_t GetInputDesc(std::string_view name) const &;

  dali::Pipeline *Unwrap() const & { return pipeline_.get(); }

  std::unique_ptr<CheckpointWrapper> GetCheckpoint(
      const daliCheckpointExternalData_t *ext) const;
  std::string_view SerializeCheckpoint(CheckpointWrapper &chk) const;
  std::unique_ptr<CheckpointWrapper> DeserializeCheckpoint(std::string_view serialized);
  void RestoreFromCheckpoint(CheckpointWrapper &chk);

 private:
  template <typename Backend>
  void FeedInputImpl(std::string_view input_name, const TensorList<Backend> &tl,
                     std::optional<std::string_view> data_id,
                     daliFeedInputFlags_t options, AccessOrder order);

  std::unique_ptr<Pipeline> pipeline_;
  mutable std::vector<std::string_view> input_names_;
};

PipelineWrapper *ToPointer(daliPipeline_h handle);
CheckpointWrapper *ToPointer(daliCheckpoint_h handle);

}  // namespace dali::c_api

Import

#include "dali/c_api_2/pipeline.h"

I/O Contract

Inputs

Name	Type	Required	Description
params	daliPipelineParams_t	Yes	Pipeline configuration with presence flags for optional fields
serialized	const void *	Yes (deserialize)	Protobuf-serialized pipeline data
length	size_t	Yes (deserialize)	Size of serialized data
input_name	std::string_view	Yes (feed)	Name of the input operator
input_data	const ITensorList *	Yes (feed)	Data to feed to the pipeline input
order	AccessOrder	No	CUDA stream ordering for async operations

Outputs

Name	Type	Description
PipelineOutputs	std::unique_ptr	Owned pipeline output object with tensor lists and traces
daliPipelineIODesc_t	struct	Input/output descriptor with name, device, dtype, ndim, layout
Pipeline *	raw pointer	Direct access to underlying DALI Pipeline via Unwrap()
CheckpointWrapper	std::unique_ptr	Checkpoint object for serialization/restoration

Usage Examples

Internal: Creating a PipelineWrapper

#include "dali/c_api_2/pipeline.h"

using namespace dali::c_api;

// Create from parameters
daliPipelineParams_t params{};
params.max_batch_size_present = true;
params.max_batch_size = 8;
params.num_threads_present = true;
params.num_threads = 4;

auto *wrapper = new PipelineWrapper(params);
wrapper->Build();
wrapper->Run();

auto outputs = wrapper->PopOutputs();
auto tl = outputs->Get(0);  // RefCountedPtr<ITensorList>

delete wrapper;

Internal: Deserializing and Feeding Input

daliPipelineParams_t overrides{};
overrides.max_batch_size_present = true;
overrides.max_batch_size = 16;

auto *wrapper = new PipelineWrapper(proto.data(), proto.size(), overrides);
wrapper->Build();

// Get feed count to know how many inputs to provide
int count = wrapper->GetFeedCount("ext_input");

// Feed input
wrapper->FeedInput("ext_input", tensor_list_ptr, "data_001",
                   DALI_FEED_INPUT_SYNC, AccessOrder(cuda_stream));

Related Pages

• Environment:NVIDIA_DALI_CUDA_GPU_Environment