Implementation:NVIDIA DALI AccessOrder
| Knowledge Sources | |
|---|---|
| Domains | Core, CUDA |
| Last Updated | 2026-02-08 16:00 GMT |
Overview
This file implements the AccessOrder class methods that manage synchronization between CUDA stream orderings and host execution contexts.
Description
The AccessOrder class abstracts the concept of execution ordering in DALI. An access order can represent a CUDA stream (device order), host-synchronous order, or a null (unspecified) order. This abstraction is used throughout DALI to express data dependencies between pipeline stages without exposing raw CUDA stream handles.
The implementation file (access_order.cc) provides the constructor that infers the device ID from a cudaStream_t handle, the wait(const AccessOrder &other) method that establishes ordering dependencies between two access orders using CUDA events, and the wait(cudaEvent_t event) method that makes the current order wait for a specific CUDA event. The wait method handles special CUDA stream handles (default stream, per-thread default stream, legacy default stream) by performing device switches via DeviceGuard when necessary. For host-order targets, it falls back to cudaStreamSynchronize or cudaEventSynchronize.
A helper function is_ambiguous_handle identifies CUDA stream handles (0, cudaStreamPerThread, cudaStreamLegacy) whose meaning depends on the current device context, requiring explicit device switching before recording or waiting on events.
Usage
Use AccessOrder when orchestrating data access across multiple CUDA streams or between GPU and CPU execution. Call wait(other) to ensure that work submitted in other completes before work in the current order proceeds. This is the standard mechanism in DALI for cross-stream synchronization within the pipeline.
Code Reference
Source Location
- Repository: NVIDIA_DALI
- File: dali/core/access_order.cc
- Lines: 1-102
Signature
class DLL_PUBLIC AccessOrder {
public:
constexpr AccessOrder() = default;
constexpr AccessOrder(cudaStream_t stream, int device_id);
AccessOrder(cudaStream_t stream);
static constexpr AccessOrder host();
cudaStream_t get() const noexcept;
cudaStream_t stream() const noexcept;
int device_id() const noexcept;
bool is_host() const noexcept;
bool is_device() const noexcept;
bool has_value() const noexcept;
void wait(const AccessOrder &other) const;
void wait(cudaEvent_t event) const;
bool operator==(const AccessOrder &other) const noexcept;
bool operator!=(const AccessOrder &other) const noexcept;
};
Import
#include "dali/core/access_order.h"
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| stream | cudaStream_t |
Yes (constructor) | CUDA stream handle; device ID is inferred automatically |
| other | const AccessOrder & |
Yes (wait) | The access order to synchronize after; if null or host, wait is a no-op |
| event | cudaEvent_t |
Yes (wait overload) | A CUDA event to wait on; throws if called on a null AccessOrder |
Outputs
| Name | Type | Description |
|---|---|---|
| N/A | void | The wait methods establish ordering constraints as side effects; they do not return values
|
Usage Examples
Synchronizing two CUDA stream orders
AccessOrder order1(stream1);
AccessOrder order2(stream2);
// Make stream2 wait for all work submitted to stream1
order2.wait(order1);
Host synchronization
AccessOrder device_order(stream);
AccessOrder host_order = AccessOrder::host();
// Blocks the host until stream completes
host_order.wait(device_order);
Waiting for a CUDA event
AccessOrder order(stream);
cudaEvent_t event;
cudaEventCreate(&event);
cudaEventRecord(event, other_stream);
// Make this stream wait for the event
order.wait(event);