Implementation:NVIDIA DALI CUDAEventPool

Knowledge Sources	NVIDIA_DALI
Domains	Core, CUDA
Last Updated	2026-02-08 16:00 GMT

Overview

This file implements the CUDAEventPool class, a thread-safe pool for reusing CUDA event objects across devices to reduce allocation overhead.

Description

CUDA events are used extensively in DALI for stream synchronization (e.g., in AccessOrder::wait). Creating and destroying CUDA events has non-trivial overhead, so the CUDAEventPool maintains per-device linked lists of reusable event objects. When an event is requested via Get, the pool first attempts to recycle a previously returned event; if none is available, a new event is created with the configured flags (default: cudaEventDisableTiming). When an event is returned via Put, it is placed back into the appropriate per-device list.

The pool uses a spinlock for thread safety and maintains a separate unused_ free list for EventEntry node objects, avoiding repeated heap allocations for the intrusive linked list nodes. The constructor queries the CUDA device count to size the per-device event array. The Purge method destroys all pooled events and releases auxiliary memory. A singleton accessor (instance()) provides a process-wide shared pool.

The implementation uses intrusive singly-linked lists with manual Pop/Push operations for minimal overhead and cache-friendly traversal.

Usage

Use CUDAEventPool::instance() to obtain the global pool, then call Get(device_id) to lease an event and Put(event, device_id) to return it. This pattern is used internally by AccessOrder::wait for cross-stream synchronization. Application code should prefer using AccessOrder directly rather than interacting with the event pool.

Code Reference

Source Location

• Repository: NVIDIA_DALI
• File: dali/core/cuda_event_pool.cc
• Lines: 1-93

Signature

class DLL_PUBLIC CUDAEventPool {
 public:
  ~CUDAEventPool();
  explicit CUDAEventPool(unsigned event_flags = cudaEventDisableTiming);

  CUDAEvent Get(int device_id = -1);
  void Put(CUDAEvent &&event, int device_id = -1);
  void Purge();

  static CUDAEventPool &instance();

 private:
  CUDAEvent GetFromPool(int device_id);
  unsigned event_flags_;

  struct EventEntry {
    EventEntry() = default;
    explicit EventEntry(CUDAEvent event, EventEntry *next = nullptr);
    CUDAEvent event;
    EventEntry *next = nullptr;
  };

  EventEntry *unused_ = nullptr;
  std::vector<EventEntry *> dev_events_;
  spinlock lock_;

  static EventEntry *Pop(EventEntry *&head);
  static void Push(EventEntry *&head, EventEntry *new_entry);
  void DeleteList(EventEntry *&head);
};

Import

#include "dali/core/cuda_event_pool.h"

I/O Contract

Inputs

Name	Type	Required	Description
event_flags	unsigned	No	CUDA event creation flags; defaults to cudaEventDisableTiming
device_id	int	No	CUDA device ordinal; defaults to -1 (current device)
event	CUDAEvent &&	Yes (Put)	CUDA event wrapper to return to the pool; ownership is transferred

Outputs

Name	Type	Description
CUDAEvent	CUDAEvent	A CUDA event wrapper object obtained from the pool or newly created
instance	CUDAEventPool &	Reference to the process-wide singleton pool

Usage Examples

Getting and returning an event

auto &pool = CUDAEventPool::instance();
int device_id = 0;

// Obtain an event (recycled or newly created)
CUDAEvent event = pool.Get(device_id);

// Use the event for synchronization
cudaEventRecord(event, stream);
cudaStreamWaitEvent(other_stream, event, 0);

// Return the event to the pool
pool.Put(std::move(event), device_id);

Using with default (current) device

auto &pool = CUDAEventPool::instance();
CUDAEvent event = pool.Get();  // uses current device
// ... use event ...
pool.Put(std::move(event));    // returns to current device pool

Related Pages

• Environment:NVIDIA_DALI_CUDA_GPU_Environment