Implementation:NVIDIA DALI DynamicScratchpad Tests

Knowledge Sources	NVIDIA_DALI
Domains	Kernels, GPU_Computing
Last Updated	2026-02-08 16:00 GMT

Overview

Provides Google Test cases that verify the correctness and measure the performance of the DynamicScratchpad class, covering stream-ordered pinned memory safety and allocation throughput.

Description

The dynamic_scratchpad_test.cc file contains two test cases within the dali::kernels::test namespace. The first test, BasicTest, validates that memory allocated through a DynamicScratchpad is usable and accessible on the correct backend. It specifically verifies that pinned memory blocks are released in stream order, preventing premature reuse while the GPU stream is still executing. The test allocates a 64 KiB buffer of pinned memory, copies data through device memory, and makes up to 1000 attempts to catch the stream still running to confirm proper ordering semantics.

The second test, Perf, is a performance benchmark that measures allocation throughput and scratchpad destruction time across different memory kinds (host, pinned, device). It performs 100,000 iterations, each creating a DynamicScratchpad instance, performing multiple allocations with random sizes following a Poisson distribution (average 1 KiB, max 64 MiB), and then measuring the destruction time. Results are reported as median, 90th percentile, 99th percentile, and mean times. A helper function ProcessResults computes and prints these statistics.

These tests serve as both correctness validation and performance regression detection for the scratchpad memory subsystem. The Perf test alternates between two CUDA streams to exercise cross-stream behavior.

Usage

Run these tests as part of the DALI kernel test suite using Google Test. The BasicTest should be run to verify stream-ordered deallocation correctness after any changes to the DynamicScratchpad or underlying memory resource implementations. The Perf test is useful for benchmarking allocation performance and detecting regressions in the memory management stack. Both tests require a CUDA-capable GPU.

Code Reference

Source Location

• Repository: NVIDIA_DALI
• File: dali/kernels/dynamic_scratchpad_test.cc
• Lines: 1-154

Signature

namespace dali {
namespace kernels {
namespace test {

TEST(DynamicScratchpad, BasicTest);

inline void ProcessResults(vector<double> &times, const string &header);

TEST(DynamicScratchpad, Perf);

}  // namespace test
}  // namespace kernels
}  // namespace dali

Import

#include "dali/kernels/dynamic_scratchpad.h"
#include <gtest/gtest.h>

I/O Contract

Inputs

Name	Type	Required	Description
N (BasicTest)	int (constant)	Yes	Buffer size: 64 KiB (64 << 10) used for the pinned memory correctness test
max_attempts (BasicTest)	int (constant)	Yes	Maximum number of retry attempts (1000) to catch the stream still running
max_attempts (Perf)	int (constant)	Yes	Number of benchmark iterations (100,000)
size_dist (Perf)	std::poisson_distribution	Yes	Random allocation size distribution with mean 1024 bytes
num_dist (Perf)	std::uniform_int_distribution<>	Yes	Random number of allocations per iteration (1 to 100)

Outputs

Name	Type	Description
Test pass/fail	Google Test assertions	ASSERT_EQ and ASSERT_TRUE verify data integrity and stream-ordered semantics
Performance statistics	Console output	Median, 90th/99th percentile, and mean allocation/destruction times in nanoseconds

Usage Examples

Running the BasicTest

// This test is executed via the Google Test framework:
// ./dali_kernels_test --gtest_filter=DynamicScratchpad.BasicTest
//
// The test verifies:
// 1. DynamicScratchpad allocates usable pinned memory
// 2. Data round-trips correctly: host -> pinned -> device -> pinned -> host
// 3. Pinned memory is not prematurely reused while the stream is running

TEST(DynamicScratchpad, BasicTest) {
  const int N = 64 << 10;  // 64 KiB
  std::vector<char> in(N);
  for (int i = 0; i < N; i++)
    in[i] = i + 42;

  auto stream = CUDAStreamPool::instance().Get();
  auto dev = mm::alloc_raw_unique<char, mm::memory_kind::device>(N);
  // ... multiple attempts to catch stream still running ...
}

Running the Performance Benchmark

// Execute via:
// ./dali_kernels_test --gtest_filter=DynamicScratchpad.Perf
//
// Output example:
// Allocation performance for host memory
// Median time:            120 ns
// 90th percentile:        250 ns
// ...
// Scratchpad destruction time
// Median time:            500 ns

Related Pages

• Environment:NVIDIA_DALI_CUDA_GPU_Environment