Implementation:InternLM Lmdeploy Stream

Knowledge Sources	InternLM_Lmdeploy
Domains	GPU_Computing, Core_Infrastructure
Last Updated	2026-02-07 15:00 GMT

Overview

Provides reference-counted CUDA stream and event wrappers with RAII lifetime management, synchronization, and inter-stream dependency support.

Description

The Stream class wraps a CUDA stream (cudaStream_t) with shared-ownership semantics via shared_ptr<StreamImpl>. StreamImpl creates a non-blocking CUDA stream with optional priority in its constructor and destroys it in the destructor. The Stream class provides Sync() for stream synchronization, Wait(event) for inter-stream dependency, handle() for access to the raw cudaStream_t, and operator bool to check validity. The static factory Stream::create(priority) constructs a new stream.

The Event class similarly wraps cudaEvent_t with shared ownership via EventImpl. Events can be created with or without timing support via Event::create(timing). Record(stream) records the event on a stream, and Sync() blocks until the event completes. Implicit conversion to cudaEvent_t is provided for use with CUDA APIs.

Both classes support default construction (invalid/null state), equality comparison, and stream output.

Usage

Used throughout TurboMind as the primary CUDA stream and event abstractions. Streams are pushed onto the thread-local Context stack and used by all GPU operations. Events provide inter-stream synchronization for overlapping computation and communication.

Code Reference

Source Location

• Repository: InternLM_Lmdeploy
• File: src/turbomind/core/stream.h
• Lines: 1-160

Signature

namespace turbomind::core {

class StreamImpl {
public:
    StreamImpl(int priority);
    ~StreamImpl();
    void Sync();
    void Wait(const Event& event);
    cudaStream_t handle() const;
};

class Stream {
public:
    Stream() = default;
    static Stream create(int priority = 0);

    void Sync();
    void Wait(const Event& event);
    cudaStream_t handle() const;
    explicit operator cudaStream_t() const;
    explicit operator bool() const noexcept;
};

class EventImpl {
public:
    explicit EventImpl(unsigned flags);
    ~EventImpl();
    void Record(const Stream& stream);
    void Sync() const;
    cudaEvent_t handle() const;
};

class Event {
public:
    Event() = default;
    static Event create(bool timing = false);
    void Record(const Stream& stream);
    void Sync() const;
    operator cudaEvent_t() const;
    explicit operator bool() const noexcept;
};

}  // namespace turbomind::core

Import

#include "src/turbomind/core/stream.h"

I/O Contract

Inputs

Name	Type	Required	Description
priority	int	Stream::create	CUDA stream priority (default 0)
timing	bool	Event::create	Whether to enable event timing (default false)
event	const Event&	Wait	Event to wait on before proceeding
stream	const Stream&	Record	Stream on which to record the event

Outputs

Name	Type	Description
handle()	cudaStream_t / cudaEvent_t	Raw CUDA handle for use with CUDA APIs
operator bool	bool	Whether the stream/event is valid (non-null)

Usage Examples

#include "src/turbomind/core/stream.h"

using namespace turbomind::core;

// Create a CUDA stream
Stream stream = Stream::create();

// Use the raw handle with CUDA APIs
cudaMemcpyAsync(dst, src, size, cudaMemcpyDefault, stream.handle());

// Synchronize the stream
stream.Sync();

// Create an event and record it
Event event = Event::create();
event.Record(stream);

// Another stream waits on the event
Stream stream2 = Stream::create();
stream2.Wait(event);

Related Pages

• Environment:InternLM_Lmdeploy_CUDA_GPU_Runtime