Implementation:InternLM Lmdeploy HostComm

Knowledge Sources	InternLM_Lmdeploy
Domains	Communication, Multi_Threading
Last Updated	2026-02-07 15:00 GMT

Overview

Defines the abstract interface and typed convenience wrappers for host-side (CPU) collective communication primitives used during model initialization and coordination.

Description

HostCommImpl is a pure virtual base class declaring the host communication API: Broadcast, AllGather, AllReduce, Sync, and Split. It uses function pointer callbacks (copy_fn, ser_fn, des_fn, reduce_fn) to support both trivially-copyable and serializable types. The HostComm wrapper holds a std::shared_ptr<HostCommImpl> for shared ownership. Free function templates provide typed interfaces: Broadcast(comm, value, root), AllGather(comm, value), and AllReduce(comm, value, op) automatically handle serialization for non-trivially-copyable types when processes are separate. The HostGroupId abstract class defines the factory pattern for creating communicators via Initialize, Export/Import, and CreateCommunicator.

Usage

Used during engine startup for coordination between worker processes/threads: exchanging configuration, synchronizing initialization state, and bootstrapping device communicators. The typed template wrappers simplify usage for common value types.

Code Reference

Source Location

• Repository: InternLM_Lmdeploy
• File: src/turbomind/comm/host_comm.h
• Lines: 1-203

Signature

namespace turbomind::comm {

enum class RedOp { kSum, kMin, kMax };

class HostCommImpl {
public:
    virtual ~HostCommImpl();
    virtual int rank() const = 0;
    virtual int n_ranks() const = 0;
    virtual bool is_same_process() const = 0;
    virtual std::shared_ptr<HostCommImpl> Split(int color, int key) = 0;
    virtual void Sync(bool blocking = false) = 0;
    virtual void Broadcast(void* data, int count, DataType dtype, int root,
                           copy_fn copy, ser_fn ser = nullptr, des_fn des = nullptr) = 0;
    virtual void AllGather(void* data, int count, DataType dtype,
                           copy_fn copy, ser_fn ser = nullptr, des_fn des = nullptr) = 0;
    virtual void AllReduce(void* data, int count, DataType dtype, RedOp red_op) = 0;
};

class HostComm {
public:
    HostComm() = default;
    HostComm(std::shared_ptr<HostCommImpl> impl);
    HostCommImpl* operator->() const noexcept;
    operator HostCommImpl*() const noexcept;
};

// Typed convenience functions
template<class T> void Broadcast(HostCommImpl* comm, T& value, int root);
template<class T> std::vector<T> AllGather(HostCommImpl* comm, const T& value);
template<class T> T AllReduce(HostCommImpl* comm, const T& value, RedOp red_op);

class HostGroupId {
public:
    virtual ~HostGroupId() = default;
    virtual void Initialize() = 0;
    virtual void Export(std::ostream& os) = 0;
    virtual void Import(std::istream& is) = 0;
    virtual HostComm CreateCommunicator(int n_ranks, int rank, int node_rank = 0) = 0;
};

std::unique_ptr<HostGroupId> CreateHostGroupId(const std::string& backend);

}  // namespace turbomind::comm

Import

#include "src/turbomind/comm/host_comm.h"

I/O Contract

Inputs

Name	Type	Required	Description
data	void*	Yes	Pointer to the data buffer for collective operations
count	int	Yes	Number of elements in the buffer
dtype	DataType	Yes	Data type of the elements
root	int	Broadcast only	Root rank for broadcast operations
red_op	RedOp	AllReduce only	Reduction operation (kSum, kMin, kMax)
copy	copy_fn	Yes	Function pointer for copying elements between buffers

Outputs

Name	Type	Description
AllGather result	std::vector<T>	Gathered values from all ranks
AllReduce result	T	Reduced value across all ranks

Usage Examples

#include "src/turbomind/comm/host_comm.h"

using namespace turbomind::comm;

// Create a host group and communicator
auto group_id = CreateHostGroupId("thread");
group_id->Initialize();
HostComm comm = group_id->CreateCommunicator(n_ranks, rank);

// Broadcast a config value from rank 0
int config_val = 128;
Broadcast(comm, config_val, 0);

// Gather values from all ranks
auto all_vals = AllGather(comm, my_value);

// Reduce with sum
int total = AllReduce(comm, local_count, RedOp::kSum);

Related Pages

• Environment:InternLM_Lmdeploy_CUDA_GPU_Runtime