Implementation:Ray project Ray Serve LongPollClientFactory

Knowledge Sources	Ray Ray Serve
Domains	Model_Serving, Distributed_Computing
Last Updated	2026-02-13 16:00 GMT

Overview

LongPollClientFactory is a singleton factory that manages the asynchronous long polling loop for receiving state updates from the Ray Serve controller.

Description

LongPollClientFactory maintains a global set of key-listener mappings, snapshot IDs, and cached object snapshots. On initialization, it creates a single-thread ScheduledExecutorService that periodically calls pollNext(), which sends a LongPollRequest with current snapshot IDs to the controller actor (supporting both Python and Java controllers). Responses are deserialized via namespace-specific deserializers, and processUpdate() dispatches updates to registered KeyListener callbacks. The factory handles timeout, actor disconnection, and task errors with appropriate logging and recovery.

Usage

Use LongPollClientFactory to register key-listener pairs that should be notified when state changes occur on the controller. Java Serve components such as routers and replicas use this factory to reactively receive deployment configuration updates, route table changes, and other state without polling the controller individually.

Code Reference

Source Location

Repository: Ray
File: java/serve/src/main/java/io/ray/serve/poll/LongPollClientFactory.java

Signature

public class LongPollClientFactory {

    public static void register(
        BaseActorHandle hostActor, Map<KeyType, KeyListener> keyListeners)

    public static synchronized void init(BaseActorHandle hostActor)

    public static synchronized void pollNext()

    public static void processUpdate(Map<KeyType, UpdatedObject> updates)

    public static void unregister(Set<KeyType> keys)

    public static synchronized void stop()

    public static boolean isInitialized()
}

Import

import io.ray.serve.poll.LongPollClientFactory;

I/O Contract

Public Static Fields

Field	Type	Description
`SNAPSHOT_IDS`	`Map<KeyType, Integer>`	Current snapshot IDs for each registered key (ConcurrentHashMap)
`OBJECT_SNAPSHOTS`	`Map<KeyType, Object>`	Cached latest object snapshots for each key (ConcurrentHashMap)
`DESERIALIZERS`	`Map<LongPollNamespace, Function<byte[], Object>>`	Namespace-specific deserializers for processing poll responses

Registered Deserializers

Namespace	Deserializer
`LongPollNamespace.ROUTE_TABLE`	`ServeProtoUtil::parseEndpointSet`
`LongPollNamespace.DEPLOYMENT_TARGETS`	Parses `DeploymentTargetInfo` from protobuf bytes

Key Methods

Method	Return Type	Description
`register(hostActor, keyListeners)`	`void`	Initializes the factory (if needed) and registers key-listener pairs, triggering an initial poll
`init(hostActor)`	`void`	Synchronized initialization: reads config from `ReplicaContext`, sets up the scheduled polling thread, and resolves the controller actor handle
`pollNext()`	`void`	Sends a `LongPollRequest` to the controller and processes the returned updates
`processUpdate(updates)`	`void`	Dispatches received updates to registered `KeyListener` callbacks and updates snapshot caches
`unregister(keys)`	`void`	Removes the specified keys from all internal maps
`stop()`	`void`	Shuts down the scheduled executor and clears all internal state
`isInitialized()`	`boolean`	Returns whether the factory has been initialized

Configuration Properties (from ReplicaContext)

Property	Default	Description
`LONG_POOL_CLIENT_ENABLED`	`true`	Whether the long poll client is enabled
`LONG_POOL_CLIENT_INTERVAL`	`10` seconds	Interval between poll cycles
`LONG_POOL_CLIENT_TIMEOUT_S`	`10` seconds	Timeout for each poll request

Usage Examples

// Register a listener for deployment target updates
Map<KeyType, KeyListener> listeners = new HashMap<>();
KeyType deploymentKey = new KeyType(
    LongPollNamespace.DEPLOYMENT_TARGETS, "my-deployment");
listeners.put(deploymentKey, updatedObject -> {
    DeploymentTargetInfo targetInfo = (DeploymentTargetInfo) updatedObject;
    // Handle the updated deployment target information
    System.out.println("Received deployment update: " + targetInfo);
});

// Register with the controller actor handle
LongPollClientFactory.register(controllerHandle, listeners);

// Later, unregister when no longer needed
LongPollClientFactory.unregister(Set.of(deploymentKey));

// Shut down the polling loop
LongPollClientFactory.stop();

Related Pages

Environment:Ray_project_Ray_Java_Build_Environment

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment