Implementation:Ray project Ray Serve LongPollClientFactory

Knowledge Sources	Ray Ray Serve
Domains	Model_Serving, Distributed_Computing
Last Updated	2026-02-13 16:00 GMT

Overview

LongPollClientFactory is a singleton factory that manages the asynchronous long polling loop for receiving state updates from the Ray Serve controller.

Description

LongPollClientFactory maintains a global set of key-listener mappings, snapshot IDs, and cached object snapshots. On initialization, it creates a single-thread ScheduledExecutorService that periodically calls pollNext(), which sends a LongPollRequest with current snapshot IDs to the controller actor (supporting both Python and Java controllers). Responses are deserialized via namespace-specific deserializers, and processUpdate() dispatches updates to registered KeyListener callbacks. The factory handles timeout, actor disconnection, and task errors with appropriate logging and recovery.

Usage

Use LongPollClientFactory to register key-listener pairs that should be notified when state changes occur on the controller. Java Serve components such as routers and replicas use this factory to reactively receive deployment configuration updates, route table changes, and other state without polling the controller individually.

Code Reference

Source Location

• Repository: Ray
• File: java/serve/src/main/java/io/ray/serve/poll/LongPollClientFactory.java

Signature

public class LongPollClientFactory {

    public static void register(
        BaseActorHandle hostActor, Map<KeyType, KeyListener> keyListeners)

    public static synchronized void init(BaseActorHandle hostActor)

    public static synchronized void pollNext()

    public static void processUpdate(Map<KeyType, UpdatedObject> updates)

    public static void unregister(Set<KeyType> keys)

    public static synchronized void stop()

    public static boolean isInitialized()
}

Import

import io.ray.serve.poll.LongPollClientFactory;

I/O Contract

Public Static Fields

Field	Type	Description
SNAPSHOT_IDS	Map<KeyType, Integer>	Current snapshot IDs for each registered key (ConcurrentHashMap)
OBJECT_SNAPSHOTS	Map<KeyType, Object>	Cached latest object snapshots for each key (ConcurrentHashMap)
DESERIALIZERS	Map<LongPollNamespace, Function<byte[], Object>>	Namespace-specific deserializers for processing poll responses

Registered Deserializers

Namespace	Deserializer
LongPollNamespace.ROUTE_TABLE	ServeProtoUtil::parseEndpointSet
LongPollNamespace.DEPLOYMENT_TARGETS	Parses DeploymentTargetInfo from protobuf bytes

Key Methods

Method	Return Type	Description
register(hostActor, keyListeners)	void	Initializes the factory (if needed) and registers key-listener pairs, triggering an initial poll
init(hostActor)	void	Synchronized initialization: reads config from ReplicaContext, sets up the scheduled polling thread, and resolves the controller actor handle
pollNext()	void	Sends a LongPollRequest to the controller and processes the returned updates
processUpdate(updates)	void	Dispatches received updates to registered KeyListener callbacks and updates snapshot caches
unregister(keys)	void	Removes the specified keys from all internal maps
stop()	void	Shuts down the scheduled executor and clears all internal state
isInitialized()	boolean	Returns whether the factory has been initialized

Configuration Properties (from ReplicaContext)

Property	Default	Description
LONG_POOL_CLIENT_ENABLED	true	Whether the long poll client is enabled
LONG_POOL_CLIENT_INTERVAL	10 seconds	Interval between poll cycles
LONG_POOL_CLIENT_TIMEOUT_S	10 seconds	Timeout for each poll request

Usage Examples

// Register a listener for deployment target updates
Map<KeyType, KeyListener> listeners = new HashMap<>();
KeyType deploymentKey = new KeyType(
    LongPollNamespace.DEPLOYMENT_TARGETS, "my-deployment");
listeners.put(deploymentKey, updatedObject -> {
    DeploymentTargetInfo targetInfo = (DeploymentTargetInfo) updatedObject;
    // Handle the updated deployment target information
    System.out.println("Received deployment update: " + targetInfo);
});

// Register with the controller actor handle
LongPollClientFactory.register(controllerHandle, listeners);

// Later, unregister when no longer needed
LongPollClientFactory.unregister(Set.of(deploymentKey));

// Shut down the polling loop
LongPollClientFactory.stop();

Related Pages

• Environment:Ray_project_Ray_Java_Build_Environment