Implementation:Ray project Ray Serve RayServeReplica Interface

Knowledge Sources	Ray Ray Serve
Domains	Model_Serving, Distributed_Computing
Last Updated	2026-02-13 16:00 GMT

Overview

RayServeReplica is the Java interface defining the contract for a Ray Serve replica actor, specifying methods for request handling, reconfiguration, health checking, graceful shutdown, and request counting.

Description

The RayServeReplica interface declares handleRequest() as the primary entry point for processing incoming requests. It provides default implementations for four additional lifecycle methods: reconfigure() for updating deployment configuration, checkHealth() for health monitoring, prepareForShutdown() for graceful shutdown, and getNumOngoingRequests() for reporting current load. The default implementations return safe no-op values, allowing implementations to override only the methods they need.

Usage

Implement RayServeReplica in classes that serve as Ray Serve replica actors. Both RayServeReplicaImpl and RayServeWrappedReplica implement this interface, enabling the controller to interact with replicas uniformly regardless of the underlying implementation.

Code Reference

Source Location

• Repository: Ray
• File: java/serve/src/main/java/io/ray/serve/replica/RayServeReplica.java

Signature

public interface RayServeReplica {

    Object handleRequest(Object requestMetadata, Object requestArgs);

    default Object reconfigure(byte[] deploymentConfigBytes) {
        return null;
    }

    default boolean checkHealth() {
        return true;
    }

    default boolean prepareForShutdown() {
        return true;
    }

    default int getNumOngoingRequests() {
        return 0;
    }
}

Import

import io.ray.serve.replica.RayServeReplica;

I/O Contract

Interface Methods

Method	Parameters	Return Type	Default	Description
handleRequest	Object requestMetadata, Object requestArgs	Object	(none -- must implement)	Primary entry point for processing incoming requests
reconfigure	byte[] deploymentConfigBytes	Object	null	Called when deployment configuration is updated; receives serialized protobuf config bytes
checkHealth	(none)	boolean	true	Health check called periodically by the controller to verify replica is functioning
prepareForShutdown	(none)	boolean	true	Called before the replica is shut down to allow graceful cleanup
getNumOngoingRequests	(none)	int	0	Reports the number of requests currently being processed by this replica

Usage Examples

// Implement the RayServeReplica interface for a custom deployment
public class MyModelReplica implements RayServeReplica {

    private MyModel model;

    public MyModelReplica() {
        this.model = new MyModel();
    }

    @Override
    public Object handleRequest(Object requestMetadata, Object requestArgs) {
        // Process the incoming request
        return model.predict(requestArgs);
    }

    @Override
    public Object reconfigure(byte[] deploymentConfigBytes) {
        // Reload model configuration if needed
        DeploymentConfig config = DeploymentConfig.fromProtoBytes(deploymentConfigBytes);
        model.updateConfig(config.getUserConfig());
        return null;
    }

    @Override
    public boolean checkHealth() {
        return model.isLoaded();
    }
}

Related Pages

• Environment:Ray_project_Ray_Java_Build_Environment