Implementation:Ray project Ray DeploymentHandle Remote

Knowledge Sources	Ray Ray Serve
Domains	Model_Serving, Load_Balancing
Last Updated	2026-02-13 17:00 GMT

Overview

Concrete tool for sending requests to Ray Serve deployments and retrieving results provided by the Ray Java Serve SDK.

Description

DeploymentHandle.remote() builds RequestMetadata (protobuf), delegates to Router.assignRequest() which routes to a random replica via ReplicaSet.tryAssignReplica(). The replica actor executes RayServeReplicaImpl.handleRequest() → invokeSingle() via reflection, and the result is returned as a DeploymentResponse wrapping an ObjectRef.

.method(name) sets a specific method to invoke on the deployment class (default: call).

Usage

Use handle.remote(args...) to send a request and get a DeploymentResponse. Call .result() on the response to block and get the value.

Code Reference

Source Location

Repository: ray-project/ray
File: java/serve/src/main/java/io/ray/serve/handle/DeploymentHandle.java (L80-123)
File: java/serve/src/main/java/io/ray/serve/handle/DeploymentResponse.java (L18-30)
File: java/serve/src/main/java/io/ray/serve/router/Router.java (L78-89)
File: java/serve/src/main/java/io/ray/serve/router/ReplicaSet.java (L90-126)

Signature

// Send request
public DeploymentResponse remote(Object... parameters)

// Set method name
public DeploymentHandle method(String methodName)

// Get result from response
public Object result()                    // blocking
public Object result(long timeoutMs)      // with timeout

Import

import io.ray.serve.handle.DeploymentHandle;
import io.ray.serve.handle.DeploymentResponse;

I/O Contract

Inputs

Name	Type	Required	Description
parameters	Object...	No	Request arguments passed to the deployment handler method
methodName	String	No	Method to invoke on the handler class (default: "call")

Outputs

Name	Type	Description
response	DeploymentResponse	Wraps an ObjectRef; call .result() to get the value

Usage Examples

Send Requests

import io.ray.serve.handle.DeploymentHandle;

// Assuming handle obtained from Serve.run()
DeploymentHandle handle = Serve.run(app);

// Simple request
Object result = handle.remote("input").result();

// Invoke specific method
Object result2 = handle.method("predict").remote("data").result();

// With timeout
Object result3 = handle.remote("input").result(5000);

Parallel Requests

import io.ray.serve.handle.DeploymentHandle;
import io.ray.serve.handle.DeploymentResponse;
import java.util.ArrayList;
import java.util.List;

// Submit multiple requests in parallel
List<DeploymentResponse> responses = new ArrayList<>();
for (int i = 0; i < 10; i++) {
    responses.add(handle.remote("request-" + i));
}

// Collect results
for (DeploymentResponse resp : responses) {
    System.out.println(resp.result());
}

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment