Overview
RayServeMetrics is a Java enum that defines all Ray Serve metric names, descriptions, and tag constants, along with a conditional execution utility for safe metric recording.
Description
Each enum constant holds a metric name string and description string corresponding to a specific Ray Serve metric such as request counters, error counters, latency measurements, and processing gauges. The class provides a static execute() method that conditionally runs metric-recording lambdas only when metrics are enabled and Ray is initialized in non-local mode, preventing metric registration failures from crashing the application in test or local environments. Tag constants for handle, endpoint, deployment, route, replica, and application are defined as static final strings.
Usage
Use RayServeMetrics to reference metric names and descriptions when registering or recording metrics in Ray Serve components. Use the execute() method as a wrapper around metric recording code to ensure it only runs when the Ray metrics system is available.
Code Reference
Source Location
- Repository: Ray
- File:
java/serve/src/main/java/io/ray/serve/metrics/RayServeMetrics.java
Signature
public enum RayServeMetrics {
SERVE_HANDLE_REQUEST_COUNTER,
SERVE_NUM_ROUTER_REQUESTS,
SERVE_DEPLOYMENT_QUEUED_QUERIES,
SERVE_DEPLOYMENT_REQUEST_COUNTER,
SERVE_DEPLOYMENT_ERROR_COUNTER,
SERVE_DEPLOYMENT_REPLICA_STARTS,
SERVE_DEPLOYMENT_PROCESSING_LATENCY_MS,
SERVE_REPLICA_PROCESSING_QUERIES;
public static void execute(Runnable runnable)
public static void enable()
public static void disable()
public String getName()
public String getDescription()
}
Import
import io.ray.serve.metrics.RayServeMetrics;
I/O Contract
Enum Constants
| Constant |
Metric Name |
Description
|
SERVE_HANDLE_REQUEST_COUNTER |
serve_handle_request_counter |
The number of handle.remote() calls made on this handle
|
SERVE_NUM_ROUTER_REQUESTS |
serve_num_router_requests |
The number of requests processed by the router
|
SERVE_DEPLOYMENT_QUEUED_QUERIES |
serve_deployment_queued_queries |
Current number of queries waiting to be assigned to a replica
|
SERVE_DEPLOYMENT_REQUEST_COUNTER |
serve_deployment_request_counter |
The number of queries processed in this replica
|
SERVE_DEPLOYMENT_ERROR_COUNTER |
serve_deployment_error_counter |
The number of exceptions that have occurred in this replica
|
SERVE_DEPLOYMENT_REPLICA_STARTS |
serve_deployment_replica_starts |
The number of times this replica has been restarted due to failure
|
SERVE_DEPLOYMENT_PROCESSING_LATENCY_MS |
serve_deployment_processing_latency_ms |
The latency for queries to be processed
|
SERVE_REPLICA_PROCESSING_QUERIES |
serve_replica_processing_queries |
The current number of queries being processed
|
Tag Constants
| Constant |
Value |
Description
|
TAG_HANDLE |
"handle" |
Tag key for handle identification
|
TAG_ENDPOINT |
"endpoint" |
Tag key for endpoint identification
|
TAG_DEPLOYMENT |
"deployment" |
Tag key for deployment name
|
TAG_ROUTE |
"route" |
Tag key for route path
|
TAG_REPLICA |
"replica" |
Tag key for replica identification
|
TAG_APPLICATION |
"application" |
Tag key for application name
|
Static Methods
| Method |
Return Type |
Description
|
execute(Runnable) |
void |
Runs the given lambda only if metrics are enabled and Ray is initialized in non-local mode
|
enable() |
void |
Enables metric recording
|
disable() |
void |
Disables metric recording
|
Usage Examples
// Safely record a metric using the execute() wrapper
RayServeMetrics.execute(() -> {
Count requestCounter = new Count(
RayServeMetrics.SERVE_DEPLOYMENT_REQUEST_COUNTER.getName(),
RayServeMetrics.SERVE_DEPLOYMENT_REQUEST_COUNTER.getDescription(),
"1",
ImmutableMap.of(
RayServeMetrics.TAG_DEPLOYMENT, "my-deployment",
RayServeMetrics.TAG_REPLICA, "replica-001"
)
);
requestCounter.inc(1.0);
});
// Disable metrics in test environments
RayServeMetrics.disable();
Related Pages
Page Connections
Double-click a node to navigate. Hold to expand connections.
