Implementation:Ray project Ray Serve RayServeMetrics

Knowledge Sources	Ray Ray Serve
Domains	Model_Serving, Distributed_Computing
Last Updated	2026-02-13 16:00 GMT

Overview

RayServeMetrics is a Java enum that defines all Ray Serve metric names, descriptions, and tag constants, along with a conditional execution utility for safe metric recording.

Description

Each enum constant holds a metric name string and description string corresponding to a specific Ray Serve metric such as request counters, error counters, latency measurements, and processing gauges. The class provides a static execute() method that conditionally runs metric-recording lambdas only when metrics are enabled and Ray is initialized in non-local mode, preventing metric registration failures from crashing the application in test or local environments. Tag constants for handle, endpoint, deployment, route, replica, and application are defined as static final strings.

Usage

Use RayServeMetrics to reference metric names and descriptions when registering or recording metrics in Ray Serve components. Use the execute() method as a wrapper around metric recording code to ensure it only runs when the Ray metrics system is available.

Code Reference

Source Location

• Repository: Ray
• File: java/serve/src/main/java/io/ray/serve/metrics/RayServeMetrics.java

Signature

public enum RayServeMetrics {
    SERVE_HANDLE_REQUEST_COUNTER,
    SERVE_NUM_ROUTER_REQUESTS,
    SERVE_DEPLOYMENT_QUEUED_QUERIES,
    SERVE_DEPLOYMENT_REQUEST_COUNTER,
    SERVE_DEPLOYMENT_ERROR_COUNTER,
    SERVE_DEPLOYMENT_REPLICA_STARTS,
    SERVE_DEPLOYMENT_PROCESSING_LATENCY_MS,
    SERVE_REPLICA_PROCESSING_QUERIES;

    public static void execute(Runnable runnable)
    public static void enable()
    public static void disable()
    public String getName()
    public String getDescription()
}

Import

import io.ray.serve.metrics.RayServeMetrics;

I/O Contract

Enum Constants

Constant	Metric Name	Description
SERVE_HANDLE_REQUEST_COUNTER	serve_handle_request_counter	The number of handle.remote() calls made on this handle
SERVE_NUM_ROUTER_REQUESTS	serve_num_router_requests	The number of requests processed by the router
SERVE_DEPLOYMENT_QUEUED_QUERIES	serve_deployment_queued_queries	Current number of queries waiting to be assigned to a replica
SERVE_DEPLOYMENT_REQUEST_COUNTER	serve_deployment_request_counter	The number of queries processed in this replica
SERVE_DEPLOYMENT_ERROR_COUNTER	serve_deployment_error_counter	The number of exceptions that have occurred in this replica
SERVE_DEPLOYMENT_REPLICA_STARTS	serve_deployment_replica_starts	The number of times this replica has been restarted due to failure
SERVE_DEPLOYMENT_PROCESSING_LATENCY_MS	serve_deployment_processing_latency_ms	The latency for queries to be processed
SERVE_REPLICA_PROCESSING_QUERIES	serve_replica_processing_queries	The current number of queries being processed

Tag Constants

Constant	Value	Description
TAG_HANDLE	"handle"	Tag key for handle identification
TAG_ENDPOINT	"endpoint"	Tag key for endpoint identification
TAG_DEPLOYMENT	"deployment"	Tag key for deployment name
TAG_ROUTE	"route"	Tag key for route path
TAG_REPLICA	"replica"	Tag key for replica identification
TAG_APPLICATION	"application"	Tag key for application name

Static Methods

Method	Return Type	Description
execute(Runnable)	void	Runs the given lambda only if metrics are enabled and Ray is initialized in non-local mode
enable()	void	Enables metric recording
disable()	void	Disables metric recording

Usage Examples

// Safely record a metric using the execute() wrapper
RayServeMetrics.execute(() -> {
    Count requestCounter = new Count(
        RayServeMetrics.SERVE_DEPLOYMENT_REQUEST_COUNTER.getName(),
        RayServeMetrics.SERVE_DEPLOYMENT_REQUEST_COUNTER.getDescription(),
        "1",
        ImmutableMap.of(
            RayServeMetrics.TAG_DEPLOYMENT, "my-deployment",
            RayServeMetrics.TAG_REPLICA, "replica-001"
        )
    );
    requestCounter.inc(1.0);
});

// Disable metrics in test environments
RayServeMetrics.disable();

Related Pages

• Environment:Ray_project_Ray_Java_Build_Environment