Implementation:Ray project Ray AutoscalingConfig Setup

Knowledge Sources	Ray Ray Serve Autoscaling
Domains	Model_Serving, Auto_Scaling
Last Updated	2026-02-13 17:00 GMT

Overview

Wrapper documentation for the AutoscalingConfig configuration class used to set up automatic replica scaling in Ray Serve Java deployments.

Description

AutoscalingConfig is a configuration POJO that specifies autoscaling behavior for a deployment. It is passed to DeploymentCreator.setAutoscalingConfig() during deployment configuration. The actual scaling decisions are made by the Python Serve controller based on metrics collected from replica actors (request count, error count, processing latency, ongoing requests).

Usage

Create an AutoscalingConfig instance, set the desired parameters, and pass it to the deployment creator before binding.

Code Reference

Source Location

Repository: ray-project/ray
File: java/serve/src/main/java/io/ray/serve/config/AutoscalingConfig.java (L5-97)

Signature

public class AutoscalingConfig {
    private int minReplicas = 1;
    private int maxReplicas = 1;
    private int targetOngoingRequests = 1;
    private double metricsIntervalS = 10.0;
    private double lookBackPeriodS = 30.0;
    private double smoothingFactor = 1.0;
    private double downscaleDelayS = 600.0;
    private double upscaleDelayS = 30.0;

    // Getters and setters for all fields
    // toProto() for protobuf serialization
}

Import

import io.ray.serve.config.AutoscalingConfig;

I/O Contract

Inputs

Name	Type	Required	Description
minReplicas	int	No	Minimum number of replicas (default: 1)
maxReplicas	int	No	Maximum number of replicas (default: 1)
targetOngoingRequests	int	No	Target requests per replica (default: 1)
metricsIntervalS	double	No	Metrics scraping interval in seconds (default: 10.0)
lookBackPeriodS	double	No	Time window for averaging metrics (default: 30.0)
smoothingFactor	double	No	Multiplicative gain factor (default: 1.0)
downscaleDelayS	double	No	Wait time before scaling down in seconds (default: 600.0)
upscaleDelayS	double	No	Wait time before scaling up in seconds (default: 30.0)

Outputs

Name	Type	Description
config	AutoscalingConfig	Configuration object to pass to DeploymentCreator

Usage Examples

Configure Autoscaling

import io.ray.serve.config.AutoscalingConfig;
import io.ray.serve.api.Serve;
import io.ray.serve.deployment.Application;

AutoscalingConfig autoscaling = new AutoscalingConfig();
autoscaling.setMinReplicas(1);
autoscaling.setMaxReplicas(10);
autoscaling.setTargetOngoingRequests(5);
autoscaling.setUpscaleDelayS(15.0);
autoscaling.setDownscaleDelayS(300.0);

Application app = Serve.deployment()
    .setName("auto-scaling-service")
    .setDeploymentDef("com.example.Handler")
    .setAutoscalingConfig(autoscaling)
    .bind();

Related Pages

Implements Principle

Principle:Ray_project_Ray_Autoscaling_And_Monitoring

Requires Environment

Environment:Ray_project_Ray_Java_Build_Environment

Uses Heuristic

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment