Implementation:Tensorflow Serving HandleReloadConfigRequest

Knowledge Sources	TensorFlow Serving
Domains	Configuration, Operations
Last Updated	2026-02-13 17:00 GMT

Overview

Concrete tool for dynamically reloading model serving configuration at runtime via gRPC, provided by the ModelServiceImpl class.

Description

ModelServiceImpl::HandleReloadConfigRequest() is a gRPC handler that accepts a ReloadConfigRequest containing a new ModelServerConfig and applies it to the running ServerCore. It:

1. Captures current metric values (if metric_names specified in request)
2. Calls ServerCore::ReloadConfig() with the new configuration
3. Records metric deltas in the response

Internally, ReloadConfig() calls AddModelsViaModelConfigList() which diffs the current and new configs, creates/updates/removes filesystem sources and adapters, and updates version label mappings.

Usage

Send a ReloadConfigRequest gRPC to the ModelService endpoint. The server must have been started with --allow_version_labels_for_unavailable_models if labels reference versions not yet loaded.

Code Reference

Source Location

• Repository: tensorflow/serving
• File: tensorflow_serving/model_servers/model_service_impl.cc
• Lines: L44-82
• Header: tensorflow_serving/model_servers/model_service_impl.h L41-43

Signature

class ModelServiceImpl final : public ModelService::Service {
 public:
  explicit ModelServiceImpl(ServerCore* core);

  ::grpc::Status HandleReloadConfigRequest(
      ::grpc::ServerContext* context,
      const ReloadConfigRequest* request,
      ReloadConfigResponse* response
  );
};

Import

#include "tensorflow_serving/model_servers/model_service_impl.h"

I/O Contract

Inputs

Name	Type	Required	Description
request->config()	ModelServerConfig	Yes	Complete new model server configuration
request->metric_names()	repeated string	No	Names of metrics to track delta across reload
context	grpc::ServerContext	Yes	gRPC call context

Outputs

Name	Type	Description
response->status()	StatusProto	Success or failure status
response->metric()	repeated Metric	Metric value changes during reload

Usage Examples

gRPC Reload Config

import grpc
from tensorflow_serving.apis import model_service_pb2_grpc
from tensorflow_serving.apis import model_management_pb2
from tensorflow_serving.config import model_server_config_pb2

# 1. Create channel and stub
channel = grpc.insecure_channel('localhost:8500')
stub = model_service_pb2_grpc.ModelServiceStub(channel)

# 2. Build new config
config = model_server_config_pb2.ModelServerConfig()
model = config.model_config_list.config.add()
model.name = 'my_model'
model.base_path = '/models/my_model'
model.model_platform = 'tensorflow'

# 3. Send reload request
request = model_management_pb2.ReloadConfigRequest()
request.config.CopyFrom(config)
response = stub.HandleReloadConfigRequest(request)

Related Pages

Implements Principle

• Principle:Tensorflow_Serving_Dynamic_Configuration_Reload