Implementation:Tensorflow Serving dynamic source router h

Knowledge Sources	Tensorflow_Serving
Domains	Model Serving, Core Framework
Last Updated	2026-02-13 00:00 GMT

Overview

DynamicSourceRouter is a SourceRouter with a fixed number of output ports and a dynamically reconfigurable map from servable name to output port.

Description

DynamicSourceRouter<T> extends SourceRouter<T> and provides a routing mechanism based on an explicit name-to-port mapping that can be updated at runtime. The router has N output ports, where the first N-1 ports are assignable via the route map and the Nth (last) port is reserved as the default route for servables not found in the map.

The route map is of type std::map<string, int> mapping servable names to port numbers. It is protected by a mutex for thread-safe reads and updates. Routes are validated: port numbers must be in range [0, N-2] (since port N-1 is reserved for the default), and out-of-range values produce an error status.

The router is created via the static Create() factory method, which validates the initial routes before construction. Routes can be updated at runtime via UpdateRoutes(), and the current routes can be read via GetRoutes().

Usage

Use DynamicSourceRouter when you need to route servable versions to different downstream targets based on their names, and the routing configuration may change at runtime (e.g., when new model types are added to the server without a restart).

Code Reference

Source Location

• Repository: Tensorflow_Serving
• File: tensorflow_serving/core/dynamic_source_router.h
• Lines: 1-146

Signature

template <typename T>
class DynamicSourceRouter final : public SourceRouter<T> {
 public:
  using Routes = std::map<string, int>;

  static Status Create(int num_output_ports, const Routes& routes,
                       std::unique_ptr<DynamicSourceRouter<T>>* result);
  ~DynamicSourceRouter() override;

  Routes GetRoutes() const;
  Status UpdateRoutes(const Routes& routes);

 protected:
  int num_output_ports() const override;
  int Route(const StringPiece servable_name,
            const std::vector<ServableData<T>>& versions) override;
};

Import

#include "tensorflow_serving/core/dynamic_source_router.h"

I/O Contract

Inputs

Name	Type	Required	Description
num_output_ports	int	Yes	Total number of output ports (must be >= 2; last port is the default)
routes	Routes (std::map<string, int>)	Yes	Initial mapping from servable name to output port number [0, N-2]

Outputs

Name	Type	Description
Create()	Status	OK if creation succeeded; InvalidArgument if routes are invalid
GetRoutes()	Routes	Current snapshot of the route map
UpdateRoutes()	Status	OK if update succeeded; InvalidArgument if new routes are invalid
Route()	int	The port number for the given servable name, or last port (default) if not in map

Usage Examples

Creating and Using a DynamicSourceRouter

#include "tensorflow_serving/core/dynamic_source_router.h"

using namespace tensorflow::serving;

DynamicSourceRouter<StoragePath>::Routes routes = {
    {"model_a", 0}, {"model_b", 1}
};

std::unique_ptr<DynamicSourceRouter<StoragePath>> router;
TF_CHECK_OK(DynamicSourceRouter<StoragePath>::Create(
    3, routes, &router));  // 3 ports: 0, 1, and 2 (default)

auto output_ports = router->GetOutputPorts();
// output_ports[0] receives model_a
// output_ports[1] receives model_b
// output_ports[2] receives everything else (default)

Updating Routes at Runtime

DynamicSourceRouter<StoragePath>::Routes new_routes = {
    {"model_a", 0}, {"model_c", 1}
};
TF_CHECK_OK(router->UpdateRoutes(new_routes));
// model_b now goes to the default port, model_c goes to port 1

Related Pages

• Principle:Tensorflow_Serving_Source_Routing