Implementation:Tensorflow Serving Tfrt Saved Model Factory

Knowledge Sources	Tensorflow_Serving
Domains	Model Serving, Model Loading
Last Updated	2026-02-13 00:00 GMT

Overview

Factory class that creates TFRT SavedModel instances from export paths, handling configuration, batching, resource estimation, warmup, and thread pool management.

Description

TfrtSavedModelFactory is a thread-safe factory responsible for loading and configuring TFRT SavedModels. The class manages the complete lifecycle of model creation: reading the MetaGraphDef, applying graph rewrites via GraphRewriter, setting up compile options (device target, grappler, TPU/GPU settings, lazy loading, MLRT), wrapping the model for batching when configured, running warmup requests, and optionally freezing the model after initialization.

⚠️ DEPRECATION WARNING: The overload of CreateTfrtSavedModelWithMetadata that returns a raw tfrt_stub::SavedModel is marked ABSL_DEPRECATED at line 104 of tfrt_saved_model_factory.h. Use the Servable overload instead. See Heuristic:Tensorflow_Serving_Warning_Deprecated_CreateTfrtSavedModel_Raw for migration guidance.

The factory supports two creation paths: one producing a raw tfrt_stub::SavedModel (deprecated) and another producing a Servable (preferred). The Servable path creates a TfrtSavedModelServable that wraps the TFRT SavedModel with thread pool factory support and request recording.

Key capabilities include:

CreateCommonSavedModelOptions: Builds TFRT SavedModel options from TfrtSavedModelConfig, setting compile options for device targeting (CPU, GPU, TPU), grappler optimization, lazy loading thresholds, batch options, and model metadata.
WrapSavedModelForBatching: Wraps the SavedModel with a shared batch scheduler using configurable queue options and allowed batch sizes.
EstimateResourceRequirement: Estimates RAM requirements from the model path.
TfrtSavedModelFactoryRegistry: A global registry allowing custom factory creation functions to be registered, enabling subclass overrides.
CreateBatchSchedulerFromConfig / CreateThreadPoolFactoryFromConfig: Helper functions to create batch schedulers and thread pool factories from configuration.

Usage

Use this factory when loading TFRT SavedModels in a serving environment. It is typically instantiated via the TfrtSavedModelFactoryRegistry and used by the TfrtSavedModelSourceAdapter to create servables for the model management pipeline.

Code Reference

Source Location

Repository: Tensorflow_Serving
Files:
- tensorflow_serving/servables/tensorflow/tfrt_saved_model_factory.h (lines 1-209)
- tensorflow_serving/servables/tensorflow/tfrt_saved_model_factory.cc (lines 1-420)

Signature

class TfrtSavedModelFactory {
 public:
  using Batcher = SharedBatchScheduler<SavedModelBatchingTask>;

  TfrtSavedModelFactory(const TfrtSavedModelConfig& config,
                        std::shared_ptr<Batcher> batch_scheduler,
                        std::unique_ptr<ThreadPoolFactory> thread_pool_factory);

  static absl::Status Create(const TfrtSavedModelConfig& config,
                              std::unique_ptr<TfrtSavedModelFactory>* factory);

  virtual absl::Status CreateTfrtSavedModelWithMetadata(
      const Loader::Metadata& metadata, const string& path,
      std::unique_ptr<Servable>* servable);

  absl::Status EstimateResourceRequirement(const string& path,
                                           ResourceAllocation* estimate) const;
};

Import

#include "tensorflow_serving/servables/tensorflow/tfrt_saved_model_factory.h"

I/O Contract

Inputs

Name	Type	Required	Description
config	`TfrtSavedModelConfig`	Yes	Configuration protobuf with batching parameters, saved_model_tags, device targeting, compile options
metadata	`Loader::Metadata`	Yes	Servable ID (name and version) for the model being loaded
path	`string`	Yes	Filesystem path to the SavedModel export directory

Outputs

Name	Type	Description
servable	`std::unique_ptr<Servable>*`	Created TfrtSavedModelServable wrapping the loaded and configured TFRT SavedModel
estimate	`ResourceAllocation*`	Estimated resource requirements (RAM) for the model
return	`Status`	OK on success; error status with details on failure

Usage Examples

Creating a Factory and Loading a Model

TfrtSavedModelConfig config;
// Configure batching, tags, device targeting, etc.
std::unique_ptr<TfrtSavedModelFactory> factory;
TF_RETURN_IF_ERROR(TfrtSavedModelFactory::Create(config, &factory));

Loader::Metadata metadata;
metadata.servable_id = {"my_model", 1};
std::unique_ptr<Servable> servable;
TF_RETURN_IF_ERROR(factory->CreateTfrtSavedModelWithMetadata(
    metadata, "/path/to/model", &servable));

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment