Implementation:Tensorflow Serving Tfrt Classifier

Knowledge Sources	Tensorflow_Serving
Domains	Model Serving, Classification
Last Updated	2026-02-13 00:00 GMT

Overview

Provides TFRT-based classification inference pipeline that validates, executes, and post-processes classification requests against a TFRT SavedModel.

Description

The TFRT Classifier module implements classification inference using the TFRT (TensorFlow RunTime) SavedModel backend. It consists of three primary functions forming a pre-process, execute, post-process pipeline. PreProcessClassification validates that the function metadata contains exactly one input tensor and one or two output tensors (classes and/or scores), ensuring they match the expected classification signature constants. PostProcessClassificationResult validates the shapes and data types of the output tensors (classes must be DT_STRING with shape [batch_size, num_classes], scores must be DT_FLOAT with the same shape) and converts them into a structured ClassificationResult protobuf. RunClassify orchestrates the full pipeline: it resolves the function name from the request's signature, pre-processes, serializes the input examples into a tensor, invokes the TFRT SavedModel, records runtime latency metrics, and post-processes the results.

Usage

Use this module when serving classification requests through the TFRT runtime. It is invoked by the TfrtSavedModelServable's Classify method and during TFRT model warmup. This is the TFRT counterpart to the standard TensorFlow classifier and should be preferred when models are loaded through the TFRT SavedModel infrastructure.

Code Reference

Source Location

• Repository: Tensorflow_Serving
• Files:
  • tensorflow_serving/servables/tensorflow/tfrt_classifier.h (lines 1-51)
  • tensorflow_serving/servables/tensorflow/tfrt_classifier.cc (lines 1-221)

Signature

// Validate function's input and output.
Status PreProcessClassification(
    const tfrt::FunctionMetadata& function_metadata);

// Validate all results and populate a ClassificationResult.
Status PostProcessClassificationResult(
    int num_examples, const std::vector<string>& output_names,
    const std::vector<Tensor>& output_tensors, ClassificationResult* result);

// Run Classification.
Status RunClassify(const tfrt::SavedModel::RunOptions& run_options,
                   const absl::optional<int64_t>& servable_version,
                   tfrt::SavedModel* saved_model,
                   const ClassificationRequest& request,
                   ClassificationResponse* response);

Import

#include "tensorflow_serving/servables/tensorflow/tfrt_classifier.h"

I/O Contract

Inputs

Name	Type	Required	Description
run_options	tfrt::SavedModel::RunOptions	Yes	Runtime options for TFRT execution including deadline and validation flags
servable_version	absl::optional<int64_t>	No	Model version to set in the response ModelSpec
saved_model	tfrt::SavedModel*	Yes	Pointer to the loaded TFRT SavedModel instance
request	ClassificationRequest	Yes	Classification request containing model_spec, input examples, and optional signature name

Outputs

Name	Type	Description
response	ClassificationResponse*	Populated response containing model_spec and ClassificationResult with per-example class labels and scores
return	Status	OK on success; InvalidArgument, FailedPrecondition, or Internal on failure

Usage Examples

Running Classification via TfrtSavedModelServable

tfrt::SavedModel::RunOptions run_options;
ClassificationRequest request;
ClassificationResponse response;
// Set up request with model_spec and input examples
Status status = RunClassify(run_options, /*servable_version=*/1,
                            saved_model, request, &response);
if (status.ok()) {
  // Process response.result().classifications()
}

Related Pages

• Principle:Tensorflow_Serving_TFRT_Inference