Implementation:Tensorflow Serving Tfrt Http Rest Api Handler Test

Knowledge Sources	Tensorflow_Serving
Domains	Testing, Model Serving API, TFRT Runtime
Last Updated	2026-02-13 00:00 GMT

Overview

Test suite validating the TFRTHttpRestApiHandler component, which processes HTTP/REST requests for TensorFlow Serving using the TFRT (TensorFlow RunTime) execution backend.

Description

This test file exercises the TFRTHttpRestApiHandler class against a real ServerCore loaded with the "half_plus_two" test model using the TFRT saved model source adapter. The fixture TFRTHttpRestApiHandlerTest initializes the TFRT global runtime with 4 inter-op threads at the suite level, configures the server core with TfrtSavedModelSourceAdapterConfig, and constructs the handler with a 10-second timeout. Tests mirror those of the standard HttpRestApiHandler but exercise the TFRT code path.

Usage

Run these tests to validate that the TFRT-backed HTTP REST API layer correctly routes requests, handles JSON payloads, produces correct inference results, and returns appropriate error messages, all using the TFRT execution engine rather than the legacy TF session runtime.

Code Reference

Source Location

• Repository: Tensorflow_Serving
• File: tensorflow_serving/model_servers/tfrt_http_rest_api_handler_test.cc
• Lines: 1-542

Test Fixture

class TFRTHttpRestApiHandlerTest : public ::testing::Test {
 public:
  static void SetUpTestSuite() {
    tfrt_stub::SetGlobalRuntime(
        tfrt_stub::Runtime::Create(/*num_inter_op_threads=*/4));
    TF_ASSERT_OK(CreateServerCore(&server_core_));
    const int total = 1;
    int count = 0;
    while ((count = server_core_->ListAvailableServableIds().size()) < total) {
      absl::SleepFor(absl::Milliseconds(500));
    }
  }
  static void TearDownTestSuite() { server_core_.reset(); }

 protected:
  TFRTHttpRestApiHandlerTest()
      : handler_(/*timeout_in_ms=*/10000, GetServerCore()) {}

  static absl::Status CreateServerCore(
      std::unique_ptr<ServerCore>* server_core);
  string GetJsonErrorMsg(const string& json);
  ServerCore* GetServerCore() { return server_core_.get(); }

  TFRTHttpRestApiHandler handler_;
 private:
  static std::unique_ptr<ServerCore> server_core_;
};

Build Target

bazel test //tensorflow_serving/model_servers:tfrt_http_rest_api_handler_test

Test Coverage

Key Test Cases

Test Name	Category	Description
kPathRegex	Routing	Validates URL path regex matches expected model/version/verb patterns
UnsupportedApiCalls	Error Handling	Verifies unsupported API paths return appropriate errors
PredictModelNameVersionErrors	Validation	Tests error responses for invalid model names and versions
PredictRequestErrors	Validation	Tests error responses for malformed predict request bodies
Predict	Inference	Validates correct predict responses using the TFRT runtime
Regress	Inference	Validates correct regress responses using the TFRT runtime
Classify	Inference	Validates correct classify responses using the TFRT runtime
GetStatus	Status	Tests model status retrieval for specific versions and all versions
GetModelMetadata	Metadata	Tests model metadata retrieval including signature definitions

Usage Examples

Test Pattern

TEST_F(TFRTHttpRestApiHandlerTest, Predict) {
  HeaderList headers;
  string output;
  TF_ASSERT_OK(handler_.ProcessRequest(
      "POST",
      absl::StrCat("/v1/models/", kTestModelName, "/versions/",
                    kTestModelVersion1, ":predict"),
      R"({"instances": [1.0, 2.0, 5.0]})",
      &headers, &output));
  EXPECT_THAT(output, HasSubstr("predictions"));
}