Implementation:Tensorflow Serving Loader Harness Test

Knowledge Sources	Tensorflow_Serving
Domains	Testing, Core Management
Last Updated	2026-02-13 00:00 GMT

Overview

Test suite validating the LoaderHarness which manages a Loader through its complete lifecycle state machine.

Description

This test file exercises the LoaderHarness class, which wraps a Loader and manages state transitions through the servable lifecycle: New, LoadRequested, LoadApproved, Loading, Ready, UnloadRequested, Quiescing, Quiesced, and Disabled. The tests use MockLoader (both StrictMock and NiceMock variants) to verify correct invocation of Load/Unload operations and state transitions. Helper functions QuiesceAndUnload() and EnableDestruction() simplify common transition sequences. Tests cover initialization, each state transition, error handling (load errors, externally signalled errors, error callbacks), additional state, duplicate request handling, retry logic, and cancellation flows.

Usage

Run these tests to verify that the LoaderHarness correctly manages the loader state machine, enforces valid state transitions, handles errors and retries, and supports cancellation. Critical after changes to the harness, loader interface, or servable lifecycle management.

Code Reference

Source Location

Repository: Tensorflow_Serving
File: tensorflow_serving/core/loader_harness_test.cc
Lines: 1-386

Test Fixture

// Walks 'harness' through a sequence of transitions from kReady to kDisabled.
void QuiesceAndUnload(LoaderHarness* const harness) {
  TF_ASSERT_OK(harness->UnloadRequested());
  TF_ASSERT_OK(harness->StartQuiescing());
  TF_ASSERT_OK(harness->DoneQuiescing());
  TF_ASSERT_OK(harness->Unload());
}

// Makes it s.t. it's legal to destruct 'harness'.
void EnableDestruction(LoaderHarness* const harness) {
  harness->Error(
      errors::Unknown("Erroring servable on purpose for shutdown"));
}

// Typical test setup pattern:
TEST(LoaderHarnessTest, Init) {
  test_util::MockLoader* loader = new StrictMock<test_util::MockLoader>;
  LoaderHarness harness(ServableId{"test", 0},
                        std::unique_ptr<Loader>(loader));
  EXPECT_EQ((ServableId{"test", 0}), harness.id());
  EXPECT_EQ(LoaderHarness::State::kNew, harness.state());
}

Build Target

bazel test //tensorflow_serving/core:loader_harness_test

Test Coverage

Key Test Cases

Test Name	Category	Description
`Init`	Initialization	Verifies initial state is kNew with correct ServableId
`LoadRequested`	State Transition	Tests transition to kLoadRequested state
`Quiesce`	State Transition	Tests full quiesce sequence: StartQuiescing to DoneQuiescing
`Load`	State Transition	Verifies loading transitions to kReady state using a separate thread
`Unload`	State Transition	Verifies unloading transitions to kDisabled state
`UnloadRequested`	State Transition	Tests transition to kUnloadRequested state
`LoadApproved`	State Transition	Tests transition to kLoadApproved state
`LoadError`	Error Handling	Verifies error state when underlying loader fails to load
`ExternallySignalledError`	Error Handling	Tests externally triggered error transitions
`ExternallySignalledErrorWithCallback`	Error Handling	Tests error callback invoked on externally signalled error
`AdditionalState`	State	Verifies additional state can be attached and retrieved
`NoAdditionalState`	State	Verifies null returned when no additional state attached
`MultipleLoadRequestsOnlyFirstOneSucceeds`	Validation	Only the first LoadRequested call succeeds; subsequent calls fail
`MultipleUnloadRequestsOnlyFirstOneSucceeds`	Validation	Only the first UnloadRequested call succeeds; subsequent calls fail
`RetryOnLoadErrorFinallySucceeds`	Retry	Loader fails initially then succeeds on retry
`RetryOnLoadErrorFinallyFails`	Retry	Loader fails on all retries, transitions to error state
`RetryOnLoadErrorCancelledLoad`	Cancellation	Load retry cancelled mid-sequence
`UnloadDueToCancelledLoad`	Cancellation	Unload path after a cancelled load operation
`UnloadDueToNonRetriableError`	Error Handling	Unload triggered by non-retriable load error

Usage Examples

Test Pattern

TEST(LoaderHarnessTest, Load) {
  test_util::MockLoader* loader = new StrictMock<test_util::MockLoader>;
  const ServableId servable_id = {"test", 0};
  LoaderHarness harness(servable_id, std::unique_ptr<Loader>(loader));

  EXPECT_CALL(*loader, LoadWithMetadata(Loader::Metadata{servable_id}))
      .WillOnce(Return(absl::OkStatus()));
  {
    std::unique_ptr<Thread> test_thread(
        Env::Default()->StartThread(ThreadOptions(), "test", [&harness]() {
          TF_ASSERT_OK(harness.LoadRequested());
          TF_ASSERT_OK(harness.LoadApproved());
          EXPECT_TRUE(harness.Load().ok());
        }));
    // Deleting the thread here forces join
  }
  EXPECT_EQ(LoaderHarness::State::kReady, harness.state());

  EnableDestruction(&harness);
}

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment