Implementation:Tensorflow Serving Basic Manager Test

Knowledge Sources	Tensorflow_Serving
Domains	Testing, Core Management
Last Updated	2026-02-13 00:00 GMT

Overview

Test suite validating the BasicManager which provides low-level servable lifecycle management with explicit load/unload control.

Description

This test file exercises the BasicManager class, the foundational manager that handles explicit servable loading, unloading, and handle retrieval. The BasicManagerTest fixture is parameterized on thread pool sizes (inline, load-only, unload-only, and both) to test all execution modes. Additional fixture classes include SetNumLoadThreadsBasicManagerTest (testing dynamic thread pool resizing), FlushFileSystemCachesTest (testing filesystem cache flushing on load), ResourceConstrainedBasicManagerTest (testing resource-aware loading), and EstimateResourcesRetriedTest (testing resource estimation retries). The test uses FakeLoader, MockLoader, EventBus, and ServableStateMonitor.

Usage

Run these tests to verify correct servable management at the BasicManager level: manage/load/unload operations, handle retrieval, event bus notifications, concurrent operations, resource constraints, retry logic, and thread pool management. Critical after changes to the basic manager or its resource tracking.

Code Reference

Source Location

• Repository: Tensorflow_Serving
• File: tensorflow_serving/core/basic_manager_test.cc
• Lines: 1-1748

Test Fixture

struct ThreadPoolSizes {
  uint64_t num_load_threads;
  uint64_t num_unload_threads;
};

class BasicManagerTest : public ::testing::TestWithParam<ThreadPoolSizes> {
 protected:
  BasicManagerTest()
      : thread_pool_sizes_(GetParam()),
        servable_event_bus_(EventBus<ServableState>::CreateEventBus()),
        servable_state_monitor_(servable_event_bus_.get()) {
    BasicManager::Options options;
    options.num_load_threads = thread_pool_sizes_.num_load_threads;
    options.num_unload_threads = thread_pool_sizes_.num_unload_threads;
    options.servable_event_bus = servable_event_bus_.get();
    options.max_num_load_retries = 10;
    options.load_retry_interval_micros = 0;
    TF_CHECK_OK(BasicManager::Create(std::move(options), &basic_manager_));
  }

  void SetUp() override {
    // Loads two servable streams (kServableName, kServableName2)
    // each with versions 1 and 2
  }

  ThreadPoolSizes thread_pool_sizes_;
  std::shared_ptr<EventBus<ServableState>> servable_event_bus_;
  ServableStateMonitor servable_state_monitor_;
  std::unique_ptr<BasicManager> basic_manager_;
};

INSTANTIATE_TEST_CASE_P(
    WithOrWithoutThreadPools, BasicManagerTest,
    ::testing::Values(
        ThreadPoolSizes{0, 0},  // without load or unload threadpools
        ThreadPoolSizes{2, 0},  // with just a load threadpool
        ThreadPoolSizes{0, 2},  // with just an unload threadpool
        ThreadPoolSizes{4, 4}   // with load and unload threadpools
    ));

Build Target

bazel test //tensorflow_serving/core:basic_manager_test

Test Coverage

Key Test Cases

Test Name	Category	Description
ServableHandleNotFoundMissingLoaderName	Error Handling	NOT_FOUND for non-existent servable name
ServableHandleNotFoundMissingVersion	Error Handling	NOT_FOUND for non-existent version
ServableHandleEarliest	Handle Retrieval	Retrieves earliest available version handle
ServableHandleLatest	Handle Retrieval	Retrieves latest available version handle
AlreadyManagedError	Validation	Error when managing an already-managed servable
StopManagingUnknownId	Validation	Error when stopping management of unknown servable
StopManagingActiveServable	Validation	Error when stopping management of active servable
StopManagingDisabledServable	Lifecycle	Successfully stops managing a disabled servable
DontStopManagingOnError	Error Handling	Servable remains managed even after load error
UpdateServingMapServableHandleLatest	Handle Retrieval	Verifies serving map updates after load/unload
ListAvailableServableIds	Listing	Lists all available servable IDs
GetAvailableServableHandles	Handle Retrieval	Bulk retrieval of available handles
GetManagedServableNames	Listing	Lists all managed servable names
GetManagedServableStateSnapshot	State	Retrieves state snapshots for managed servables
MultipleManageCallsUsesFirstServable	Validation	Ensures first manage call wins
ErroneousServable	Error Handling	Tests handling of servables with errors
DestructOnNonServingThread	Lifecycle	Safe destruction from non-serving thread
AdditionalState	State	Tests additional state attached to servable harness
ConcurrentLoadsOnlyOneSucceeds	Concurrency	Only one concurrent load for same servable succeeds
ConcurrentUnloadsOnlyOneSucceeds	Concurrency	Only one concurrent unload for same servable succeeds
RetryOnLoadErrorFinallySucceeds	Retry	Load retry eventually succeeds
RetryOnLoadErrorFinallyFails	Retry	Load retry eventually fails
RetryOnLoadErrorCancelledLoad	Retry	Load retry cancelled mid-retry
PreLoadHook	Hooks	Tests pre-load hook execution before servable loading
ConcurrentLoads	Resource Constraints	Resource-constrained concurrent loads
InsufficientResources	Resource Constraints	Rejects loads exceeding available resources
ResourcesReleasedAfterUnload	Resource Constraints	Resources freed after unloading
ThreadPoolSwapped	Thread Pool	Dynamic thread pool resizing

Usage Examples

Test Pattern

// Creates a ServableData around a FakeLoader.
ServableData<std::unique_ptr<Loader>> CreateServable(
    const ServableId& id,
    const absl::Status load_status = absl::OkStatus()) {
  std::unique_ptr<Loader> loader(new FakeLoader(id.version, load_status));
  return CreateServableData(id, std::move(loader));
}

TEST_P(BasicManagerTest, ServableHandleNotFoundMissingLoaderName) {
  ServableHandle<int64_t> handle;
  const absl::Status status = basic_manager_->GetServableHandle(
      ServableRequest::Latest(absl::StrCat(kServableName, "missing")),
      &handle);
  ASSERT_FALSE(status.ok()) << status;
  EXPECT_EQ(error::NOT_FOUND, status.code());
}

Related Pages

• Implementation:Tensorflow_Serving_Aspired_Versions_Manager_Test
• Implementation:Tensorflow_Serving_Loader_Harness_Test