Implementation:Tensorflow Serving Aspired Versions Manager Benchmark
| Knowledge Sources | |
|---|---|
| Domains | Testing, Core Management |
| Last Updated | 2026-02-13 00:00 GMT |
Overview
Benchmark suite measuring the read performance of AspiredVersionsManager under various concurrency and update patterns.
Description
This benchmark file measures read throughput of the AspiredVersionsManager class, simulating the common access pattern of high read rates and low update rates. The BenchmarkState class encapsulates all benchmark state, handling setup/teardown of the manager, concurrent read threads, and optional periodic update threads. It uses SimpleLoader to create servable versions and AvailabilityPreservingPolicy for version management. Benchmarks vary along two axes: whether reads do simulated work (beyond mutex contention) and whether concurrent updates are occurring.
Usage
Run these benchmarks to measure the read-path latency and throughput of AspiredVersionsManager, especially when evaluating changes to locking strategies, servable handle retrieval, or concurrent update handling. Use --benchmark_min_time=60.0 for stable results.
Code Reference
Source Location
- Repository: Tensorflow_Serving
- File:
tensorflow_serving/core/aspired_versions_manager_benchmark.cc - Lines: 1-398
Test Fixture
class BenchmarkState {
public:
BenchmarkState(const int interval_micros, const bool do_work)
: interval_micros_(interval_micros), do_work_(do_work) {
AspiredVersionsManager::Options options;
options.manage_state_interval_micros = -1;
options.aspired_version_policy.reset(new AvailabilityPreservingPolicy());
TF_CHECK_OK(AspiredVersionsManager::Create(std::move(options), &manager_));
}
void RunBenchmark(::testing::benchmark::State& state, int num_threads);
private:
void SetUp();
void TearDown();
void RunReads(int iters);
void RunUpdate();
void StartServing(int64_t loader_version);
int64_t GetLatestVersion(bool do_work);
absl::Notification all_read_threads_scheduled_;
std::unique_ptr<PeriodicFunction> update_thread_;
std::unique_ptr<AspiredVersionsManager> manager_;
const int interval_micros_;
bool do_work_;
};
Build Target
bazel run -c opt --dynamic_mode=off \
tensorflow_serving/core:aspired_versions_manager_benchmark -- \
--benchmarks=.
Test Coverage
Key Test Cases
| Test Name | Category | Description |
|---|---|---|
BM_Work_NoUpdates_Reads |
Read Performance | Measures read throughput with simulated work, no concurrent updates |
BM_Work_FrequentUpdates_Reads |
Read+Update | Measures read throughput with simulated work under frequent concurrent updates |
BM_NoWork_NoUpdates_Reads |
Mutex Contention | Measures pure mutex contention on reads with no work and no updates |
BM_NoWork_FrequentUpdates_Reads |
Mutex Contention | Measures mutex contention on reads with concurrent updates, no work |
BM_GetServableHandle |
Handle Retrieval | Benchmarks raw GetServableHandle call performance |
Usage Examples
Test Pattern
void BenchmarkState::StartServing(const int64_t loader_version) {
std::unique_ptr<Loader> loader(new SimpleLoader<int64_t>(
[loader_version](std::unique_ptr<int64_t>* const servable) {
servable->reset(new int64_t);
**servable = loader_version;
return absl::OkStatus();
},
SimpleLoader<int64_t>::EstimateNoResources()));
std::vector<ServableData<std::unique_ptr<Loader>>> versions;
versions.push_back({{kServableName, loader_version}, std::move(loader)});
manager_->GetAspiredVersionsCallback()(kServableName, std::move(versions));
// Invoke policy actions to load/quiesce/delete
test_util::AspiredVersionsManagerTestAccess(manager_.get())
.HandlePendingAspiredVersionsRequests();
test_util::AspiredVersionsManagerTestAccess(manager_.get())
.InvokePolicyAndExecuteAction();
}