Implementation:Tensorflow Serving AspiredVersionsManager
| Knowledge Sources | |
|---|---|
| Domains | Version_Management, Reliability |
| Last Updated | 2026-02-13 17:00 GMT |
Overview
Concrete tool for orchestrating model version load/unload transitions using policy-driven decision making, provided by the TensorFlow Serving core framework.
Description
AspiredVersionsManager is the high-level manager that implements both the Manager and Target<Loader> interfaces. It receives aspired-versions callbacks from source adapters, compares them against currently loaded versions, and uses an AspiredVersionPolicy to determine load/unload ordering.
Key internal methods:
- EnqueueAspiredVersionsRequest() — Receives new aspired versions from sources
- ProcessAspiredVersionsRequest() — Processes queued requests, updating the aspired set
- GetNextAction() — Queries the policy for the next load/unload action
- PerformAction() — Executes the action via the underlying BasicManager
- InvokePolicyAndExecuteAction() — Combines GetNextAction and PerformAction in a loop
A periodic thread (configurable interval, default 100ms) calls InvokePolicyAndExecuteAction() to drive state transitions.
Usage
AspiredVersionsManager is created by ServerCore during initialization. Users don't interact with it directly; they influence its behavior through version policies in ModelConfig and the aspired_version_policy in ServerCore::Options.
Code Reference
Source Location
- Repository: tensorflow/serving
- File: tensorflow_serving/core/aspired_versions_manager.cc
- Lines: L158-193 (Create), L264-282 (EnqueueAspiredVersionsRequest), L386-435 (GetNextAction + PerformAction)
- Header: tensorflow_serving/core/aspired_versions_manager.h L85-346
- Policies: tensorflow_serving/core/availability_preserving_policy.cc L47-88, resource_preserving_policy.cc L24-61
Signature
class AspiredVersionsManager : public Manager, public Target<std::unique_ptr<Loader>> {
public:
struct Options {
std::unique_ptr<ResourceTracker> resource_tracker;
std::unique_ptr<AspiredVersionPolicy> aspired_version_policy;
int64_t manage_state_interval_micros = 100000; // 100ms
uint32 num_load_threads = 0;
uint32 num_unload_threads = 0;
uint32 max_num_load_retries = 5;
int64_t load_retry_interval_micros = 60000000; // 60s
Env* env = Env::Default();
// ... additional options
};
static absl::Status Create(
Options options,
std::unique_ptr<AspiredVersionsManager>* manager
);
};
Import
#include "tensorflow_serving/core/aspired_versions_manager.h"
#include "tensorflow_serving/core/availability_preserving_policy.h"
#include "tensorflow_serving/core/resource_preserving_policy.h"
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| options | Options | Yes | Configuration including policy, thread counts, retry settings |
| aspired_version_policy | unique_ptr<AspiredVersionPolicy> | Yes | Policy governing load/unload ordering |
| aspired versions callback | vector<ServableData<unique_ptr<Loader>>> | Yes | From FileSystemStoragePathSource via adapters |
Outputs
| Name | Type | Description |
|---|---|---|
| Loaded versions | ServableHandle | Available for inference via GetServableHandle() |
| State transitions | Events | Published to ServableStateMonitor via EventBus |
Usage Examples
ServerCore Creates Manager (C++)
// Typically called inside ServerCore::Create()
AspiredVersionsManager::Options manager_options;
manager_options.aspired_version_policy.reset(
new AvailabilityPreservingPolicy());
manager_options.num_load_threads = options.num_load_threads;
manager_options.max_num_load_retries = options.max_num_load_retries;
std::unique_ptr<AspiredVersionsManager> manager;
TF_RETURN_IF_ERROR(AspiredVersionsManager::Create(
std::move(manager_options), &manager));