Implementation:Tensorflow Serving Fast Read Dynamic Ptr
| Knowledge Sources | |
|---|---|
| Domains | Concurrency, Utility |
| Last Updated | 2026-02-13 00:00 GMT |
Overview
A thread-safe smart pointer wrapper optimized for frequent concurrent reads with infrequent updates, using sharded read pointers to minimize contention.
Description
FastReadDynamicPtr<T> manages an object that is read frequently from multiple threads but updated rarely. It achieves high read throughput by sharding read pointers across CPU cores, so that concurrent readers do not contend on the same cache line. The class provides two key operations: get() returns a shared_ptr to the current object (fast path, lock-per-shard), and Update() atomically swaps in a new object and blocks until all outstanding readers of the old object have released their references, then returns the old object as a unique_ptr. Internally, it uses a ShareableOwnedPtr that maintains an atomic reference count and an absl::Notification to signal when all shares have been released. The ShardedReadPtrs holder maintains a double-buffered array of shared pointers per CPU shard, using an atomic index to flip between the two buffers during updates, ensuring temporal consistency so no reader ever goes "back in time." False sharing between shards is prevented by padding each shard to at least 64 bytes.
Usage
Use this when you need to serve a read-heavy, update-light object across many threads, such as serving model data or configuration that gets updated periodically. It is the foundation for managing servable versions in TensorFlow Serving.
Code Reference
Source Location
- Repository: Tensorflow_Serving
- File:
tensorflow_serving/util/fast_read_dynamic_ptr.h - Lines: 1-342
Signature
template <typename T,
typename ReadPtrHolder = internal_read_ptr_holder::ShardedReadPtrs<T>>
class FastReadDynamicPtr {
public:
using ReadPtr = std::shared_ptr<const T>;
using OwnedPtr = std::unique_ptr<T>;
explicit FastReadDynamicPtr(OwnedPtr = nullptr);
~FastReadDynamicPtr();
OwnedPtr Update(OwnedPtr new_object);
ReadPtr get() const;
};
Import
#include "tensorflow_serving/util/fast_read_dynamic_ptr.h"
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| new_object | std::unique_ptr<T> |
Yes (for Update) | The new object to swap in; may be null |
Outputs
| Name | Type | Description |
|---|---|---|
| get() | std::shared_ptr<const T> |
Read-only pointer to the current object; may be null |
| Update() | std::unique_ptr<T> |
The previous object, returned after all readers have released it |
Usage Examples
Basic Read/Update Pattern
// Initialization
FastReadDynamicPtr<HeavyWeightObject> ptr{InitialValue()};
// From updating thread (infrequent):
std::unique_ptr<HeavyWeightObject> new_obj(LongRunningFunction());
std::unique_ptr<HeavyWeightObject> old_obj = ptr.Update(std::move(new_obj));
// old_obj can be reused or will be destroyed
// From reading thread (frequent, performance-critical):
auto object = ptr.get();
if (object != nullptr) {
HandleRequest(object.get());
}