Implementation:Tensorflow Serving servable handle h

Knowledge Sources	Tensorflow_Serving
Domains	Model Serving, Core Framework
Last Updated	2026-02-13 00:00 GMT

Overview

ServableHandle is a typed smart pointer that provides safe, ref-counted access to a loaded servable object retrieved from a Manager.

Description

The ServableHandle<T> class template is the primary mechanism through which frontend code accesses loaded servable objects. It wraps an UntypedServableHandle (an internal type-erased handle) and performs type-safe downcasting via AnyPtr. The handle keeps the underlying servable alive for the duration of its existence through shared ownership of the Loader.

The class provides smart-pointer semantics: operator->(), operator*(), get(), and operator bool(). Comparison operators check both the servable pointer and the ServableId, so handles to the same pointer but different versions are not equal.

UntypedServableHandle is the non-templatized abstract base class used internally by Managers. SharedPtrHandle is a concrete implementation that uses shared_ptr<Loader> for reference counting.

Handles should not be held for long periods, as holding a handle can delay servable reloading. The servable object T is shared among multiple concurrent requests, so thread-safe usage must be ensured by the caller.

Usage

Use ServableHandle when retrieving a servable from a Manager via GetServableHandle(). The handle provides typed access to the underlying servable object and keeps it alive. Release the handle promptly after use to allow servable version transitions.

Code Reference

Source Location

• Repository: Tensorflow_Serving
• File: tensorflow_serving/core/servable_handle.h
• Lines: 1-148

Signature

class UntypedServableHandle {
 public:
  virtual ~UntypedServableHandle() = default;
  virtual const ServableId& id() const = 0;
  virtual AnyPtr servable() = 0;
};

template <typename T>
class ServableHandle {
 public:
  ServableHandle() = default;
  const ServableId& id() const;
  T& operator*() const;
  T* operator->() const;
  T* get() const;
  operator bool() const;
};

class SharedPtrHandle final : public UntypedServableHandle {
 public:
  explicit SharedPtrHandle(const ServableId& id, std::shared_ptr<Loader> loader);
  AnyPtr servable() override;
  const ServableId& id() const override;
};

Import

#include "tensorflow_serving/core/servable_handle.h"

I/O Contract

Inputs

Name	Type	Required	Description
untyped_handle	std::unique_ptr<UntypedServableHandle>	Yes (internal)	Type-erased handle created by Manager; passed via private constructor

Outputs

Name	Type	Description
id()	const ServableId&	The identifier of the servable this handle points to
operator->()	T*	Pointer-style access to the servable object
operator*()	T&	Dereference access to the servable object
get()	T*	Raw pointer to the servable object
operator bool()	bool	Returns true if the handle holds a valid (non-null) servable

Usage Examples

Getting and Using a ServableHandle

#include "tensorflow_serving/core/servable_handle.h"
#include "tensorflow_serving/core/manager.h"

using namespace tensorflow::serving;

// Define or use an existing servable type
class MyServable {
 public:
  void Predict(const Input& input, Output* output);
};

// Get the handle from the manager
ServableHandle<MyServable> handle;
TF_RETURN_IF_ERROR(manager->GetServableHandle(
    ServableRequest::Latest("my_model"), &handle));

// Use the handle as a smart pointer
handle->Predict(input, &output);

// Handle goes out of scope, releasing the servable reference

Checking Handle Validity

ServableHandle<MyServable> handle;
Status status = manager->GetServableHandle(request, &handle);
if (status.ok() && handle) {
  // Safe to use handle
  handle->DoWork();
}

Related Pages

• Principle:Tensorflow_Serving_Servable_Access_Pattern