Implementation:Tensorflow Serving Serving Session

Knowledge Sources	Tensorflow_Serving
Domains	Model Serving, Session Management
Last Updated	2026-02-13 00:00 GMT

Overview

Provides read-only session wrappers that block state-changing operations (Create, Extend, Close) while allowing Run() for safe inference access.

Description

The Serving Session module defines a hierarchy of session wrapper classes that enforce read-only access patterns for TensorFlow sessions used in serving contexts:

ServingSession is an abstract base class extending TensorFlow's Session. It marks Create(), Extend(), and Close() as final methods that return error statuses, preventing any state modifications. Subclasses are expected to implement only the Run() methods for inference operations.

ServingSessionWrapper wraps an existing Session instance and delegates all three Run() overloads (basic, with RunOptions, and with ThreadPoolOptions) and ListDevices() to the wrapped session. All state-changing methods inherited from ServingSession return errors. This class is used by WrapSession in bundle_factory_util to convert a regular session into a read-only serving session.

SessionWrapperIgnoreThreadPoolOptions extends ServingSessionWrapper and overrides the Run() variant that accepts thread::ThreadPoolOptions to ignore those options, delegating to the simpler Run() overload instead. This is specifically designed for RemoteSession compatibility, where the Run() method with thread pool options is not implemented.

All wrapper classes prevent copy construction via TF_DISALLOW_COPY_AND_ASSIGN.

Usage

Use ServingSession as a base class for custom session implementations that should only support Run(). Use ServingSessionWrapper when wrapping an existing session to enforce read-only access in a serving pipeline. Use SessionWrapperIgnoreThreadPoolOptions when the underlying session does not support thread pool option overrides (e.g., remote sessions).

Code Reference

Source Location

• Repository: Tensorflow_Serving
• File: tensorflow_serving/servables/tensorflow/serving_session.h (lines 1-126)

Signature

class ServingSession : public Session {
 public:
  Status Create(const GraphDef& graph) final;
  Status Extend(const GraphDef& graph) final;
  Status Close() final;
};

class ServingSessionWrapper : public ServingSession {
 public:
  explicit ServingSessionWrapper(std::unique_ptr<Session> wrapped);
  Status Run(...) override;  // Three overloads
  Status ListDevices(std::vector<DeviceAttributes>* response) override;
};

class SessionWrapperIgnoreThreadPoolOptions : public ServingSessionWrapper {
 public:
  explicit SessionWrapperIgnoreThreadPoolOptions(
      std::unique_ptr<Session> wrapped);
  Status Run(const RunOptions& run_options, ...,
             const thread::ThreadPoolOptions& thread_pool_options) override;
};

Import

#include "tensorflow_serving/servables/tensorflow/serving_session.h"

I/O Contract

Inputs

Name	Type	Required	Description
wrapped	std::unique_ptr<Session>	Yes	The underlying TensorFlow session to wrap
inputs	vector<pair<string, Tensor>>	Yes	Named input tensors for Run()
output_tensor_names	vector<string>	Yes	Names of desired output tensors

Outputs

Name	Type	Description
outputs	vector<Tensor>*	Output tensors from the wrapped session's Run()
Create/Extend/Close	Status	Always returns Unimplemented error

Usage Examples

Wrapping a Session for Serving

std::unique_ptr<Session> session = /* create session */;
// Wrap to make it read-only
session.reset(new ServingSessionWrapper(std::move(session)));

// Run() works normally
std::vector<Tensor> outputs;
Status status = session->Run({{"input", tensor}}, {"output"}, {}, &outputs);

// State-changing operations are blocked
Status close_status = session->Close();  // Returns error

Related Pages

• Principle:Tensorflow_Serving_Session_Safety