Implementation:Tensorflow Serving Threadpool Executor
| Knowledge Sources | |
|---|---|
| Domains | Concurrency, Utility |
| Last Updated | 2026-02-13 00:00 GMT |
Overview
A concrete Executor implementation that uses a TensorFlow thread pool to execute scheduled closures across multiple threads.
Description
ThreadPoolExecutor is a concrete implementation of the Executor interface that wraps a thread::ThreadPool from TensorFlow's core library. It is constructed with an Env pointer, a thread pool name (for debugging/profiling), and the number of threads. The Schedule() method delegates to the underlying ThreadPool's Schedule method. On destruction, the ThreadPoolExecutor waits for all scheduled work to complete before destroying the thread pool. The number of threads must be greater than zero.
Usage
Use ThreadPoolExecutor as the standard executor implementation for production serving workloads where you need concurrent execution of closures across a configurable number of threads. It is typically passed to components like the HTTP server or model manager that require an Executor.
Code Reference
Source Location
- Repository: Tensorflow_Serving
- File:
tensorflow_serving/util/threadpool_executor.h - Lines: 1-52
Signature
class ThreadPoolExecutor : public Executor {
public:
ThreadPoolExecutor(Env* env, const string& thread_pool_name, int num_threads);
~ThreadPoolExecutor() override;
void Schedule(std::function<void()> fn) override;
private:
thread::ThreadPool thread_pool_;
};
Import
#include "tensorflow_serving/util/threadpool_executor.h"
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| env | Env* |
Yes | TensorFlow environment used to start threads |
| thread_pool_name | const string& |
Yes | Name for the thread pool (used in logging/profiling) |
| num_threads | int |
Yes | Number of threads in the pool; must be > 0 |
| fn | std::function<void()> |
Yes (Schedule) | Closure to execute in the thread pool |
Outputs
| Name | Type | Description |
|---|---|---|
| (none) | void | Closures are executed asynchronously in the thread pool |
Usage Examples
Creating and Using a ThreadPoolExecutor
auto executor = std::make_unique<ThreadPoolExecutor>(
Env::Default(), "serving_pool", /*num_threads=*/4);
executor->Schedule([]() {
// This runs in one of the 4 pool threads
HandleRequest();
});
// On destruction, waits for all pending work to complete