Implementation:Tensorflow Serving HTTPServer Interface

Knowledge Sources	Tensorflow_Serving
Domains	HTTP, Abstraction
Last Updated	2026-02-13 00:00 GMT

Overview

An abstract interface defining the API contract for HTTP server implementations, including server lifecycle management, request handler registration, and request dispatching.

Description

This header defines three key abstractions for the HTTP server layer. EventExecutor is an abstract non-blocking executor for processing I/O or callback events (similar to Executor but specifically for the HTTP server context). ServerOptions is a configuration class holding port numbers, IP addresses, and the event executor. RequestHandlerOptions controls per-handler settings such as automatic gzip decompression of request bodies and maximum uncompressed size. HTTPServerInterface is the central abstract class defining the full server lifecycle: StartAcceptingRequests() to begin listening, Terminate() to initiate shutdown, WaitForTermination() (with optional timeout) to block until safe destruction, RegisterRequestHandler() for exact-URI-path matching, and RegisterRequestDispatcher() for custom URI dispatching logic (e.g., regex-based). The RequestHandler typedef is a function taking a ServerRequestInterface pointer, and RequestDispatcher is a function that examines a request and returns a handler or nullptr. Implementations must be thread-safe and support multiple instances per process.

Usage

Use HTTPServerInterface as the abstraction when writing code that depends on an HTTP server but should not be coupled to a specific implementation. Application code registers handlers and dispatchers against this interface.

Code Reference

Source Location

• Repository: Tensorflow_Serving
• File: tensorflow_serving/util/net_http/server/public/httpserver_interface.h
• Lines: 1-213

Signature

class EventExecutor {
 public:
  virtual ~EventExecutor() = default;
  virtual void Schedule(std::function<void()> fn) = 0;
};

class ServerOptions {
 public:
  void AddPort(int port);
  void AddIPAddress(absl::string_view ip_address);
  void SetExecutor(std::unique_ptr<EventExecutor> executor);
};

class RequestHandlerOptions {
 public:
  RequestHandlerOptions& set_auto_uncompress_input(bool should_uncompress);
  RequestHandlerOptions& set_auto_uncompress_max_size(int64_t size);
};

typedef std::function<void(ServerRequestInterface*)> RequestHandler;
typedef std::function<RequestHandler(ServerRequestInterface*)> RequestDispatcher;

class HTTPServerInterface {
 public:
  virtual bool StartAcceptingRequests() = 0;
  virtual bool is_accepting_requests() const = 0;
  virtual int listen_port() const = 0;
  virtual void Terminate() = 0;
  virtual bool is_terminating() const = 0;
  virtual void WaitForTermination() = 0;
  virtual bool WaitForTerminationWithTimeout(absl::Duration timeout) = 0;
  virtual void RegisterRequestHandler(absl::string_view uri,
                                      RequestHandler handler,
                                      const RequestHandlerOptions& options) = 0;
  virtual void RegisterRequestDispatcher(RequestDispatcher dispatcher,
                                         const RequestHandlerOptions& options) = 0;
};

Import

#include "tensorflow_serving/util/net_http/server/public/httpserver_interface.h"

I/O Contract

Inputs

Name	Type	Required	Description
port	int	Yes	Port to listen on (0 for ephemeral)
executor	std::unique_ptr<EventExecutor>	Yes	Executor for I/O event polling
uri	absl::string_view	Yes (RegisterRequestHandler)	URI path for handler registration
handler	RequestHandler	Yes	Callback for handling matched requests

Outputs

Name	Type	Description
listen_port()	int	The actual port being listened on
is_accepting_requests()	bool	Whether the server is currently accepting requests

Usage Examples

Server Configuration and Handler Registration

auto options = std::make_unique<ServerOptions>();
options->AddPort(0);  // ephemeral port
options->SetExecutor(std::make_unique<MyEventExecutor>());

auto server = CreateEvHTTPServer(std::move(options));

RequestHandlerOptions handler_opts;
handler_opts.set_auto_uncompress_input(true);

server->RegisterRequestHandler("/v1/models/predict",
    [](ServerRequestInterface* req) {
      // Handle predict request
      req->Reply();
    },
    handler_opts);

Related Pages

• Principle:Tensorflow_Serving_HTTP_Server_Interface