Implementation:Tensorflow Serving EvHTTP Server
| Knowledge Sources | |
|---|---|
| Domains | HTTP, Networking |
| Last Updated | 2026-02-13 00:00 GMT |
Overview
A libevent-based HTTP server implementation that handles request dispatching, URI-based handler registration, graceful termination, and event loop management.
Description
EvHTTPServer implements HTTPServerInterface using libevent's evhttp API. On Initialize(), it creates an event_base and evhttp instance, configures allowed HTTP methods (GET, POST, HEAD, PUT, DELETE, OPTIONS, PATCH), and sets up a generic request callback. StartAcceptingRequests() binds to the configured port (with IPv6 fallback to IPv4), starts the event loop on a thread provided by the executor, and notifies via an absl::Notification. Request dispatching (DispatchEvRequest) first checks for exact URI path matches in the uri_handlers_ map, then falls through to registered dispatchers. Matched requests are wrapped in EvHTTPRequest objects and scheduled for handler execution via the configured EventExecutor. The server tracks pending operations with an atomic counter (IncOps/DecOps) to support graceful shutdown. Terminate() removes the listener socket from within the event loop, and WaitForTermination() (or the timeout variant) blocks until all pending operations complete before exiting the event loop. Thread safety for handler registration is ensured via an absl::Mutex. The libevent threading support is initialized once via absl::call_once with evthread_use_pthreads.
Usage
Use this as the HTTP server backend for TensorFlow Serving's REST API. It is created via the CreateEvHTTPServer() factory function and configured with ServerOptions specifying ports and an executor.
Code Reference
Source Location
- Repository: Tensorflow_Serving
- File:
tensorflow_serving/util/net_http/server/internal/evhttp_server.cc - Lines: 1-426
Signature
class EvHTTPServer : public HTTPServerInterface {
public:
explicit EvHTTPServer(std::unique_ptr<ServerOptions> options);
~EvHTTPServer();
bool Initialize();
bool StartAcceptingRequests() override;
bool is_accepting_requests() const override;
int listen_port() const override;
void Terminate() override;
bool is_terminating() const override;
void WaitForTermination() override;
bool WaitForTerminationWithTimeout(absl::Duration timeout) override;
void RegisterRequestHandler(absl::string_view uri,
RequestHandler handler,
const RequestHandlerOptions& options) override;
void RegisterRequestDispatcher(RequestDispatcher dispatcher,
const RequestHandlerOptions& options) override;
bool EventLoopSchedule(std::function<void()> fn);
};
Import
#include "tensorflow_serving/util/net_http/server/internal/evhttp_server.h"
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| options | std::unique_ptr<ServerOptions> |
Yes | Server configuration including ports and executor |
| uri | absl::string_view |
Yes (RegisterRequestHandler) | URI path for exact-match handler registration |
| handler | RequestHandler |
Yes | Function to handle requests matching the URI |
| dispatcher | RequestDispatcher |
No | Custom URI dispatching logic (e.g., regex-based) |
Outputs
| Name | Type | Description |
|---|---|---|
| listen_port() | int |
The actual port the server is listening on (resolves ephemeral port 0) |
| is_accepting_requests() | bool |
Whether the server has started accepting requests |
Usage Examples
Creating and Running the Server
auto options = std::make_unique<ServerOptions>();
options->AddPort(8500);
options->SetExecutor(std::make_unique<MyEventExecutor>());
auto server = CreateEvHTTPServer(std::move(options));
server->RegisterRequestHandler("/v1/models",
[](ServerRequestInterface* req) {
req->WriteResponseString("{\"status\": \"ok\"}");
req->Reply();
},
RequestHandlerOptions());
server->StartAcceptingRequests();
// ... server is running ...
server->Terminate();
server->WaitForTermination();