Environment:Tensorflow Serving Docker Runtime Environment
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, Containerization |
| Last Updated | 2026-02-13 17:00 GMT |
Overview
Docker container environment based on Ubuntu 20.04 exposing gRPC (8500) and REST (8501) ports for running TensorFlow Serving with configurable model paths.
Description
This environment provides a containerized runtime for TensorFlow Serving using Docker. The minimal serving image is based on Ubuntu 20.04 and includes only the pre-built `tensorflow_model_server` binary. Models are mounted into the container via Docker bind mounts or volumes. The container exposes two network ports: 8500 for gRPC and 8501 for REST API. Configuration is driven by environment variables (`MODEL_NAME`, `MODEL_BASE_PATH`) and command-line flags.
Usage
Use this environment for production deployment of TensorFlow Serving without building from source. It is the standard deployment method documented in the official guides. Available image variants include CPU-only, GPU-enabled, MKL-optimized, and development builds. This is the prerequisite for the Kubernetes deployment workflow.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| Container Runtime | Docker Engine | nvidia-docker2 required for GPU images |
| OS (Host) | Linux recommended | Docker Desktop available for macOS/Windows |
| Network | Ports 8500, 8501 available | gRPC and REST API respectively |
| Disk | Model files accessible | Bind mount or Docker volume |
Dependencies
Docker Images
| Image | Purpose | Base |
|---|---|---|
| `tensorflow/serving` | Minimal CPU serving | Ubuntu 20.04 |
| `tensorflow/serving:latest-gpu` | Minimal GPU serving | nvidia/cuda:12.2.0-base-ubuntu20.04 |
| `tensorflow/serving:nightly-devel` | Full build environment | Ubuntu 20.04 |
| `tensorflow/serving:latest-devel-gpu` | GPU build environment | nvidia/cuda:12.2.0-base-ubuntu20.04 |
| `tensorflow/serving:latest-devel-mkl` | MKL-optimized build | Ubuntu 20.04 |
Exposed Ports
- Port `8500`: gRPC API
- Port `8501`: HTTP/REST API
Credentials
The following environment variables configure the serving container:
- `MODEL_NAME`: Name of the model to serve (default: `model`)
- `MODEL_BASE_PATH`: Base path for model directories (default: `/models`)
Quick Install
# Pull and run the latest CPU serving image
docker pull tensorflow/serving
# Serve a model
docker run -p 8500:8500 -p 8501:8501 \
--mount type=bind,source=/path/to/my_model,target=/models/my_model \
-e MODEL_NAME=my_model \
tensorflow/serving
# For GPU serving
docker run --gpus all -p 8500:8500 -p 8501:8501 \
--mount type=bind,source=/path/to/my_model,target=/models/my_model \
-e MODEL_NAME=my_model \
tensorflow/serving:latest-gpu
Code Evidence
Docker entrypoint generates serving command from `docker.md:62-63`:
tensorflow_model_server --port=8500 --rest_api_port=8501 \
--model_name=${MODEL_NAME} --model_base_path=${MODEL_BASE_PATH}/${MODEL_NAME}
Port exposure from `Dockerfile:40-43`:
# gRPC
EXPOSE 8500
# REST
EXPOSE 8501
Development Docker build configuration from `Dockerfile.devel:16-18`:
ARG TF_SERVING_VERSION_GIT_BRANCH=master
ARG TF_SERVING_VERSION_GIT_COMMIT=HEAD
ENV DEBIAN_FRONTEND=noninteractive
Build-in-docker script default image from `run_in_docker.sh:72`:
# Default Docker image is tensorflow/serving:nightly-devel
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
| `No versions of servable my_model found under base path /models/my_model` | Model directory not mounted correctly | Verify bind mount source contains versioned subdirectory (e.g., `/path/to/my_model/1/`) |
| `docker: Error response from daemon: could not select device driver "nvidia"` | nvidia-docker2 not installed for GPU image | Install `nvidia-docker2` and restart Docker daemon |
| Port already in use | Another service on 8500/8501 | Map to different host ports: `-p 9500:8500 -p 9501:8501` |
Compatibility Notes
- GPU images: Require NVIDIA Container Toolkit (`nvidia-docker2`) and compatible NVIDIA drivers on the host.
- Model directory structure: Models must be in versioned subdirectories (e.g., `my_model/1/saved_model.pb`). The base path excludes the version number.
- Multi-model serving: Use `--model_config_file` instead of `MODEL_NAME`/`MODEL_BASE_PATH` environment variables for serving multiple models.