Environment:SeldonIO Seldon core Docker Compose Local Environment
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, Local_Development |
| Last Updated | 2026-02-13 14:00 GMT |
Overview
Docker Compose-based local development environment running all Seldon Core 2 services (Kafka, scheduler, envoy, agents, inference servers) on a single machine.
Description
This environment provides a complete local Seldon Core 2 stack via Docker Compose. It includes Kafka (Bitnami 3.8.1), Zookeeper, the scheduler, envoy proxy, model and pipeline gateways, dataflow engine, agent, inference servers (MLServer and/or Triton), and optional observability tools (Prometheus, Grafana, Jaeger). Compose overlays select between MLServer (`all-mlserver.yaml`) and Triton (`all-triton.yaml`) backends, host networking (`all-host-network.yaml`) or internal Docker networking (`all-internal.yaml`), and GPU support (`all-gpu.yaml`).
Usage
Use this environment for local development, testing, and learning with Seldon Core 2. It does not require a Kubernetes cluster and runs entirely in Docker containers. It is the prerequisite for running any of the local example notebooks and smoke tests.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| OS | Linux / macOS / Windows (WSL2) | Docker Desktop or Docker Engine required |
| Docker | Docker Engine 20+ | Docker Compose v2 included |
| Memory | 8Gi minimum | Kafka + scheduler + envoy + agent + inference server |
| Disk | 20Gi free | Triton image is ~11GB; MLServer is ~2GB |
| CPU | x86-64-v2+ (SSE4.2) | Same requirement as Kubernetes deployment |
Dependencies
System Packages
- `docker` (Engine 20+)
- `docker-compose` (v2+ recommended)
- `make` (for Makefile targets)
- `seldon` CLI (for inference commands)
Container Images
- `docker.io/bitnami/kafka:3.8.1`
- `docker.io/bitnami/zookeeper:3.8`
- `seldonio/seldon-scheduler:latest`
- `seldonio/seldon-envoy:latest`
- `seldonio/seldon-modelgateway:latest`
- `seldonio/seldon-pipelinegateway:latest`
- `seldonio/seldon-dataflow-engine:latest`
- `seldonio/seldon-agent:latest`
- `seldonio/seldon-rclone:latest`
- `seldonio/mlserver:1.7.1` (MLServer backend)
- `nvcr.io/nvidia/tritonserver:23.03-py3` (Triton backend, optional)
Credentials
No credentials are required for local development. The environment uses PLAINTEXT for all communication channels.
Optional environment variables:
- `CUSTOM_IMAGE_TAG`: Override the default image tag for all Seldon images
- `GPU_ENABLED`: Set to `1` to enable NVIDIA GPU runtime
- `LOCAL_MODEL_FOLDER`: Path to local model artifacts (mounted at `/mnt/models`)
Quick Install
# Clone the repository
git clone https://github.com/SeldonIO/seldon-core.git
cd seldon-core/scheduler
# Start with MLServer backend (internal networking)
make deploy-local-mlserver
# Or start with Triton backend
make deploy-local-triton
# With GPU support
GPU_ENABLED=1 make deploy-local-triton
Code Evidence
Docker Compose base services from `scheduler/all-base.yaml`:
services:
kafka:
image: docker.io/bitnami/kafka:3.8.1
zookeeper:
image: docker.io/bitnami/zookeeper:3.8
scheduler:
image: seldonio/seldon-scheduler:latest
envoy:
image: seldonio/seldon-envoy:latest
GPU overlay from `scheduler/all-gpu.yaml`:
services:
triton:
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
Default port mappings from `scheduler/all-internal.yaml`:
# Envoy: 9000 (inference), 9001 (mirror)
# Scheduler: 9004 (gRPC)
# Kafka: 9092
# Prometheus: 9090
# Grafana: 3000
# Jaeger: 16686
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
| `Cannot connect to the Docker daemon` | Docker not running | Start Docker Engine or Docker Desktop |
| `port 9000 already in use` | Another service on port 9000 | Stop conflicting service or change port mapping |
| `Triton image pull timeout` | Triton image is ~11GB | Use MLServer backend for faster startup; pre-pull with `docker pull nvcr.io/nvidia/tritonserver:23.03-py3` |
| `nvidia runtime not found` | NVIDIA Container Toolkit not installed | Install nvidia-docker2 or nvidia-container-toolkit |
Compatibility Notes
- MLServer vs Triton: Use `all-mlserver.yaml` overlay for sklearn/xgboost/lightgbm/huggingface models. Use `all-triton.yaml` for TensorFlow/PyTorch/ONNX models.
- Networking modes: Host networking (`all-host-network.yaml`) shares host ports directly; internal networking (`all-internal.yaml`) uses Docker DNS with explicit port mappings.
- GPU support: Requires NVIDIA Container Toolkit and `GPU_ENABLED=1` environment variable.
- Local models: Mount a local directory with `LOCAL_MODEL_FOLDER=/path/to/models` for serving local artifacts.