Principle:Bentoml BentoML Deployment Configuration
Overview
Deployment Configuration is the principle of separating infrastructure concerns from service code, enabling the same BentoML service to be deployed with different resource profiles across environments without code changes.
Concept
Configuring cloud deployment parameters for scaling, hardware, and environment allows teams to tailor infrastructure to specific workload requirements. By externalizing these settings from the service implementation, deployment configuration becomes a declarative specification that can be versioned, reviewed, and promoted across environments independently.
Theory
Deployment configuration separates infrastructure concerns (instance types, scaling policies, secrets) from service code. This enables the same service to be deployed with different resource profiles across environments without code changes. The key benefits of this separation include:
- Environment parity - The same Bento artifact runs in development, staging, and production with only configuration differences
- Infrastructure as Code - Deployment settings can be stored in YAML files, versioned in git, and reviewed through standard code review processes
- Declarative scaling - Scaling policies (min/max replicas) are declared up front rather than imperatively managed
- Secret management - Environment variables and secrets are injected at deployment time, never baked into the Bento artifact
- Strategy selection - Different deployment strategies (e.g., rolling updates) can be specified per environment
Configuration Layers
Deployment configuration supports multiple layers of specification, with later layers overriding earlier ones:
- Defaults - BentoCloud provides sensible defaults for all parameters
- Config file - A YAML file specifying deployment parameters
- Config dict - Programmatic dictionary of parameters passed via the API
- Explicit parameters - Individual parameters passed directly to the API call
- Extras - Additional provider-specific configuration not covered by standard parameters
Key Configuration Areas
Scaling
- scaling_min - Minimum number of replicas (scale-to-zero supported with 0)
- scaling_max - Maximum number of replicas for auto-scaling
Hardware
- instance_type - The compute instance specification (CPU, memory, GPU type)
- strategy - Deployment strategy for updates
Environment
- envs - Environment variables injected into the running service
- secrets - Named secrets from BentoCloud secret store
- labels - Key-value metadata for organizing and filtering deployments
Access Control
- access_authorization - Whether the endpoint requires authentication to invoke
- cluster - Target cluster for multi-cluster deployments
YAML Configuration Example
name: my-service-production
bento: my_service:latest
cluster: gcp-us-central1
access_authorization: true
scaling:
min: 2
max: 10
instance_type: gpu.a10.1
strategy: RollingUpdate
envs:
- name: MODEL_CACHE_DIR
value: /tmp/models
secrets:
- prod-api-keys
labels:
- key: team
value: ml-platform
Metadata
| Property | Value |
|---|---|
| Principle | Deployment Configuration |
| Domain | ML_Serving, Cloud_Deployment, Infrastructure |
| Workflow | BentoCloud_Deployment |
| Related Concepts | Infrastructure as Code, Declarative Configuration, GitOps |
| Implementation | Implementation:Bentoml_BentoML_DeploymentConfigParameters |