Workflow:Mlflow Mlflow Model Serving
| Knowledge Sources | |
|---|---|
| Domains | ML_Ops, Model_Deployment, Inference |
| Last Updated | 2026-02-13 20:00 GMT |
Overview
End-to-end process for deploying MLflow models as REST API endpoints for real-time inference, including local serving, Docker containerization, and cloud deployment.
Description
This workflow covers the procedure for taking a logged or registered MLflow model and deploying it as a prediction service. MLflow provides a built-in scoring server that wraps any logged model in a REST API with standardized input/output formats. The server accepts JSON, CSV, or Parquet payloads and returns predictions. For production deployments, models can be containerized as Docker images or deployed to cloud platforms (Databricks, AWS SageMaker, Azure ML) using the deployments plugin interface.
Key capabilities:
- Local model serving with built-in Flask/gunicorn server
- Docker image generation for containerized deployment
- Plugin-based deployment to cloud ML platforms
- Standardized REST API with health checks and prediction endpoints
- Support for multiple input formats (JSON, CSV, Parquet)
Usage
Execute this workflow when you have a trained and logged model that needs to serve real-time predictions via a REST API. This applies to local development testing, staging validation, and production deployment scenarios across various infrastructure targets.
Execution Steps
Step 1: Prepare Model Artifact
Ensure the model is logged to MLflow with proper packaging — including dependencies, model signature, and input example. The model URI (run-based or registry-based) will be used to reference the model for deployment.
Key considerations:
- The model must have a valid MLmodel metadata file
- Dependencies (conda.yaml or requirements.txt) ensure reproducible environments
- Model signatures enable automatic input validation at serving time
Step 2: Launch Local Serving
Start the MLflow scoring server locally using the CLI command. The server loads the model and exposes a REST API on the specified port. This is suitable for development testing and validation before production deployment.
Key considerations:
- The server exposes /invocations for predictions and /ping for health checks
- Multiple workers can be configured for concurrent request handling
- Environment restoration recreates the model's dependency environment
Step 3: Test Predictions
Send test requests to the serving endpoint to validate model behavior. The server accepts multiple input formats including DataFrame-oriented JSON (split or records), instances-based JSON, CSV, and Parquet binary payloads.
Key considerations:
- Use the dataframe_split format for structured tabular data
- For chat models, use the OpenAI-compatible chat completion format
- Response format matches the model's output signature
Step 4: Build Docker Image (Optional)
Generate a Docker image containing the model, its dependencies, and the scoring server for portable deployment. The generated image includes environment setup, model artifacts, and the serving entrypoint.
Key considerations:
- Docker images include all dependencies for self-contained deployment
- Custom base images can be specified for specific runtime requirements
- GPU support is available for deep learning models
Step 5: Deploy to Production
Deploy the model to a production environment using the MLflow deployments interface. The plugin-based system supports multiple deployment targets including Databricks Model Serving, AWS SageMaker, Azure ML, and Kubernetes. Each target has specific configuration options for compute, scaling, and routing.
Key considerations:
- The deployments plugin interface provides create, update, delete, and predict operations
- Each deployment target has specific configuration requirements
- Deployment supports both real-time endpoints and batch inference