Implementation:Predibase Lorax Container Entrypoint

Knowledge Sources	LoRAX Docker Entrypoint
Domains	Infrastructure, MLOps
Last Updated	2026-02-08 02:00 GMT

Overview

Concrete tool for containerized server initialization provided by the LoRAX Docker entrypoint script.

Description

The container-entrypoint.sh script serves as the Docker ENTRYPOINT for LoRAX containers. It handles three responsibilities: syncing model weights from S3, launching the lorax-launcher process, and uploading weights back to S3 on container shutdown. The script traps SIGTERM/SIGKILL signals to ensure graceful weight upload before termination.

Usage

This script is automatically invoked when the Docker container starts. It is not called directly by users but is configured as the container entrypoint. Use it when building the LoRAX Docker image or deploying via Docker/Kubernetes.

Code Reference

Source Location

Repository: LoRAX
File: container-entrypoint.sh
Lines: 1-44

Signature

#!/bin/bash

# Key functions:
upload()              # Syncs local model weights back to S3 on shutdown
is_launcher_running() # Checks if lorax-launcher PID is still alive

# Main flow:
# 1. trap upload SIGTERM SIGKILL
# 2. ./sync.sh                    (download weights from S3)
# 3. lorax-launcher "$@" &        (launch server with passthrough args)
# 4. while is_launcher_running; sleep 1; done
# 5. upload                        (sync weights back on exit)

Import

# Used as Docker ENTRYPOINT, not imported directly
# In Dockerfile: ENTRYPOINT ["./container-entrypoint.sh"]

I/O Contract

Inputs

Name	Type	Required	Description
$@	CLI args	No	All arguments passed through to lorax-launcher
MODEL_ID	env var	Yes	HuggingFace model ID (e.g., "mistralai/Mistral-7B-v0.1")
HF_CACHE_BUCKET	env var	Yes	S3 bucket name for weight cache
HUGGINGFACE_HUB_CACHE	env var	No	Local cache directory path

Outputs

Name	Type	Description
Running container	Process	lorax-launcher process running in background
S3 sync	Side effect	Model weights synced to/from S3 bucket

Usage Examples

Docker Run

# Launch LoRAX container with GPU support
docker run --gpus all \
  -e MODEL_ID=mistralai/Mistral-7B-Instruct-v0.1 \
  -e HF_CACHE_BUCKET=my-model-cache \
  -p 3000:80 \
  ghcr.io/predibase/lorax:latest \
  --max-input-length 1024 \
  --max-total-tokens 2048

Related Pages

Implements Principle

Principle:Predibase_Lorax_Container_Environment_Setup

Requires Environment

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment