Implementation:PrefectHQ Prefect Worker CLI Commands

Metadata
Source	Repo: Prefect
Source	Doc: Prefect Workers
Domains	Deployment, Orchestration
Last Updated	2026-02-09 00:00 GMT

Overview

Concrete CLI commands for configuring Prefect work infrastructure including GCLs, work pools, and workers.

Description

Three Prefect CLI commands set up the per-worker concurrency infrastructure. This is an External Tool Doc since the commands are run outside of Python code. The commands are:

• prefect gcl create — create Global Concurrency Limits
• prefect work-pool create — create work pools
• prefect worker start — start worker processes

Code Reference

• Repository: https://github.com/PrefectHQ/prefect
• File: examples/per_worker_task_concurrency.py (L130-155) for documentation of commands
• Commands:

# Create per-worker GCL (2 concurrent GPU tasks per worker)
prefect gcl create gpu:gpu-1 --limit 2
prefect gcl create gpu:gpu-2 --limit 2

# Create work pool
prefect work-pool create ml-pool --type process

# Start workers with unique identities
WORKER_ID=gpu-1 prefect worker start --pool ml-pool --limit 10
WORKER_ID=gpu-2 prefect worker start --pool ml-pool --limit 10

• Import: N/A (CLI commands, not Python imports)

I/O Contract

Command	Input	Type	Description
gcl create	name	str	GCL name like gpu:worker-id
gcl create	--limit	int	Max concurrent slots
work-pool create	name	str	Pool name
work-pool create	--type	str	Pool type e.g. "process"
worker start	--pool	str	Work pool name
worker start	--limit	int	Max concurrent flow runs
worker start	WORKER_ID	env var	Worker identity for GCL scoping

Outputs: GCLs, work pools, and worker processes registered in Prefect server.

Usage Example

# 1. Create Global Concurrency Limits for each GPU worker
prefect gcl create gpu:gpu-1 --limit 2
prefect gcl create gpu:gpu-2 --limit 2

# 2. Create a process-based work pool
prefect work-pool create ml-pool --type process

# 3. Deploy flows
prefect deploy --all

# 4. Start workers (on separate machines)
# Machine 1:
WORKER_ID=gpu-1 prefect worker start --pool ml-pool --limit 10
# Machine 2:
WORKER_ID=gpu-2 prefect worker start --pool ml-pool --limit 10

# 5. Submit work
for i in {1..20}; do
  prefect deployment run process-image/process-image --param image_id=$i
done

Related Pages

• Principle:PrefectHQ_Prefect_Worker_Configuration
• Environment:PrefectHQ_Prefect_Python_Runtime_Environment
• Heuristic:PrefectHQ_Prefect_Concurrency_Limit_Scoping