Environment:PrefectHQ Prefect AI Integration Credentials
| Knowledge Sources | |
|---|---|
| Domains | Infrastructure, AI, Credentials |
| Last Updated | 2026-02-09 22:00 GMT |
Overview
API credentials and optional dependencies required for AI-powered Prefect workflows using pydantic-ai with OpenAI or Anthropic backends.
Description
Prefect's AI workflows (data analyst agent, database cleanup with AI approval) use `pydantic-ai` as the LLM framework, wrapping it in Prefect's `PrefectAgent` for durable execution with retries and observability. These workflows require external LLM API keys and additional Python packages beyond the core Prefect installation. The AI agent configuration supports multiple model backends (OpenAI, Anthropic) with model-specific timeouts, retry strategies, and tool integrations.
Usage
This environment is required when running AI-powered workflows that use `pydantic-ai` agents. Specifically, the AI Data Analyst Agent workflow requires `OPENAI_API_KEY`, and the Database Cleanup with AI Approval workflow requires `ANTHROPIC_API_KEY`. Workflows validate these credentials at startup and exit with a clear error if missing.
System Requirements
| Category | Requirement | Notes |
|---|---|---|
| OS | Any (Linux, macOS, Windows) | No hardware-specific requirements |
| Network | Outbound HTTPS to LLM API endpoints | `api.openai.com`, `api.anthropic.com` |
| Python | >= 3.10 | Same as core Prefect requirement |
Dependencies
Python Packages
- `pydantic-ai[prefect]` (includes `PrefectAgent` and `TaskConfig`)
- `pydantic` >= 2.10.1
- `pandas` (for data analysis workflows)
Model-Specific Packages
- `openai` (for `openai:gpt-4o` model backend)
- `anthropic` (for `AnthropicModel` with `claude-sonnet-4-5-20250929`)
Optional MCP Integration
- `prefect-mcp` (for MCP server providing read-only Prefect tools to AI agents)
Credentials
The following environment variables must be set depending on the AI workflow:
- `OPENAI_API_KEY`: OpenAI API key for AI Data Analyst Agent workflow. Validated at startup with `sys.exit(1)` if missing.
- `ANTHROPIC_API_KEY`: Anthropic API key for Database Cleanup with AI Approval workflow.
- `DO_NOT_TRACK`: Set to `1`, `true`, or `yes` to disable Prefect telemetry (unrelated to AI, but relevant for privacy-conscious deployments).
Quick Install
# For AI Data Analyst (OpenAI)
pip install prefect "pydantic-ai[prefect]" pandas openai
export OPENAI_API_KEY="sk-..."
# For AI Database Cleanup (Anthropic)
pip install prefect "pydantic-ai[prefect]" anthropic
export ANTHROPIC_API_KEY="sk-ant-..."
# For MCP integration (optional)
pip install prefect-mcp
Code Evidence
API key validation from `examples/ai_data_analyst_with_pydantic_ai.py:330-333`:
if not os.getenv("OPENAI_API_KEY"):
print("❌ Error: OPENAI_API_KEY environment variable not set")
sys.exit(1)
PrefectAgent durable execution wrapper from `examples/ai_data_analyst_with_pydantic_ai.py:172-181`:
prefect_agent = PrefectAgent(
agent=analysis_agent,
task_config=TaskConfig(
timeout_seconds=60.0,
retries=3,
retry_delay_seconds=[1.0, 2.0, 4.0],
),
)
MCP server configuration from `examples/ai_database_cleanup_with_approval.py:52-54`:
mcp_server = MCPServerStdio(
"prefect", "uvx", args=["--from", "prefect-mcp", "prefect-mcp-server"]
)
Common Errors
| Error Message | Cause | Solution |
|---|---|---|
| `OPENAI_API_KEY environment variable not set` | Missing API key | `export OPENAI_API_KEY="sk-..."` before running the workflow |
| `ModuleNotFoundError: No module named 'pydantic_ai'` | pydantic-ai not installed | `pip install "pydantic-ai[prefect]"` |
| `AuthenticationError: Invalid API key` | Invalid or expired API key | Verify your API key is valid and has sufficient quota |
| `TimeoutError` during AI agent execution | LLM response exceeded timeout | Increase `timeout_seconds` in `TaskConfig` (default: 60s for OpenAI, 120s for Anthropic) |
Compatibility Notes
- OpenAI model: AI Data Analyst uses `openai:gpt-4o` with 60s timeout and 3 retries with exponential backoff [1, 2, 4]s.
- Anthropic model: Database Cleanup uses `claude-sonnet-4-5-20250929` with 120s timeout and 2 retries.
- MCP Server: The `prefect-mcp-server` provides read-only tools for AI agents to query Prefect state. Requires `uvx` (from `uv`) for execution.
- Durable execution: `PrefectAgent` wraps LLM calls as Prefect tasks with automatic retries, meaning transient API failures are handled transparently.