Principle:Microsoft Playwright Create Test with Agent Fixture
| Knowledge Sources | |
|---|---|
| Domains | AI_Testing, Browser_Automation, Test_Fixtures |
| Last Updated | 2026-02-11 00:00 GMT |
Overview
Extending a test framework with AI agent fixtures provides managed LLM-powered browser automation that integrates seamlessly with test lifecycle hooks, caching, and artifact collection.
Description
Modern test frameworks provide a fixture mechanism: reusable, composable building blocks that handle setup and teardown around test execution. When AI-agent-driven testing is introduced, the agent itself becomes a fixture that must be:
- Created before each test with proper LLM provider configuration
- Injected into the test function so the test author can issue natural language commands
- Torn down after each test, collecting usage metrics, saving cache files, and attaching diagnostic artifacts
The agent fixture pattern separates concerns cleanly. Test authors write natural language instructions without worrying about provider configuration, caching strategy, or artifact management. Infrastructure teams configure the agent once in a shared fixture definition, and all tests inherit that configuration.
Key lifecycle phases:
- Configuration resolution: Merge global defaults, project-level options, and per-test overrides into a final agent configuration.
- Agent instantiation: Create the agent from the page fixture and resolved configuration.
- Test execution: The test uses the agent to perform actions, assert expectations, and extract data.
- Artifact collection: After the test completes, usage statistics are attached as test artifacts for reporting.
- Cache management: On test pass, the agent's LLM response cache is persisted so subsequent runs can replay without hitting the LLM.
Usage
Apply this principle when:
- Integrating AI agents into an existing test suite that uses a fixture-based framework
- Establishing a consistent agent configuration across many tests
- Implementing caching strategies to reduce LLM costs in CI pipelines
- Collecting agent usage metrics for cost monitoring and optimization
- Supporting multiple execution modes (cached-only, LLM-for-missing, always-LLM)
Theoretical Basis
The agent fixture pattern follows the general Dependency Injection principle applied to test frameworks. The lifecycle can be modeled as:
TestLifecycle:
1. SETUP
a. Resolve agentOptions from configuration hierarchy
b. Obtain page fixture (browser page already created)
c. Create agent = page.agent(agentOptions)
2. EXECUTE
a. Pass agent to test function
b. Test calls agent.perform(), agent.expect(), agent.extract()
c. Agent communicates with LLM, executes browser actions
3. TEARDOWN
a. Collect agent usage (turns, inputTokens, outputTokens)
b. If usage.turns > 0: attach agent-usage.json artifact
c. If test passed: persist cache to disk for future replay
d. Dispose agent resources
Cache template resolution:
The cache path typically follows a template pattern that incorporates test metadata:
cachePathTemplate = "{testDir}/{testFilePath}-cache.json"
Resolved for a test at tests/login.spec.ts:
cachePath = "tests/login.spec.ts-cache.json"
This ensures each test file gets its own cache, preventing cross-test contamination while enabling cache reuse across test runs.
Execution modes:
A well-designed agent fixture supports multiple execution modes controlled by a CLI flag:
| Mode | Behavior | Use Case |
|---|---|---|
| none (default) | Use cache only; skip uncached calls | Fast CI runs, no LLM cost |
| missing | Use cache when available; call LLM for uncached | Incremental test development |
| all | Always call LLM, ignoring cache | Full validation, cache regeneration |
Composability:
The agent fixture composes with other test fixtures (page, context, browser) through the framework's dependency injection system. This means:
- The agent automatically gets a fresh page for each test
- Browser context isolation is maintained
- Parallel test execution works without agent conflicts