Principle:Microsoft Playwright Verify Expectations with Agent

Knowledge Sources	Playwright AI Testing Playwright Assertions
Domains	AI_Testing, Browser_Automation, Test_Assertions
Last Updated	2026-02-11 00:00 GMT

Overview

Using AI agents to verify expected page states described in natural language leverages LLM understanding to map human-readable expectations to concrete browser assertions without requiring explicit selectors or values.

Description

Traditional test assertions require the test author to specify exact selectors, text values, URLs, or element states. AI-driven assertion verification allows expectations to be expressed in natural language, with the agent autonomously determining how to verify them against the actual page state.

This principle addresses several challenges:

• Selector fragility: Traditional assertions break when DOM structure changes. Natural language expectations like "the cart shows 3 items" are resilient to UI refactoring.
• Complex state verification: Some expectations involve multiple elements, visual layout, or contextual understanding that is difficult to express with simple assertions.
• Readability: Natural language expectations serve as self-documenting test specifications that non-technical stakeholders can understand.
• Adaptive verification: The agent can handle variations in how information is presented (e.g., "3 items", "Items: 3", or "Your cart (3)") without hardcoding each variant.

The agent maps natural language expectations to a set of specialized assertion tools that verify specific aspects of the page: element visibility, text content, input values, list contents, URL, and page title.

Important distinction from perform(): While perform() executes actions that change page state, expect() only observes and verifies. The agent should never perform clicks, navigation, or data entry during an expect() call -- it should only read the page and evaluate whether the expectation is met.

Usage

Apply this principle when:

• You need to assert page state without hardcoding selectors
• You want human-readable test expectations that serve as documentation
• The UI under test changes frequently but the functional expectations remain stable
• You are verifying complex page states that would require multiple traditional assertions
• You want to combine AI-driven actions (perform) with AI-driven verification (expect) for end-to-end test coverage

Theoretical Basis

The AI-driven assertion model can be understood as a mapping from natural language to structured verification:

ExpectationVerification(expectation):
  snapshot = capturePageState(page)

  // LLM maps the natural language expectation to concrete checks
  checks = LLM.decompose(expectation, snapshot)

  for check in checks:
    tool = selectVerificationTool(check)
    result = tool.verify(page, check.parameters)

    if result == FAIL:
      throw AssertionError(expectation, actual=snapshot)

  return SUCCESS

Verification tool taxonomy:

The assertion tools form a complete coverage of common verification needs:

Category	Verification Tool	What It Checks
Element presence	expect_visible	An element matching a description is visible on the page
Text content	expect_visible_text	Specific text is visible on the page
Input state	expect_value	An input field contains an expected value
List content	expect_list_visible	A list of items matches expected content
Navigation state	expect_url	The current URL matches a pattern
Page metadata	expect_title	The page title matches an expected value

Timeout model:

Unlike perform(), which may loop for many actions, expect() is typically bounded by a timeout rather than an action count. This is because verification should be fast -- the page is already in the expected state (or not). The timeout accounts for:

• Dynamic content that may still be loading
• Animations or transitions completing
• Asynchronous data fetching

timeout_resolution:
  1. Per-call timeout (options.timeout)
  2. Agent-level expectTimeout (_expectTimeout)
  3. Default: 5000ms

Assertion failure reporting:

When an AI-driven assertion fails, the error should include:

• The original natural language expectation
• The actual page state (snapshot) at the time of failure
• Which specific verification check failed
• Contextual information to help diagnose the discrepancy

Related Pages

Implemented By

• Implementation:Microsoft_Playwright_Agent_Expect