Principle:Microsoft Semantic kernel Search Provider Test Fixtures

Overview

Testing pattern that uses mock web search responses to validate search plugin implementations across multiple providers (Bing, Brave, Google). These fixtures ensure correct result parsing, snippet extraction, URL handling, and site filtering without requiring live API keys or network access.

Description

Search plugins in the Semantic Kernel integrate external web search APIs to provide grounding data for AI-driven workflows. Each search provider returns results in a provider-specific format, and the corresponding plugin must correctly parse these responses into a common result model. Test fixtures provide pre-recorded API responses that enable deterministic validation of this parsing logic.

Provider-Specific Response Formats

Each search provider has a distinct response structure:

Bing Web Search API: Returns results in a webPages object containing a value array. Each result includes name, url, snippet, dateLastCrawled, and optional deep links.

{
  "webPages": {
    "value": [
      {
        "name": "Example Result",
        "url": "https://example.com",
        "snippet": "This is a search result snippet...",
        "dateLastCrawled": "2024-01-15T00:00:00Z"
      }
    ]
  }
}

Brave Search API: Returns results in a web object containing a results array. Each result includes title, url, description, and an optional age field indicating content freshness.

{
  "web": {
    "results": [
      {
        "title": "Example Result",
        "url": "https://example.com",
        "description": "This is a search result description...",
        "age": "2 days ago"
      }
    ]
  }
}

Google Custom Search API: Returns results in an items array. Each result includes title, link, snippet, and optional pagemap metadata containing structured data.

{
  "items": [
    {
      "title": "Example Result",
      "link": "https://example.com",
      "snippet": "This is a search result snippet...",
      "pagemap": {}
    }
  ]
}

Site Filtering

Some search plugins support site filtering, which restricts results to specific domains. The Bing site filter test fixture validates that the plugin correctly appends site: operators to queries and correctly parses filtered result sets. This is important for enterprise scenarios where search results should be scoped to trusted sources.

What the Test Fixtures Validate

Result Parsing: Correct extraction of search results from provider-specific JSON response structures.
Snippet Extraction: Accurate extraction of text snippets or descriptions that are used for AI grounding.
URL Handling: Correct parsing and normalization of result URLs across different provider formats.
Site Filtering: Validation that domain-scoped searches produce correctly filtered result sets.
Pagination Metadata: Correct interpretation of total result counts and pagination tokens where provided.
Empty and Error Responses: Graceful handling of queries that return no results or produce API errors.

Usage

Use search provider test fixtures in the following scenarios:

Developing Search Plugin Connectors: When implementing a new search provider connector, create mock response fixtures based on the provider's API documentation to drive test-driven development.
Testing Search Integration: Validate that the search plugin correctly transforms provider-specific responses into the common Semantic Kernel search result model.
Adding Support for New Search Providers: Follow the established fixture pattern to create mock responses for a new provider, maintaining consistency across the test suite.
Debugging Result Parsing Issues: When search results are incorrectly parsed or missing fields, compare the actual API response against the test fixture to identify the discrepancy.

Theoretical Basis

Search API Contract Testing

Each test fixture encodes the API contract between the Semantic Kernel search plugin and the external search provider. The fixture represents the expected response structure for a given query, and tests verify that the plugin produces correct, normalized results from this input. Contract testing ensures that the integration layer remains stable even as the underlying API evolves.

Provider-Agnostic Plugin Validation

The Semantic Kernel defines a common search result interface that all search provider plugins must implement. Test fixtures for each provider validate that the plugin correctly maps provider-specific fields (e.g., Bing's snippet vs. Brave's description vs. Google's snippet) to this common model. This validation ensures that consuming code can use search results from any provider interchangeably, without knowledge of the underlying API format. The provider-agnostic design supports runtime provider switching and simplifies higher-level orchestration logic.

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment