Principle:Microsoft Semantic kernel OpenAPI Test Fixtures

Overview

Testing pattern that uses versioned OpenAPI specification documents to validate the Semantic Kernel's plugin import and function generation pipeline. By maintaining test fixtures across OpenAPI v2.0 (Swagger), v3.0, and v3.1, the Semantic Kernel ensures its parsers correctly extract functions, parameters, security schemes, and response schemas from real-world API specifications.

Description

The Semantic Kernel's Functions layer includes parsers that can import OpenAPI specification documents and generate callable kernel functions from them. This capability is central to the plugin architecture, allowing developers to expose any REST API as a set of SK functions.

Multi-Version Parser Support

The OpenAPI specification has evolved through several major versions, each introducing new features and structural changes:

OpenAPI v2.0 (Swagger): The original widely-adopted specification format, using a flat structure with paths, definitions, and securityDefinitions.
OpenAPI v3.0: Introduced components for schema reuse, requestBody as a distinct concept, and support for multiple servers.
OpenAPI v3.1: Aligned JSON Schema support with the latest JSON Schema draft, added webhooks, and refined content negotiation.

Test fixtures for each version ensure that the SK parser handles version-specific constructs correctly.

What the Test Fixtures Validate

The test fixture documents exercise the following parser capabilities:

Function Extraction: Each API operation (GET, POST, PUT, DELETE) is correctly mapped to a kernel function with the appropriate name and description.
Parameter Handling: Path parameters, query parameters, header parameters, and request body schemas are correctly parsed and mapped to function parameters with appropriate types and validation.
Security Scheme Parsing: API key, OAuth2, and bearer token security definitions are correctly extracted and associated with the appropriate operations.
Response Schema Interpretation: Response body schemas are parsed to enable type-safe result handling and downstream processing.
Content Type Handling: Different media types (application/json, multipart/form-data, etc.) are correctly supported in request and response definitions.

# Example: OpenAPI v3.0 test fixture excerpt
openapi: "3.0.1"
info:
  title: "Test Plugin API"
  version: "1.0.0"
paths:
  /items/{itemId}:
    get:
      operationId: getItem
      summary: "Retrieve an item by ID"
      parameters:
        - name: itemId
          in: path
          required: true
          schema:
            type: string
      responses:
        "200":
          description: "Successful response"
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Item"

Messaging-Specific Fixtures

In addition to general-purpose API fixtures, specialized test documents (such as the Messages OpenAPI fixture) validate parsing of APIs that involve complex nested schemas, pagination patterns, and domain-specific conventions found in messaging and communication APIs.

Usage

Use OpenAPI test fixtures in the following scenarios:

Extending the OpenAPI Parser: When adding support for new OpenAPI features (such as webhooks in v3.1 or discriminators), add corresponding entries to the test fixtures and verify that the parser handles them correctly.
Debugging Plugin Import Issues: When a user reports that an OpenAPI spec does not import correctly, create a minimal reproduction in the test fixtures and use it to isolate the parser behavior.
Adding Support for New OpenAPI Versions: When a new OpenAPI specification version is released, create a new test fixture document that exercises the new features and verify parser compatibility.
Regression Testing: The fixture suite serves as a regression test bed, ensuring that changes to the parser do not break existing functionality across any supported specification version.

Theoretical Basis

OpenAPI Specification Standards

The OpenAPI Specification (formerly Swagger) is an industry-standard format for describing REST APIs. Each version defines a JSON/YAML schema for API descriptions:

v2.0 (Swagger): Defined by the original Swagger specification, using a monolithic document structure with top-level paths, definitions, and securityDefinitions.
v3.0: A major restructuring that introduced the components object for reusable schemas, separated request bodies from parameters, and added support for multiple server definitions with variable substitution.
v3.1: Aligned the schema object with JSON Schema 2020-12, enabling full JSON Schema compatibility including if/then/else, prefixItems, and other advanced constructs.

Contract-Based Testing Methodology

The test fixtures implement a contract-based testing approach, where the OpenAPI documents serve as contracts that define expected API behavior. The parser is tested against these contracts to ensure it produces kernel functions that correctly reflect the API's capabilities. This approach provides:

Deterministic Test Results: Fixtures produce the same results regardless of external service availability.
Version Coverage: Each specification version has dedicated fixtures ensuring comprehensive parser coverage.
Documentation as Tests: The fixtures simultaneously serve as documentation of supported OpenAPI features and as executable test cases.

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment