Principle:Microsoft Semantic kernel Connector Embeddings Test Fixtures

Overview

Testing pattern that uses mock embedding API responses to validate Semantic Kernel connector implementations across multiple AI providers. By maintaining provider-specific mock response fixtures for Google, Vertex AI, HuggingFace, and MistralAI, the Semantic Kernel ensures correct response parsing, embedding vector extraction, and error handling without requiring live API calls.

Description

Each AI connector in the Semantic Kernel abstracts a provider's embedding API behind a common interface. To validate these connectors reliably, the test suite uses mock response fixtures that mirror the actual JSON structures returned by each provider's API.

Provider-Specific Response Structures

Each embedding provider returns responses in a distinct format, requiring provider-specific parsing logic:

Google AI (Gemini): Returns embeddings within a embedding object containing a values array of floating-point numbers. The response may include model metadata and token counts.

{
  "embedding": {
    "values": [0.0123, -0.0456, 0.0789, ...]
  }
}

Vertex AI: Uses Google Cloud's Vertex AI endpoint format, returning predictions that contain embeddings with a values field and associated statistics such as truncated flags and token counts.

{
  "predictions": [
    {
      "embeddings": {
        "values": [0.0123, -0.0456, 0.0789, ...],
        "statistics": {
          "truncated": false,
          "token_count": 8
        }
      }
    }
  ]
}

HuggingFace: Returns a simple array of embedding vectors, where each vector corresponds to an input text. The response may be a flat array or wrapped in a response object depending on the inference endpoint type.

[
  [0.0123, -0.0456, 0.0789, ...]
]

MistralAI: Follows an OpenAI-compatible response format with a data array containing objects that each include an embedding vector, along with model and usage metadata.

{
  "id": "embd-abc123",
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "embedding": [0.0123, -0.0456, 0.0789, ...],
      "index": 0
    }
  ],
  "model": "mistral-embed",
  "usage": {
    "prompt_tokens": 8,
    "total_tokens": 8
  }
}

What the Test Fixtures Validate

The mock response fixtures enable testing of:

Response Deserialization: Correct parsing of provider-specific JSON structures into internal embedding representations.
Vector Extraction: Accurate extraction of floating-point embedding vectors from nested response objects.
Batch Handling: Correct processing of multiple embeddings returned in a single API response.
Metadata Extraction: Parsing of usage statistics, token counts, and model identifiers.
Error Response Handling: Validation that connector code gracefully handles error responses, rate limits, and malformed payloads.

Usage

Use connector embedding test fixtures in the following scenarios:

Developing New Connectors: When implementing a new embedding provider connector, create mock response fixtures based on the provider's API documentation and use them to drive test-driven development.
Debugging Connector Issues: When a connector fails to parse embedding responses, compare the actual API response against the test fixture to identify structural discrepancies.
Adding Support for New Embedding Providers: Follow the established fixture pattern to create mock responses for the new provider, ensuring consistency with the existing test infrastructure.
Validating Provider API Changes: When a provider updates their API response format, update the test fixtures accordingly and verify that the connector handles both old and new formats.

Theoretical Basis

Mock-Based Unit Testing Methodology

Mock-based testing replaces external service dependencies with controlled test doubles that produce deterministic outputs. For embedding connectors, this means substituting HTTP responses from AI providers with pre-recorded JSON fixtures. This approach eliminates test flakiness caused by network issues, rate limits, or API key requirements, and enables fast, repeatable test execution.

API Contract Testing

Each mock fixture encodes the API contract between the Semantic Kernel connector and the provider's embedding service. The fixture represents the expected response structure, and tests verify that the connector correctly implements its side of the contract. When a provider changes their API, the fixture update makes the contract change explicit and reviewable.

Provider Abstraction Validation

The Semantic Kernel defines a common embedding interface that all connectors must implement. Test fixtures for each provider validate that the connector correctly translates provider-specific response formats into this common interface. This ensures that higher-level code can work with embeddings from any provider without knowledge of the underlying API structure, validating the abstraction's correctness.

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment