Principle:Microsoft Agent framework Sample Validation

Knowledge Sources	Microsoft_Agent_framework
Domains	Testing, CI_CD, Quality_Assurance
Last Updated	2026-02-11 17:00 GMT

Overview

Principle of automatically discovering and executing all code samples in a repository to validate they remain functional as the SDK evolves.

Description

Sample Validation addresses a common problem in SDK repositories: code samples in documentation and sample directories can silently break as APIs change. This principle mandates automated, concurrent execution of all sample scripts with:

1. Automatic discovery: Recursively find all executable sample files, filtering out infrastructure files.
2. Concurrent execution: Run samples in parallel to minimize validation time.
3. Failure classification: Distinguish between genuine errors and expected failures (e.g., interactive samples that require human input, timeouts for long-running samples).
4. Graceful handling of interactive samples: Pipe empty stdin so interactive samples fail with a known error type rather than hanging indefinitely.

This ensures that SDK users encounter working examples and that regressions in sample code are caught during CI.

Usage

Apply this principle when maintaining a repository with runnable code samples that should be validated in CI pipelines. It is essential for any SDK that provides extensive example code alongside its API.

Theoretical Basis

Pseudo-code Logic:

# Abstract sample validation algorithm (NOT real implementation)
samples = discover_samples(samples_dir, exclude=["_*", "__pycache__"])

with parallel_executor(max_workers=N) as pool:
    for sample in samples:
        result = pool.submit(
            run_with_timeout(sample, timeout=60, stdin="")
        )

    for result in as_completed(pool):
        classify_result(result)  # timeout | input_hang | error | success

report(results)
exit(1 if any_failures else 0)

Key properties:

• Empty stdin trick: Sending empty input causes interactive input() calls to raise EOFError immediately, preventing hangs
• Timeout enforcement: Prevents any single sample from blocking the entire validation run
• Categorized reporting: Helps maintainers distinguish between broken samples and expected interactive failures

Related Pages

• Implementation:Microsoft_Agent_framework_Run_All_Samples_Script