Implementation:Microsoft Semantic kernel InProcessRuntime

Overview

InProcessRuntime is the in-memory agent message-passing runtime in Microsoft Semantic Kernel that manages communication between orchestrated agents. It implements the Actor Model pattern using in-process message queues, handling the mechanics of routing messages between agents during concurrent, group chat, and handoff orchestrations.

Source file: dotnet/src/Agents/Runtime/InProcess/InProcessRuntime.cs:L18-100

Principle page: Orchestration Runtime

Principle:Microsoft_Semantic_kernel_Orchestration_Runtime

Code Reference

Class Definition

public class InProcessRuntime
{
    public InProcessRuntime();

    public Task StartAsync(CancellationToken cancellationToken = default);
    public Task RunUntilIdleAsync(CancellationToken cancellationToken = default);
}

Key Methods

Method	Return Type	Description
InProcessRuntime()	constructor	Creates a new in-process runtime instance. The runtime is not yet active; StartAsync must be called before use.
StartAsync(CancellationToken)	Task	Initializes the runtime and prepares it to route messages between agents. Must be called before any orchestration is invoked.
RunUntilIdleAsync(CancellationToken)	Task	Blocks until all agents have finished processing and no messages remain in flight. Typically called after GetValueAsync to ensure complete cleanup.

Runtime Lifecycle States

[Created] → StartAsync() → [Running] → InvokeAsync() → [Processing] → RunUntilIdleAsync() → [Idle]

State	Description
Created	Runtime instantiated but not yet active. Cannot route messages.
Running	Runtime initialized and ready to accept orchestration invocations.
Processing	Orchestration in progress; agents are exchanging messages through the runtime.
Idle	All agents have completed; no messages in flight. Safe to collect results.

I/O Contract

Input (Lifecycle)

new InProcessRuntime()
└── Creates runtime in "Created" state

runtime.StartAsync(cancellationToken?)
├── Transitions to "Running" state
└── Must be called before orchestration.InvokeAsync()

orchestration.InvokeAsync(input, runtime, cancellationToken?)
├── Registers orchestration agents with the runtime
├── Routes initial input to the appropriate agent(s)
└── Returns OrchestrationResult<T> for result collection

runtime.RunUntilIdleAsync(cancellationToken?)
├── Waits for all pending messages to be processed
└── Ensures clean state after orchestration completes

Output (Message Routing)

The runtime does not produce direct output to the caller. Instead, it internally:

InProcessRuntime internal behavior:
├── Maintains in-memory message queues for each registered agent
├── Routes messages between agents based on orchestration rules:
│   ├── Concurrent: Sends same input to all agents in parallel
│   ├── GroupChat: Routes each agent's output to the next agent per manager rules
│   └── Handoff: Routes output to the agent specified by the handoff decision
├── Tracks agent completion status
└── Signals idle when all agents complete and queues are empty

Usage Examples

Basic Runtime Lifecycle

// Create and start
InProcessRuntime runtime = new();
await runtime.StartAsync();

// Use with an orchestration
ConcurrentOrchestration orchestration = new(writer, reviewer);
OrchestrationResult<string[]> result = await orchestration.InvokeAsync(
    "Write about the ocean", runtime);

// Collect results and wait for idle
string[] results = await result.GetValueAsync(TimeSpan.FromSeconds(300));
await runtime.RunUntilIdleAsync();

Runtime with Concurrent Orchestration

ChatCompletionAgent optimist = new()
{
    Name = "Optimist",
    Instructions = "Provide an optimistic perspective on the topic.",
    Kernel = kernel
};

ChatCompletionAgent pessimist = new()
{
    Name = "Pessimist",
    Instructions = "Provide a pessimistic perspective on the topic.",
    Kernel = kernel
};

InProcessRuntime runtime = new();
await runtime.StartAsync();

ConcurrentOrchestration orchestration = new(optimist, pessimist);
OrchestrationResult<string[]> result = await orchestration.InvokeAsync(
    "The future of artificial intelligence", runtime);

string[] perspectives = await result.GetValueAsync(TimeSpan.FromSeconds(300));
await runtime.RunUntilIdleAsync();

Console.WriteLine($"Optimist: {perspectives[0]}");
Console.WriteLine($"Pessimist: {perspectives[1]}");

Runtime with Group Chat Orchestration

InProcessRuntime runtime = new();
await runtime.StartAsync();

GroupChatOrchestration groupChat = new(
    new RoundRobinGroupChatManager { MaximumInvocationCount = 5 },
    writer, reviewer);

OrchestrationResult<string> result = await groupChat.InvokeAsync(
    "Compose a limerick about programming", runtime);

string finalOutput = await result.GetValueAsync(TimeSpan.FromSeconds(300));
await runtime.RunUntilIdleAsync();

Console.WriteLine($"Final output: {finalOutput}");

Runtime with Handoff Orchestration

InProcessRuntime runtime = new();
await runtime.StartAsync();

HandoffOrchestration handoff = new(
    OrchestrationHandoffs.StartWith(triageAgent)
        .Add(triageAgent, [billingAgent, refundAgent])
        .Add(billingAgent, [triageAgent]),
    triageAgent, billingAgent, refundAgent);

OrchestrationResult<string> result = await handoff.InvokeAsync(
    "I need a refund for order #5678", runtime);

string resolution = await result.GetValueAsync(TimeSpan.FromSeconds(300));
await runtime.RunUntilIdleAsync();

Console.WriteLine($"Resolution: {resolution}");

Multiple Sequential Orchestrations on Same Runtime

InProcessRuntime runtime = new();
await runtime.StartAsync();

// First orchestration: concurrent analysis
ConcurrentOrchestration concurrent = new(sentimentAgent, summaryAgent);
var concurrentResult = await concurrent.InvokeAsync("Analyze this text...", runtime);
string[] analyses = await concurrentResult.GetValueAsync(TimeSpan.FromSeconds(300));

// Second orchestration: group chat refinement
GroupChatOrchestration groupChat = new(
    new RoundRobinGroupChatManager { MaximumInvocationCount = 3 },
    writerAgent, editorAgent);
var chatResult = await groupChat.InvokeAsync(
    $"Refine this analysis: {analyses[0]}", runtime);
string refined = await chatResult.GetValueAsync(TimeSpan.FromSeconds(300));

await runtime.RunUntilIdleAsync();

Related Pages

• Orchestration Runtime (Principle) -- The conceptual foundation for the runtime.
• Orchestration Patterns (Implementation) -- The orchestration classes that use the runtime.
• Multi-Agent Orchestration (Principle) -- The multi-agent patterns powered by the runtime.
• OrchestrationResult GetValueAsync (Implementation) -- Collecting results after runtime execution.
• Orchestration Result Collection (Principle) -- The principle of result collection.
• ChatCompletionAgent (Implementation) -- The agents managed by the runtime.