Implementation:Mlc ai Web llm Get Tool Call From Output

Overview

Get_Tool_Call_From_Output implements the Mlc_ai_Web_llm_Tool_Call_Extraction principle by providing the getToolCallFromOutputMessage function that parses raw model output JSON into typed tool call objects, and the engine integration that invokes it during chat completion processing.

Source Reference

• Parser function: src/support.ts, Lines 129-204
• Engine integration (non-streaming): src/engine.ts, Lines 854-868
• Engine integration (streaming): src/engine.ts, Lines 626-654
• Repository: mlc-ai/web-llm

Code Reference

Function Signature (Overloaded)

// src/support.ts L129-136
export function getToolCallFromOutputMessage(
  outputMessage: string,
  isStreaming: false,
): Array<ChatCompletionMessageToolCall>;

export function getToolCallFromOutputMessage(
  outputMessage: string,
  isStreaming: true,
): Array<ChatCompletionChunk.Choice.Delta.ToolCall>;

Implementation

// src/support.ts L137-204
export function getToolCallFromOutputMessage(
  outputMessage: string,
  isStreaming: boolean,
):
  | Array<ChatCompletionMessageToolCall>
  | Array<ChatCompletionChunk.Choice.Delta.ToolCall> {
  // 1. Parse outputMessage to JSON object
  let toolCallsObject;
  try {
    toolCallsObject = JSON.parse(outputMessage);
  } catch (err) {
    throw new ToolCallOutputParseError(outputMessage, err as Error);
  }

  // 2. Expect to be an array
  if (!(toolCallsObject instanceof Array)) {
    throw new ToolCallOutputInvalidTypeError("array");
  }

  // 3. Parse each tool call and populate tool_calls
  const numToolCalls = toolCallsObject.length;
  const tool_calls = [];
  for (let id = 0; id < numToolCalls; id++) {
    const curToolCall = toolCallsObject[id];
    if (curToolCall.name === undefined || curToolCall.arguments === undefined) {
      throw new ToolCallOutputMissingFieldsError(
        ["name", "arguments"],
        curToolCall,
      );
    }
    tool_calls.push({
      name: curToolCall.name,
      arguments: JSON.stringify(curToolCall.arguments),
    });
  }

  // 4. Return based on whether it is streaming or not
  if (isStreaming) {
    const tool_calls_result: Array<ChatCompletionChunk.Choice.Delta.ToolCall> = [];
    for (let id = 0; id < numToolCalls; id++) {
      const curToolCall = tool_calls[id];
      tool_calls_result.push({
        index: id,
        function: {
          name: curToolCall.name,
          arguments: curToolCall.arguments,
        },
        type: "function",
      });
    }
    return tool_calls_result;
  } else {
    const tool_calls_result: Array<ChatCompletionMessageToolCall> = [];
    for (let id = 0; id < numToolCalls; id++) {
      const curToolCall = tool_calls[id];
      tool_calls_result.push({
        id: id.toString(),
        function: {
          name: curToolCall.name,
          arguments: curToolCall.arguments,
        },
        type: "function",
      });
    }
    return tool_calls_result;
  }
}

Engine Integration (Non-Streaming)

// src/engine.ts L854-888
// 3. Post processing for function calling
const isFunctionCalling =
  request.tools !== undefined && request.tools !== null;
let tool_calls: Array<ChatCompletionMessageToolCall> | undefined;
if (
  selectedPipeline.getFinishReason() === "stop" &&
  isFunctionCalling
) {
  // If stopped due to length or abort, cannot output return tool_calls field
  finish_reason = "tool_calls";
  tool_calls = getToolCallFromOutputMessage(
    outputMessage,
    /*isStreaming=*/ false,
  );
}

choices.push({
  finish_reason: finish_reason,
  index: i,
  logprobs: /* ... */,
  message: isFunctionCalling
    ? {
        content: null,
        tool_calls: tool_calls,
        role: "assistant",
      }
    : {
        content: outputMessage,
        role: "assistant",
      },
});

I/O Contract

Function signature:

Parameter	Type	Description
outputMessage	string	Raw JSON string from the model output
isStreaming	boolean	Whether the response is streaming

Return type (non-streaming, isStreaming: false):

Array<ChatCompletionMessageToolCall>
// Each element:
{
  id: string;           // Array index as string ("0", "1", ...)
  function: {
    name: string;       // Function name
    arguments: string;  // JSON-serialized arguments string
  };
  type: "function";
}

Return type (streaming, isStreaming: true):

Array<ChatCompletionChunk.Choice.Delta.ToolCall>
// Each element:
{
  index: number;        // Array index as number (0, 1, ...)
  function: {
    name: string;       // Function name
    arguments: string;  // JSON-serialized arguments string
  };
  type: "function";
}

Error types thrown:

Error	When	Contains
ToolCallOutputParseError	JSON.parse fails	Original message string and underlying Error
ToolCallOutputInvalidTypeError	Parsed result is not an array	Expected type string ("array")
ToolCallOutputMissingFieldsError	Element missing required fields	List of required fields and the offending object

Key transformation: The arguments field is transformed from a JSON object (as output by the model) to a JSON string (as required by the OpenAI API convention) via JSON.stringify(curToolCall.arguments).

Usage Examples

Internal usage (how the engine calls it):

// This is internal engine code -- users do not call this directly
import { getToolCallFromOutputMessage } from "./support";

// Model outputs: '[{"name":"get_weather","arguments":{"location":"NYC"}}]'
const rawOutput = '[{"name":"get_weather","arguments":{"location":"NYC"}}]';

const toolCalls = getToolCallFromOutputMessage(rawOutput, false);
// Result:
// [
//   {
//     id: "0",
//     function: {
//       name: "get_weather",
//       arguments: '{"location":"NYC"}'  // Note: string, not object
//     },
//     type: "function"
//   }
// ]

Multiple tool calls extraction:

// Model outputs two function calls
const rawOutput = JSON.stringify([
  { name: "get_weather", arguments: { location: "Pittsburgh, PA", unit: "celsius" } },
  { name: "get_weather", arguments: { location: "Tokyo, Japan", unit: "celsius" } },
]);

const toolCalls = getToolCallFromOutputMessage(rawOutput, false);
// Result: Array of 2 ChatCompletionMessageToolCall objects
// toolCalls[0].id === "0"
// toolCalls[1].id === "1"
// toolCalls[0].function.arguments === '{"location":"Pittsburgh, PA","unit":"celsius"}'

Streaming mode extraction:

const rawOutput = '[{"name":"get_weather","arguments":{"location":"NYC"}}]';

const toolCalls = getToolCallFromOutputMessage(rawOutput, true);
// Result:
// [
//   {
//     index: 0,              // number, not string
//     function: {
//       name: "get_weather",
//       arguments: '{"location":"NYC"}'
//     },
//     type: "function"
//   }
// ]

Consuming extracted tool calls in application code:

const reply = await engine.chat.completions.create(request);

if (reply.choices[0].finish_reason === "tool_calls") {
  for (const toolCall of reply.choices[0].message.tool_calls!) {
    const functionName = toolCall.function.name;
    const args = JSON.parse(toolCall.function.arguments);
    console.log(`Call ${functionName} with`, args);
  }
}

Related Pages

• Principle:Mlc_ai_Web_llm_Tool_Call_Extraction
• Mlc_ai_Web_llm_Chat_Completion_Tool -- Tool definitions that drive model output format
• Mlc_ai_Web_llm_Function_Calling_Model_Ids -- Schema and system prompt that constrain model output
• Mlc_ai_Web_llm_Tool_Choice_Request -- Tool choice determines whether extraction runs
• Mlc_ai_Web_llm_Tool_Execution_Pattern -- Consuming extracted tool calls in the execution loop