Principle:Mlc ai Web llm Tool Definition

Overview

Tool Definition is the technique of declaring callable functions as tools that a language model can invoke during conversation to access external capabilities. In the mlc-ai/web-llm framework, tools are declared following the OpenAI function calling specification, enabling browser-based LLMs to interact with external systems through structured interfaces.

Description

Tool definition follows the OpenAI function calling specification. Each tool is described by three essential components:

• name -- A unique identifier for the function (must be alphanumeric with underscores and dashes, up to 64 characters).
• description -- A natural language explanation of what the function does, used by the model to decide when and how to invoke the tool.
• parameters -- A JSON Schema object specifying the accepted parameters, their types, descriptions, and which are required.

The model uses these descriptions at inference time to decide whether a tool call is appropriate for the user's query and how to construct valid arguments. Tools enable LLMs to interact with external systems (APIs, databases, calculators, sensors) in a structured, type-safe way rather than producing free-form text responses.

In web-llm, tools are passed as an array of ChatCompletionTool objects in the ChatCompletionRequest.tools field. Each tool object wraps a FunctionDefinition and is typed with type: "function" (currently the only supported tool type).

Usage

Use tool definition when building agentic applications where the LLM needs to call external functions. Define tools with clear, specific descriptions and precise parameter schemas so the model can correctly choose when and how to call them.

When to use:

• Building conversational agents that need access to live data (weather, stock prices, databases)
• Creating applications where the LLM must perform actions (sending messages, creating records)
• Implementing multi-step workflows where the model orchestrates multiple API calls

When not to use:

• Simple text generation tasks with no external interaction needed
• Models that are not in the functionCallingModelIds list (they will not reliably produce structured tool call output)

Theoretical Basis

Function calling maps natural language intent to structured API calls. The model is trained or prompted to follow a specific reasoning pipeline:

1. Analyze user intent -- Understand what the user is asking for and whether it requires external data or actions.
2. Match intent to available tool descriptions -- Compare the user request against the descriptions of all provided tools.
3. Generate structured JSON arguments -- Construct a JSON object conforming to the tool's parameter schema.
4. Return the function call instead of free text -- Output a structured tool call object rather than a natural language response.

In web-llm, the Hermes-2-Pro models follow the Hermes-2 function calling format where the model outputs a JSON array of {"name": ..., "arguments": ...} objects. The engine automatically:

• Injects a system prompt containing tool definitions wrapped in <tools></tools> XML tags
• Configures response_format to enforce JSON schema output matching the official Hermes-2 function call schema
• Parses the resulting JSON output into typed ChatCompletionMessageToolCall objects

The JSON Schema used to validate function calls is:

{
  "properties": {
    "arguments": {"title": "Arguments", "type": "object"},
    "name": {"title": "Name", "type": "string"}
  },
  "required": ["arguments", "name"],
  "title": "FunctionCall",
  "type": "object"
}

I/O Contract

Input:

• An array of ChatCompletionTool objects, each containing a FunctionDefinition with name, optional description, and optional parameters (JSON Schema).

Output:

• The engine injects these tool definitions into the model's system prompt and constrains output to the function call JSON schema.
• The model's response is parsed into ChatCompletionMessageToolCall objects accessible via response.choices[0].message.tool_calls.

Constraints:

• Tool name must be alphanumeric with underscores and dashes, maximum 64 characters.
• parameters must be a valid JSON Schema object.
• Only type: "function" is currently supported as a tool type.
• Only models listed in functionCallingModelIds support tool definitions.

Usage Examples

Defining a single tool with required and optional parameters:

import * as webllm from "@mlc-ai/web-llm";

const tools: Array<webllm.ChatCompletionTool> = [
  {
    type: "function",
    function: {
      name: "get_current_weather",
      description: "Get the current weather in a given location",
      parameters: {
        type: "object",
        properties: {
          location: {
            type: "string",
            description: "The city and state, e.g. San Francisco, CA",
          },
          unit: {
            type: "string",
            enum: ["celsius", "fahrenheit"],
          },
        },
        required: ["location"],
      },
    },
  },
];

Passing tools into a chat completion request:

const request: webllm.ChatCompletionRequest = {
  stream: false,
  messages: [
    {
      role: "user",
      content: "What is the current weather in celsius in Pittsburgh and Tokyo?",
    },
  ],
  tool_choice: "auto",
  tools: tools,
};

const reply = await engine.chat.completions.create(request);
// reply.choices[0].message.tool_calls contains the parsed tool calls
console.log(reply.choices[0].message.tool_calls);

Defining a tool with no parameters:

const tools: Array<webllm.ChatCompletionTool> = [
  {
    type: "function",
    function: {
      name: "get_server_time",
      description: "Get the current server time in UTC",
      // parameters omitted -- function accepts no arguments
    },
  },
];

Related Pages

• Implementation:Mlc_ai_Web_llm_Chat_Completion_Tool
• Mlc_ai_Web_llm_Function_Calling_Model_Selection -- Selecting models that support function calling
• Mlc_ai_Web_llm_Tool_Choice_Configuration -- Controlling which tools the model invokes
• Mlc_ai_Web_llm_Tool_Call_Extraction -- Parsing tool calls from model output
• Mlc_ai_Web_llm_Tool_Execution_Loop -- Executing tool calls and continuing conversation