Implementation:Mlc ai Web llm Function Calling Model Ids

Overview

Function_Calling_Model_Ids implements the Mlc_ai_Web_llm_Function_Calling_Model_Selection principle by providing the concrete list of validated model identifiers and the system prompt template used for function calling in web-llm.

Source Reference

• Model IDs: src/config.ts, Lines 297-303
• System Prompt and Schema: src/support.ts, Lines 98-121
• Validation Logic: src/openai_api_protocols/chat_completion.ts, Lines 550-593
• Repository: mlc-ai/web-llm

Code Reference

Supported Model Identifiers

// src/config.ts L297-303
/**
 * Models that support function calling (i.e. usage of
 * `ChatCompletionRequest.tools`). More to come.
 */
export const functionCallingModelIds = [
  "Hermes-2-Pro-Llama-3-8B-q4f16_1-MLC",
  "Hermes-2-Pro-Llama-3-8B-q4f32_1-MLC",
  "Hermes-2-Pro-Mistral-7B-q4f16_1-MLC",
  "Hermes-3-Llama-3.1-8B-q4f32_1-MLC",
  "Hermes-3-Llama-3.1-8B-q4f16_1-MLC",
];

Function Call JSON Schema

// src/support.ts L105
export const officialHermes2FunctionCallSchema =
  '{"properties": {"arguments": {"title": "Arguments", "type": "object"}, ' +
  '"name": {"title": "Name", "type": "string"}}, "required": ["arguments", "name"], ' +
  '"title": "FunctionCall", "type": "object"}';

// src/support.ts L111
export const officialHermes2FunctionCallSchemaArray =
  `{"type":"array","items":${officialHermes2FunctionCallSchema}}`;

System Prompt Template

// src/support.ts L116-121
export const hermes2FunctionCallingSystemPrompt =
  `You are a function calling AI model. You are provided with function ` +
  `signatures within <tools></tools> XML tags. You may call one or more ` +
  `functions to assist with the user query. Don't make assumptions about ` +
  `what values to plug into functions. Here are the available tools: ` +
  `<tools> ${MessagePlaceholders.hermes_tools}  </tools>. ` +
  `Use the following pydantic model json schema for each tool call you ` +
  `will make: ${officialHermes2FunctionCallSchema} For each function call ` +
  `return a json object.`;

Validation and Injection Logic

// src/openai_api_protocols/chat_completion.ts L550-593
// 7. Function calling hardcoded handlings
if (request.tools !== undefined && request.tools !== null) {
  // 7.1 Check if model supports function calling
  if (!functionCallingModelIds.includes(currentModelId)) {
    throw new UnsupportedModelIdError(currentModelId, functionCallingModelIds);
  }

  // 7.2 Hard coded support for Hermes2Pro
  if (currentModelId.startsWith("Hermes-2-Pro-")) {
    // 7.2.1 Update response format to use json schema
    request.response_format = {
      type: "json_object",
      schema: officialHermes2FunctionCallSchemaArray,
    } as ResponseFormat;

    // 7.2.2 Modify system prompt to provide tools usage
    const hermes2SystemMessage = hermes2FunctionCallingSystemPrompt.replace(
      MessagePlaceholders.hermes_tools,
      JSON.stringify(request.tools),
    );
    // Prepend system message
    request.messages.unshift({
      role: "system",
      content: hermes2SystemMessage,
    });
  }
}

I/O Contract

Exports:

Export	Type	Description
functionCallingModelIds	string[]	Array of model IDs that support ChatCompletionRequest.tools
officialHermes2FunctionCallSchema	string	JSON Schema string for a single function call object
officialHermes2FunctionCallSchemaArray	string	JSON Schema string for an array of function call objects
hermes2FunctionCallingSystemPrompt	string	System prompt template with MessagePlaceholders.hermes_tools placeholder

Import:

import { functionCallingModelIds } from "@mlc-ai/web-llm";

Validation behavior:

• If request.tools is provided and the current model is not in functionCallingModelIds, UnsupportedModelIdError is thrown.
• If the model starts with "Hermes-2-Pro-", the engine automatically injects the system prompt and response format.
• If the user provides a custom response_format alongside tools, CustomResponseFormatError is thrown.
• If the user provides a custom system message alongside tools, CustomSystemPromptError is thrown.

Usage Examples

Iterating over available function calling models:

import { functionCallingModelIds } from "@mlc-ai/web-llm";

console.log("Available function calling models:");
for (const modelId of functionCallingModelIds) {
  console.log(`  - ${modelId}`);
}
// Output:
//   - Hermes-2-Pro-Llama-3-8B-q4f16_1-MLC
//   - Hermes-2-Pro-Llama-3-8B-q4f32_1-MLC
//   - Hermes-2-Pro-Mistral-7B-q4f16_1-MLC
//   - Hermes-3-Llama-3.1-8B-q4f32_1-MLC
//   - Hermes-3-Llama-3.1-8B-q4f16_1-MLC

Creating an engine with a validated model:

import * as webllm from "@mlc-ai/web-llm";

const selectedModel = webllm.functionCallingModelIds[0];
// "Hermes-2-Pro-Llama-3-8B-q4f16_1-MLC"

const engine = await webllm.CreateMLCEngine(selectedModel, {
  initProgressCallback: (report) => {
    console.log(report.text);
  },
});

Runtime validation (automatic):

// When tools are passed, the engine automatically validates
// that the loaded model is in functionCallingModelIds.
// No explicit check needed by the user.
const reply = await engine.chat.completions.create({
  messages: [{ role: "user", content: "Get weather for NYC" }],
  tools: tools,
  tool_choice: "auto",
});
// If model is not in functionCallingModelIds, throws UnsupportedModelIdError

Related Pages

• Principle:Mlc_ai_Web_llm_Function_Calling_Model_Selection
• Mlc_ai_Web_llm_Chat_Completion_Tool -- Tool type definitions used with these models
• Mlc_ai_Web_llm_Tool_Choice_Request -- Tool choice options for validated models
• Mlc_ai_Web_llm_Get_Tool_Call_From_Output -- Parsing output from these models