Implementation:Microsoft Autogen Agbench CLI

Property	Value
Source	https://github.com/microsoft/autogen
Domains	Benchmarking CLI Command_Line_Interface Testing
Last Updated	2026-02-11 17:00 GMT

Overview

The main command-line interface entry point for AutoGenBench that routes subcommands to specialized handlers for running, tabulating, linting, and managing benchmark scenarios.

Description

The cli.py module serves as the primary entry point for the autogenbench command-line tool. It implements a dispatcher pattern that routes user commands to appropriate subcommand handlers. The module defines a CommandSpec typed dictionary structure to maintain command metadata including command names, descriptions, and handler functions. The main() function parses command-line arguments and delegates execution to one of four primary subcommands: run (execute benchmark configurations), tabulate (analyze and format results), lint (validate benchmark configurations), and remove_missing (clean up incomplete benchmark runs). The implementation includes comprehensive help text generation with dynamic formatting that adjusts to the longest command name for aligned display. Version information is accessible through the --version flag, and detailed help is available via --help. The module imports specialized CLI handlers from sibling modules and maintains a clean separation of concerns by delegating all substantive functionality to these specialized handlers.

Usage

Use this module when you need to:

• Provide a unified command-line interface for all AutoGenBench operations
• Route user commands to appropriate benchmark management functions
• Display version information or comprehensive help text
• Validate command-line arguments before delegating to subcommand handlers
• Maintain a consistent CLI experience across different benchmark operations

Code Reference

Source Location: python/packages/agbench/src/agbench/cli.py

Signature:

def main(args: Optional[List[str]] = None) -> None:
    """
    Main entry point for the autogenbench CLI tool.

    Parses command-line arguments and dispatches to appropriate subcommand handlers.
    If no args provided, uses sys.argv. Expects format: autogenbench COMMAND ARGS

    Args:
        args: Optional list of command-line arguments (defaults to sys.argv)

    Returns:
        None (exits with appropriate status code)
    """

Import:

from agbench.cli import main

I/O Contract

Inputs

Parameter	Type	Required	Description
args	Optional[List[str]]	No	Command-line arguments list. If None, uses sys.argv. Expected format: [program_name, command, ...command_args]

Outputs

Output	Type	Description
Return value	None	Function exits with sys.exit() status codes
Exit code 0	int	Successful command execution or help display
Exit code 2	int	Invalid command or missing arguments
stdout	str	Version information when --version is used
stderr	str	Usage text, help text, or error messages

Usage Examples

Basic Command Execution:

# Display version information
autogenbench --version

# Display help text
autogenbench --help

# Run a benchmark configuration
autogenbench run path/to/config.json

# Tabulate results from a previous run
autogenbench tabulate path/to/runlogs

# Lint a benchmark configuration
autogenbench lint path/to/logfile.txt

# Remove folders with missing results
autogenbench remove_missing path/to/runlogs

Programmatic Usage:

from agbench.cli import main

# Call with explicit arguments
main(["autogenbench", "run", "config.json"])

# Call with default arguments (uses sys.argv)
main()

# Access version information programmatically
from agbench.version import __version__
print(f"AutoGenBench version {__version__}")

Command Specification Structure:

from typing import Callable, Optional, Sequence
from typing_extensions import TypedDict

class CommandSpec(TypedDict):
    """Structure for defining CLI commands"""
    command: str  # Command name (e.g., "run", "tabulate")
    description: str  # Brief description for help text
    function: Optional[Callable[[Sequence[str]], None]]  # Handler function

# Example command specification
command_spec = {
    "command": "run",
    "description": "run a given benchmark configuration",
    "function": run_cli,
}

Related Pages

• Agbench_Linter_CLI - Log file analysis and error detection
• Agbench_Remove_Missing_Cmd - Cleanup of incomplete benchmark runs
• Agbench_Tabulate_Cmd - Results tabulation and statistics generation
• Agbench_Run_Cmd - Benchmark execution handler (imported but not detailed here)
• CLI_Dispatcher - Command-line routing pattern
• Argument_Parsing - Command-line argument processing