Implementation:MarketSquare Robotframework browser Libdoc Gen Stub

Libdoc and Gen Stub

Type

External Tool Doc

Source

• Stub generation: Browser/gen_stub.py, lines 23-91
• Documentation task: tasks.py, lines 842-883

Tools

Two tools work together to produce complete documentation for the Browser library:

1. robot.libdoc.libdoc (External)

Robot Framework's built-in documentation generator. It instantiates the library, introspects all keywords, and produces HTML documentation.

from robot.libdoc import libdoc

# Basic usage (built-in keywords only)
libdoc("Browser", "docs/Browser.html")

# With plugins included
libdoc("Browser::plugins=path/to/Plugin.py", "docs/Browser.html")

Parameters:

Parameter	Description
Library specification	"Browser" or "Browser::plugins=path". The :: syntax passes constructor arguments.
Output path	Path to the output HTML file.

The ::plugins= syntax causes libdoc to instantiate the Browser library with the specified plugins, so plugin keywords appear in the generated documentation.

2. Browser/gen_stub.py (Internal)

A script that generates Python type stub files (.pyi) for IDE autocompletion. It reads hand-written stub files from mypy_stub/ and combines them into Browser/browser.pyi.

gen_stub.py: Detailed Walkthrough

Step 1: Discover Keyword Method Names

import Browser

BR: Browser.Browser = Browser.Browser()
KW_METHOD_NAMES = [kw.__name__ for kw in BR.keywords.values()]
ADDED_KW = []

The script instantiates the Browser library (without plugins) and extracts the Python method names for all registered keywords. The BR.keywords dictionary maps keyword names to their underlying method objects.

Step 2: parse_kw_module_lines

def parse_kw_module_lines(lines: list[str]):
    for line in lines:
        for method_name in KW_METHOD_NAMES:
            if f"def {method_name}(" in line and method_name not in ADDED_KW:
                ADDED_KW.append(method_name)
                yield line

This function scans a list of lines (from a .pyi stub file) and yields lines that contain method definitions matching known keyword names. The ADDED_KW list prevents duplicate entries.

Logic:

• For each line in the input, check if it contains def method_name( for any known keyword method
• If found and not already added, yield the line and mark it as added
• This extracts typed method signatures from the stub files

Step 3: parse_kw_stubs

def parse_kw_stubs():
    for file in Path("mypy_stub/Browser/keywords").rglob("*.pyi"):
        with file.open("r", encoding="utf-8") as f:
            lines = f.readlines()
            yield from parse_kw_module_lines(lines)

Iterates over all .pyi files in the mypy_stub/Browser/keywords/ directory and extracts keyword method signatures from each.

Step 4: Assemble the Combined Stub File

pyi_boilerplate = """\
# Copyright 2020-     Robot Framework Foundation
# ...
from datetime import datetime
from concurrent.futures import Future
from os import PathLike

from assertionengine import AssertionOperator
from robot.utils import DotDict
from Browser.utils import (
    ClockType, CLockAdvanceType, FormatterTypes, CookieType,
    CookieSameSite, DownloadInfo, NewPageDetails, BrowserInfo,
    # ... many more type imports ...
)
from Browser.utils.data_types import (
    MouseButton, KeyboardModifier, ScrollBehavior,
    # ... more type imports ...
)
"""

with Path("Browser/browser.pyi").open("w", encoding="utf-8") as stub_file:
    stub_file.write(pyi_boilerplate)
    with Path("mypy_stub/Browser/browser.pyi").open("r", encoding="utf-8") as init_file:
        stub_file.write(init_file.read())
    stub_file.write("\n")
    for line in parse_kw_stubs():
        stub_file.write(line)

The output file Browser/browser.pyi is assembled from three parts:

1. Boilerplate -- License header and import statements for all types used in keyword signatures
2. Browser class stub -- The Browser class definition from mypy_stub/Browser/browser.pyi, containing the __init__ signature
3. Keyword method stubs -- Individual method signatures extracted from mypy_stub/Browser/keywords/*.pyi

Step 5: Fix robotlibcore Import

lines = []
with Path("Browser/browser.pyi").open("r", encoding="utf-8") as stub_file:
    for line in stub_file.readlines():
        if "from robotlibcore import DynamicCore" in line:
            lines.append(
                "from robotlibcore import DynamicCore # type: ignore\n"
            )
        else:
            lines.append(line)
with Path("Browser/browser.pyi").open("w", encoding="utf-8") as stub_file:
    stub_file.writelines(lines)

A post-processing step adds a # type: ignore comment to the robotlibcore import, since robotlibcore does not ship type stubs and would otherwise cause type checker warnings.

tasks.py: docs Task

The docs task in tasks.py (lines 842-883) wraps libdoc with additional processing:

@task
def docs(c, version=None):
    """Generate library keyword documentation.

    Args:
        version: Creates keyword documentation with version
        suffix in the name. Documentation is moved to docs/versions
        folder.
    """
    output = ROOT_DIR / "docs" / "Browser.html"
    libdoc("Browser", str(output))
    # ... HTML post-processing with BeautifulSoup ...
    if version is not None:
        target = (
            ROOT_DIR / "docs" / "versions"
            / f"Browser-{version.replace('v', '')}.html"
        )
        output.rename(target)

The task:

1. Calls libdoc("Browser", output) to generate the base HTML
2. Post-processes the HTML with BeautifulSoup to inject Google Analytics tracking scripts
3. Optionally moves the output to a versioned location if a version argument is provided

File Structure

File	Purpose
mypy_stub/Browser/browser.pyi	Hand-written stub for the Browser class __init__ and class-level definitions
mypy_stub/Browser/keywords/*.pyi	Hand-written stubs for each keyword module (one file per module)
Browser/gen_stub.py	Script that combines stubs into the final Browser/browser.pyi
Browser/browser.pyi	Generated combined type stub file used by IDEs
docs/Browser.html	Generated HTML keyword documentation
docs/versions/Browser-{ver}.html	Generated versioned documentation snapshots
tasks.py	Invoke task definitions including the docs task

Generating Documentation with Plugins

To generate documentation that includes plugin keywords, use the ::plugins= syntax with libdoc:

# Command line
python -m robot.libdoc "Browser::plugins=path/to/MyPlugin.py" output.html

# Multiple plugins
python -m robot.libdoc "Browser::plugins=PluginA.py,PluginB.py" output.html

# Plugin with arguments
python -m robot.libdoc "Browser::plugins=mypackage.MyPlugin;arg1" output.html

# Programmatic
from robot.libdoc import libdoc

libdoc("Browser::plugins=path/to/MyPlugin.py", "output.html")

Plugin keywords will appear in the generated HTML alongside built-in keywords. They are identifiable by the Plugin tag.

Limitations

• The gen_stub.py script only generates stubs for built-in keywords (it instantiates Browser() without plugins)
• Plugin keyword stubs are not included in the generated Browser/browser.pyi file
• For IDE support with plugin keywords, plugin authors must provide their own .pyi stub files or rely on IDE runtime introspection
• The docs task in tasks.py also does not include plugins by default; the user must modify the libdoc call to include plugins

Related

• MarketSquare_Robotframework_browser_Plugin_Documentation_Generation -- Principle: documentation generation for plugins
• MarketSquare_Robotframework_browser_PluginParser_Parse_Plugins -- How plugins are loaded (required for libdoc to discover plugin keywords)
• MarketSquare_Robotframework_browser_Keyword_Decorator_Pattern -- How keyword docstrings and tags feed into documentation