Implementation:MarketSquare Robotframework browser Libdoc Gen Stub

Libdoc and Gen Stub

Type

External Tool Doc

Source

Stub generation: Browser/gen_stub.py, lines 23-91
Documentation task: tasks.py, lines 842-883

Tools

Two tools work together to produce complete documentation for the Browser library:

1. robot.libdoc.libdoc (External)

Robot Framework's built-in documentation generator. It instantiates the library, introspects all keywords, and produces HTML documentation.

from robot.libdoc import libdoc

# Basic usage (built-in keywords only)
libdoc("Browser", "docs/Browser.html")

# With plugins included
libdoc("Browser::plugins=path/to/Plugin.py", "docs/Browser.html")

Parameters:

Parameter	Description
Library specification	`"Browser"` or `"Browser::plugins=path"`. The `::` syntax passes constructor arguments.
Output path	Path to the output HTML file.

The ::plugins= syntax causes libdoc to instantiate the Browser library with the specified plugins, so plugin keywords appear in the generated documentation.

2. Browser/gen_stub.py (Internal)

A script that generates Python type stub files (.pyi) for IDE autocompletion. It reads hand-written stub files from mypy_stub/ and combines them into Browser/browser.pyi.

gen_stub.py: Detailed Walkthrough

Step 1: Discover Keyword Method Names

import Browser

BR: Browser.Browser = Browser.Browser()
KW_METHOD_NAMES = [kw.__name__ for kw in BR.keywords.values()]
ADDED_KW = []

The script instantiates the Browser library (without plugins) and extracts the Python method names for all registered keywords. The BR.keywords dictionary maps keyword names to their underlying method objects.

Step 2: parse_kw_module_lines

def parse_kw_module_lines(lines: list[str]):
    for line in lines:
        for method_name in KW_METHOD_NAMES:
            if f"def {method_name}(" in line and method_name not in ADDED_KW:
                ADDED_KW.append(method_name)
                yield line

This function scans a list of lines (from a .pyi stub file) and yields lines that contain method definitions matching known keyword names. The ADDED_KW list prevents duplicate entries.

Logic:

For each line in the input, check if it contains def method_name( for any known keyword method
If found and not already added, yield the line and mark it as added
This extracts typed method signatures from the stub files

Step 3: parse_kw_stubs

def parse_kw_stubs():
    for file in Path("mypy_stub/Browser/keywords").rglob("*.pyi"):
        with file.open("r", encoding="utf-8") as f:
            lines = f.readlines()
            yield from parse_kw_module_lines(lines)

Iterates over all .pyi files in the mypy_stub/Browser/keywords/ directory and extracts keyword method signatures from each.

Step 4: Assemble the Combined Stub File

pyi_boilerplate = """\
# Copyright 2020-     Robot Framework Foundation
# ...
from datetime import datetime
from concurrent.futures import Future
from os import PathLike

from assertionengine import AssertionOperator
from robot.utils import DotDict
from Browser.utils import (
    ClockType, CLockAdvanceType, FormatterTypes, CookieType,
    CookieSameSite, DownloadInfo, NewPageDetails, BrowserInfo,
    # ... many more type imports ...
)
from Browser.utils.data_types import (
    MouseButton, KeyboardModifier, ScrollBehavior,
    # ... more type imports ...
)
"""

with Path("Browser/browser.pyi").open("w", encoding="utf-8") as stub_file:
    stub_file.write(pyi_boilerplate)
    with Path("mypy_stub/Browser/browser.pyi").open("r", encoding="utf-8") as init_file:
        stub_file.write(init_file.read())
    stub_file.write("\n")
    for line in parse_kw_stubs():
        stub_file.write(line)

The output file Browser/browser.pyi is assembled from three parts:

Boilerplate -- License header and import statements for all types used in keyword signatures
Browser class stub -- The Browser class definition from mypy_stub/Browser/browser.pyi, containing the __init__ signature
Keyword method stubs -- Individual method signatures extracted from mypy_stub/Browser/keywords/*.pyi

Step 5: Fix robotlibcore Import

lines = []
with Path("Browser/browser.pyi").open("r", encoding="utf-8") as stub_file:
    for line in stub_file.readlines():
        if "from robotlibcore import DynamicCore" in line:
            lines.append(
                "from robotlibcore import DynamicCore # type: ignore\n"
            )
        else:
            lines.append(line)
with Path("Browser/browser.pyi").open("w", encoding="utf-8") as stub_file:
    stub_file.writelines(lines)

A post-processing step adds a # type: ignore comment to the robotlibcore import, since robotlibcore does not ship type stubs and would otherwise cause type checker warnings.

tasks.py: docs Task

The docs task in tasks.py (lines 842-883) wraps libdoc with additional processing:

@task
def docs(c, version=None):
    """Generate library keyword documentation.

    Args:
        version: Creates keyword documentation with version
        suffix in the name. Documentation is moved to docs/versions
        folder.
    """
    output = ROOT_DIR / "docs" / "Browser.html"
    libdoc("Browser", str(output))
    # ... HTML post-processing with BeautifulSoup ...
    if version is not None:
        target = (
            ROOT_DIR / "docs" / "versions"
            / f"Browser-{version.replace('v', '')}.html"
        )
        output.rename(target)

The task:

Calls libdoc("Browser", output) to generate the base HTML
Post-processes the HTML with BeautifulSoup to inject Google Analytics tracking scripts
Optionally moves the output to a versioned location if a version argument is provided

File Structure

File	Purpose
`mypy_stub/Browser/browser.pyi`	Hand-written stub for the `Browser` class `__init__` and class-level definitions
`mypy_stub/Browser/keywords/*.pyi`	Hand-written stubs for each keyword module (one file per module)
`Browser/gen_stub.py`	Script that combines stubs into the final `Browser/browser.pyi`
`Browser/browser.pyi`	Generated combined type stub file used by IDEs
`docs/Browser.html`	Generated HTML keyword documentation
`docs/versions/Browser-{ver}.html`	Generated versioned documentation snapshots
`tasks.py`	Invoke task definitions including the `docs` task

Generating Documentation with Plugins

To generate documentation that includes plugin keywords, use the ::plugins= syntax with libdoc:

# Command line
python -m robot.libdoc "Browser::plugins=path/to/MyPlugin.py" output.html

# Multiple plugins
python -m robot.libdoc "Browser::plugins=PluginA.py,PluginB.py" output.html

# Plugin with arguments
python -m robot.libdoc "Browser::plugins=mypackage.MyPlugin;arg1" output.html

# Programmatic
from robot.libdoc import libdoc

libdoc("Browser::plugins=path/to/MyPlugin.py", "output.html")

Plugin keywords will appear in the generated HTML alongside built-in keywords. They are identifiable by the Plugin tag.

Limitations

The gen_stub.py script only generates stubs for built-in keywords (it instantiates Browser() without plugins)
Plugin keyword stubs are not included in the generated Browser/browser.pyi file
For IDE support with plugin keywords, plugin authors must provide their own .pyi stub files or rely on IDE runtime introspection
The docs task in tasks.py also does not include plugins by default; the user must modify the libdoc call to include plugins

MarketSquare_Robotframework_browser_Plugin_Documentation_Generation -- Principle: documentation generation for plugins
MarketSquare_Robotframework_browser_PluginParser_Parse_Plugins -- How plugins are loaded (required for libdoc to discover plugin keywords)
MarketSquare_Robotframework_browser_Keyword_Decorator_Pattern -- How keyword docstrings and tags feed into documentation

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment