Implementation:Ray project Ray Custom Sphinx Directives

Knowledge Sources	Ray
Domains	Documentation, Build_System, Sphinx_Extensions
Last Updated	2026-02-13

Overview

Custom Sphinx Directives is a comprehensive Python module that provides custom Sphinx directives, build hooks, and content management utilities for the Ray documentation site.

Description

This module (custom_directives.py) serves as the backbone of Ray's custom documentation infrastructure, extending Sphinx with project-specific features. It defines the DownloadAndPreprocessEcosystemDocs class for fetching and preprocessing external ecosystem library documentation from GitHub, the LinkcheckSummarizer for aggregating and reporting broken links at the end of builds, and navbar parsing utilities that read navbar.yml to generate the site navigation structure. The module also implements an example gallery system that generates pages from YAML configurations, a sidebar navigation caching mechanism via preload_sidebar_nav, and page context update hooks that add feedback form URLs to each page. It tracks git-managed files to control "Edit on GitHub" link visibility and manages version JSON generation for the ReadTheDocs version switcher.

Usage

This module is imported by conf.py and its functions are connected to Sphinx build lifecycle events. Modify it when changing the site navigation structure, adding new example gallery configurations, updating ecosystem documentation sources, or customizing the link checking behavior. The exported symbols include DownloadAndPreprocessEcosystemDocs, update_context, LinkcheckSummarizer, setup_context, and collect_example_orphans.

Code Reference

Source Location

doc/source/custom_directives.py

Signature

__all__ = [
    "DownloadAndPreprocessEcosystemDocs",
    "update_context",
    "LinkcheckSummarizer",
    "setup_context",
    "collect_example_orphans",
]

class DownloadAndPreprocessEcosystemDocs:
    """Downloads and preprocesses ecosystem library docs from GitHub."""
    def __init__(self, base_path: str) -> None: ...
    @staticmethod
    def get_latest_release_tag(repo: str) -> str: ...
    @staticmethod
    def get_file_from_github(repo, ref, path_to_get_from_repo, path_to_save_on_disk) -> None: ...
    def write_new_docs(self, *args, **kwargs): ...
    def write_original_docs(self, *args, **kwargs): ...

class LinkcheckSummarizer:
    """Hook into the linkcheck logger to display a summary at the end."""
    def add_handler_to_linkcheck(self, *args, **kwargs): ...
    def summarize(self, *args, **kwargs): ...

def feedback_form_url(project, page) -> str: ...
def update_context(app, pagename, templatename, context, doctree): ...
def parse_navbar_config(app, config): ...
def preload_sidebar_nav(get_toctree, pathto, root_doc, pagename): ...
def setup_context(app, pagename, templatename, context, doctree): ...
def collect_example_orphans(confdir, srcdir) -> set: ...

Import

from custom_directives import (
    DownloadAndPreprocessEcosystemDocs,
    update_context,
    LinkcheckSummarizer,
    parse_navbar_config,
    setup_context,
    pregenerate_example_rsts,
    generate_versions_json,
    collect_example_orphans,
)

I/O Contract

Inputs

Input	Type	Description
base_path	str/Path	Absolute path to the documentation source directory
navbar.yml	YAML file	Navigation configuration defining site menu structure
EXAMPLE_GALLERY_CONFIGS	list of str	Paths to YAML files defining example galleries (data, serve, train)
EXTERNAL_MARKDOWN_FILES	list of tuples	External docs to download: (repo, ref, path, save_path)
FEEDBACK_FORM_FMT	str	GitHub issue URL template for feedback forms
TRACKED_FILES	list of str	Git-tracked files used to determine "Edit on GitHub" visibility
app (Sphinx)	sphinx.application.Sphinx	Sphinx application instance providing build context
config (Sphinx)	sphinx.config.Config	Sphinx configuration providing settings like navbar_content_path

Outputs

Output	Type	Description
Page context updates	dict	Adds feedback_form_url and navigation data to page rendering context
Sidebar HTML	bs4.BeautifulSoup	Cached and formatted sidebar navigation HTML
Preprocessed ecosystem docs	.md files	Downloaded and preprocessed markdown files from external repos
Example gallery RST files	.rst files	Pre-generated RST pages for example galleries
Broken links summary	log output	Summary of broken links found during link checking
versions.json	JSON file	Version metadata for the ReadTheDocs version switcher
Orphan document set	set of str	Set of document names to be marked as orphans

Usage Examples

Using DownloadAndPreprocessEcosystemDocs to manage external documentation:

from custom_directives import DownloadAndPreprocessEcosystemDocs

base_path = pathlib.Path(__file__).parent
github_docs = DownloadAndPreprocessEcosystemDocs(base_path)

# Connected to Sphinx events in conf.py:
app.connect("builder-inited", github_docs.write_new_docs)
app.connect("build-finished", github_docs.write_original_docs)

Setting up link check summarization:

from custom_directives import LinkcheckSummarizer

linkcheck_summarizer = LinkcheckSummarizer()
app.connect("builder-inited", linkcheck_summarizer.add_handler_to_linkcheck)
app.connect("build-finished", linkcheck_summarizer.summarize)

Connecting context update hooks:

from custom_directives import update_context, parse_navbar_config, setup_context

app.connect("html-page-context", update_context)
app.add_config_value("navbar_content_path", "navbar.yml", "env")
app.connect("config-inited", parse_navbar_config)
app.connect("html-page-context", setup_context)

Related Pages

• Ray_project_Ray_Sphinx_Configuration - Main Sphinx conf.py that imports and uses these directives
• Ray_project_Ray_Sphinx_Callouts_Extension - Custom callout extension complementing these directives
• Ray_project_Ray_PyTorch_Hyperparameter_Tuning_Tutorial - Example tutorial rendered through this build system
• Ray - Ray project repository