Implementation:Confident ai Deepeval Changelog Generator
| Knowledge Sources | |
|---|---|
| Domains | Developer_Tooling, Release_Management |
| Last Updated | 2026-02-14 09:30 GMT |
Overview
CLI script that automatically generates and maintains per-year MDX changelog files by extracting merged pull requests from git history, enriching them via the GitHub API, and optionally producing AI-written release notes.
Description
The Changelog Generator is a standalone Python script located at `.scripts/changelog/generate.py` that automates the creation of formatted changelog entries for the DeepEval project. It parses git tags and merge commits to identify PRs in each release, fetches PR titles and bodies from the GitHub API, and optionally uses an LLM (via DeepEval's `GPTModel`) to produce human-readable changelog bullets classified into categories such as New Feature, Bug Fix, Improvement, Backward Incompatible Change, Experimental Feature, and Security. The output is merged into existing per-year MDX changelog files organized by month, category, and version.
The script defines several key data types: `Commit` and `Pull` dataclasses for git/GitHub data, `AiReleaseNote` (Pydantic model) for LLM-generated structured entries, and `AiMonthSummary` for monthly prose summaries. It uses Rich for progress bar UI and supports features like tag ranges, year-based generation, ignore lists, dry-run mode, and contributor attribution.
Usage
Run this script from the repository root when preparing release notes for new versions. It is typically invoked as part of the release process to update the documentation site's changelog section.
Code Reference
Source Location
- Repository: Confident_ai_Deepeval
- File: .scripts/changelog/generate.py
- Lines: 1-1505
Signature
def main() -> int:
"""
CLI entry point. Parses arguments and orchestrates changelog generation.
Supported modes:
--latest Process only the most recent tag
--tag TAG Process a single tag
--range FROM TO Process all tags between FROM and TO
--year YEAR Process all tags for a given year
Options:
--output-dir DIR Directory for output MDX files (default: docs/changelog)
--ai Enable AI-generated release notes via LLM
--ai-model MODEL LLM model name (default: gpt-4.1-mini)
--dry-run Preview without writing files
"""
Import
# This is a standalone script, not an importable module.
# Run from repository root:
# python .scripts/changelog/generate.py --latest
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| --latest | flag | No | Process only the most recent git tag |
| --tag | str | No | Process a single specified git tag |
| --range | (str, str) | No | Process tags between FROM and TO (inclusive) |
| --year | int | No | Process all tags for a given year |
| --output-dir | str | No | Output directory for MDX files (default: docs/changelog) |
| --ai | flag | No | Enable AI-generated release note entries |
| --ai-model | str | No | LLM model name for AI mode (default: gpt-4.1-mini) |
| --dry-run | flag | No | Preview changes without writing files |
| GITHUB_TOKEN | env var | No | GitHub API token for fetching PR data (avoids rate limits) |
Outputs
| Name | Type | Description |
|---|---|---|
| changelog-{year}.mdx | File | Per-year MDX changelog files written to output directory |
| return code | int | 0 on success, non-zero on failure |
Usage Examples
Generate Changelog for Latest Release
# Generate changelog entry for the most recent tag
python .scripts/changelog/generate.py --latest
# With AI-generated release notes
python .scripts/changelog/generate.py --latest --ai
# Dry run to preview without writing
python .scripts/changelog/generate.py --latest --ai --dry-run
Generate Changelog for a Year
# Generate changelog for all 2025 releases
python .scripts/changelog/generate.py --year 2025 --ai
# Generate for a specific tag range
python .scripts/changelog/generate.py --range v1.0.0 v2.0.0 --ai