Implementation:Iterative Dvc Repo Gc

Knowledge Sources	Iterative_Dvc
Domains	Cache_Management, Storage_Optimization
Last Updated	2026-02-10 10:00 GMT

Overview

The Repo_Gc implementation performs garbage collection on unused cache and remote objects in a DVC repository. It resides in dvc/repo/gc.py (167 lines) and is the core logic behind the dvc gc command.

from dvc.repo.gc import gc

Function Signature

@locked
def gc(
    self: "Repo",
    all_branches: bool = False,
    cloud: bool = False,
    remote: Optional[str] = None,
    with_deps: bool = False,
    all_tags: bool = False,
    all_commits: bool = False,
    all_experiments: bool = False,
    force: bool = False,
    jobs: Optional[int] = None,
    repos: Optional[list[str]] = None,
    workspace: bool = False,
    commit_date: Optional[str] = None,
    rev: Optional[str] = None,
    num: Optional[int] = None,
    not_in_remote: bool = False,
    dry: bool = False,
    skip_failed: bool = False,
):

Parameters

Parameter	Type	Default	Description
`self`	`Repo`	N/A	The DVC repository instance
`all_branches`	`bool`	`False`	Keep objects referenced by all branches
`cloud`	`bool`	`False`	Also garbage collect objects from the remote storage
`remote`	`Optional[str]`	`None`	Name of a specific remote to garbage collect; requires `cloud` or `not_in_remote`
`with_deps`	`bool`	`False`	Include dependencies when computing used objects
`all_tags`	`bool`	`False`	Keep objects referenced by all tags
`all_commits`	`bool`	`False`	Keep objects referenced by all commits
`all_experiments`	`bool`	`False`	Keep objects referenced by all experiments
`force`	`bool`	`False`	Force garbage collection
`jobs`	`Optional[int]`	`None`	Number of parallel jobs for remote operations
`repos`	`Optional[list[str]]`	`None`	Additional repository paths whose objects should also be preserved
`workspace`	`bool`	`False`	Keep objects referenced by the current workspace
`commit_date`	`Optional[str]`	`None`	Keep objects from commits after this date
`rev`	`Optional[str]`	`None`	Keep objects from a specific revision
`num`	`Optional[int]`	`None`	Number of commits from `rev` to include; requires `rev`
`not_in_remote`	`bool`	`False`	Remove only objects that are already in the remote; mutually exclusive with `cloud`
`dry`	`bool`	`False`	Perform a dry run without actually deleting objects
`skip_failed`	`bool`	`False`	Skip revisions that fail to load rather than aborting

Internal Mechanics

Argument Validation

The _validate_args helper enforces constraints:

--remote requires either --cloud or --not-in-remote.
--not-in-remote and --cloud are mutually exclusive.
At least one scope flag (-w, -a, -T, --all-experiments, --all-commits, --date, or --rev) must be set.
--num can only be used with --rev.

Object Collection

The function collects "used" object IDs from:

The current repository (self)
Any additional repositories specified in the repos parameter

All repositories are locked during the collection phase using ExitStack:

from contextlib import ExitStack

with ExitStack() as stack:
    for repo in all_repos:
        stack.enter_context(repo.lock)
    for repo in [*all_repos, self]:
        for odb, obj_ids in repo.used_objs(...).items():
            odb_to_obj_ids.setdefault(odb, set()).update(obj_ids)

Remote Object Merging

When cloud or not_in_remote is enabled, _merge_remote_obj_ids is called to consolidate remote object references. It merges default remote objects into the specific remote ODB entries for both md5 and md5-dos2unix hash schemes.

Not-In-Remote Mode

The _used_obj_ids_not_in_remote helper queries the remote to determine which object IDs already exist remotely and returns only objects whose hashes are not found on the remote, allowing selective cleanup of objects that are safely backed up.

Cache Cleanup

Garbage collection is performed by iterating over all cache schemes (using self.cache.by_scheme()) and calling dvc_data.hashfile.gc.gc for each ODB:

from dvc_data.hashfile.gc import gc as ogc

for scheme, odb in self.cache.by_scheme():
    num_removed = ogc(odb, used_obj_ids, jobs=jobs, dry=dry)

Remote Cleanup

If cloud=True, the function also performs garbage collection on the remote ODB and clears the remote index after removal.

Usage Example

from dvc.repo import Repo

with Repo() as repo:
    # Keep workspace objects, remove everything else from cache
    repo.gc(workspace=True)

    # Dry run across all branches and tags
    repo.gc(all_branches=True, all_tags=True, dry=True)

    # Clean remote storage for objects not in any branch
    repo.gc(all_branches=True, cloud=True, remote="myremote")

Dependencies

Module	Purpose
`dvc.repo.locked`	Decorator ensuring the repository lock is held during execution
`dvc.exceptions.InvalidArgumentError`	Raised when invalid flag combinations are provided
`dvc_data.hashfile.gc`	Low-level garbage collection on object databases
`dvc_data.hashfile.db.get_index`	Retrieves the index for a remote ODB to clear after GC
`contextlib.ExitStack`	Manages multiple repository locks during multi-repo collection

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment