Implementation:Iterative Dvc Repo Imp Url

Knowledge Sources	Iterative_Dvc
Domains	Data_Import, Remote_Storage
Last Updated	2026-02-10 10:00 GMT

Overview

The Repo_Imp_Url implementation imports data from a URL into a DVC-tracked pipeline stage. It resides in dvc/repo/imp_url.py (90 lines) and is the core logic behind the dvc import-url command.

from dvc.repo.imp_url import imp_url

Function Signature

@locked
@scm_context
def imp_url(
    self: "Repo",
    url,
    out=None,
    erepo=None,
    frozen=True,
    no_download=False,
    no_exec=False,
    remote=None,
    to_remote=False,
    jobs=None,
    force=False,
    fs_config=None,
    version_aware: bool = False,
):

Parameters

Parameter	Type	Default	Description
self	Repo	N/A	The DVC repository instance
url	str	required	Source URL to import data from
out	str or None	None	Local output path; defaults to the basename of the URL
erepo	dict or None	None	External repository configuration for imports from other DVC repos
frozen	bool	True	Whether the stage should be frozen (not re-executed on dvc repro)
no_download	bool	False	Create the stage without downloading the data
no_exec	bool	False	Create the stage without executing or downloading
remote	str or None	None	Target remote for --to-remote transfers
to_remote	bool	False	Transfer data directly to a remote instead of local storage
jobs	int or None	None	Number of parallel jobs for the transfer
force	bool	False	Overwrite the output if it already exists
fs_config	dict or None	None	Filesystem-specific configuration overrides
version_aware	bool	False	Enable version-aware imports (e.g., for cloud storage versioned buckets)

Return Value

Returns the created Stage object representing the URL import pipeline stage.

Internal Mechanics

Argument Validation

The function enforces several constraints on parameter combinations:

if to_remote and (no_exec or no_download or version_aware):
    raise InvalidArgumentError(
        "--no-exec/--no-download/--version-aware cannot be combined with --to-remote"
    )

if not to_remote and remote:
    raise InvalidArgumentError("--remote can't be used without --to-remote")

In-Repository URL Detection

When importing from a path within the same repository, the URL is converted to a relative path:

if (
    erepo is None
    and os.path.exists(url)
    and path_isin(os.path.abspath(url), self.root_dir)
):
    url = relpath(url, wdir)

Version-Aware Configuration

When version_aware=True, the filesystem configuration is updated to enable version tracking:

if version_aware:
    if fs_config is None:
        fs_config = {}
    fs_config["version_aware"] = True

Stage Creation

A single-stage pipeline is created with the URL as a dependency:

stage = self.stage.create(
    single_stage=True,
    validate=False,
    fname=path,
    wdir=wdir,
    deps=[url],
    outs=[out],
    erepo=erepo,
    fs_config=fs_config,
)

Execution Modes

The function supports three distinct execution modes:

Mode	Condition	Behavior
No execution	no_exec=True	Stage outputs are ignored (not tracked); no data is fetched
To remote	to_remote=True	Data is transferred directly to a remote ODB; outputs are locally ignored; dependency metadata is saved
Standard	Neither flag set	Stage is run normally; for version-aware sources, can_push is disabled on outputs

if no_exec:
    stage.ignore_outs()
elif to_remote:
    remote_odb = self.cloud.get_remote_odb(remote, "import-url")
    stage.outs[0].transfer(url, odb=remote_odb, jobs=jobs)
    stage.outs[0].ignore()
    stage.save_deps()
    stage.md5 = stage.compute_md5()
else:
    if stage.deps[0].fs.version_aware:
        stage.outs[0].can_push = False
    stage.run(jobs=jobs, no_download=no_download)

Persistence

After execution, the stage is frozen (if frozen=True) and saved to disk:

stage.frozen = frozen
stage.dump()
return stage

Usage Example

from dvc.repo import Repo

with Repo() as repo:
    # Import from an HTTP URL
    stage = repo.imp_url("https://example.com/data.csv")

    # Import directly to a remote
    stage = repo.imp_url(
        "s3://bucket/large_dataset/",
        to_remote=True,
        remote="myremote",
        jobs=8,
    )

    # Create a stage without downloading data
    stage = repo.imp_url(
        "gs://bucket/model.pkl",
        out="model.pkl",
        no_download=True,
    )

Decorators

Decorator	Purpose
@locked	Ensures the repository lock is held during execution
@scm_context	Manages SCM (Git) context for tracking generated files

Dependencies

Module	Purpose
dvc.exceptions.InvalidArgumentError	Raised for invalid parameter combinations
dvc.exceptions.OutputDuplicationError	Raised when stage output conflicts with an existing stage
dvc.repo.scm_context	Decorator for SCM-aware operations
dvc.utils.resolve_output	Resolves the local output path
dvc.utils.resolve_paths	Resolves path, working directory, and output for stage creation
dvc.utils.fs.path_isin	Checks if a path is inside a given directory

See Also

• Repo_Imp_Db -- Imports data from database sources
• Repo_Get -- Downloads data without creating a pipeline stage