Heuristic:Iterative Dvc Shell Execution Pitfalls
| Knowledge Sources | |
|---|---|
| Domains | Debugging, Shell_Integration |
| Last Updated | 2026-02-10 10:00 GMT |
Overview
Shell execution best practices to prevent environment variable contamination from shell config files during DVC stage execution.
Description
When DVC runs pipeline stage commands, it spawns a subprocess using the user's default shell (from the `SHELL` environment variable). Without precautions, the shell may read its config files (`.bashrc`, `.zshrc`, `config.fish`), which can modify environment variables and alter the behavior of the executed command. DVC mitigates this by passing `--noprofile --norc` (bash), `--no-rcs` (zsh), or `--no-config` (fish >= 3.3.0) flags to suppress config loading.
Usage
Be aware of this heuristic when debugging unexpected behavior in pipeline stages or when stages produce different results on different machines. If a stage command depends on environment variables, those variables may be overwritten by shell configs on some systems but not others.
The Insight (Rule of Thumb)
- Action: DVC automatically adds shell suppression flags (`--noprofile`, `--norc`, `--no-rcs`, `--no-config`) when executing stage commands.
- Action: On Windows, `shell=True` is used with `cmd.exe` (no suppression needed). On Unix, `shell=False` with explicit shell invocation.
- Value: Reproducible stage execution across environments.
- Trade-off: Shell aliases and functions defined in config files are not available in stage commands. Users must use full command paths.
- Compatibility: Fish shell below version 3.3.0 does not support `--no-config`, and DVC emits a warning for this case.
Reasoning
The root cause is documented in GitHub issues #1307 and #2506. Users reported that `dvc repro` produced different results than running the same command manually because their shell config files modified `PATH`, `PYTHONPATH`, or other variables. The fix ensures DVC stage execution is deterministic regardless of the user's shell configuration.
The fish shell case is particularly tricky: only fish >= 3.3.0 supports `--no-config`. DVC checks the fish version at runtime and warns users on older versions that their `config.fish` may interfere with stage execution.
Code Evidence
Shell suppression from `dvc/stage/run.py:58-70`:
def _make_cmd(executable, cmd):
if executable is None:
return cmd
opts = {
"zsh": ["--no-rcs"],
"bash": ["--noprofile", "--norc"],
"fish": [],
}
name = os.path.basename(executable).lower()
opt = opts.get(name, [])
if name == "fish" and _fish_supports_no_config(executable):
opt.append("--no-config")
return [executable, *opt, "-c", cmd]
Fish version check from `dvc/stage/run.py:18-39`:
@cache
def _fish_supports_no_config(executable) -> bool:
try:
output = subprocess.check_output(
[executable, "--version"], text=True,
)
version = Version(output.split(" ")[-1].strip())
version_to_check = Version("3.3.0")
return version >= version_to_check
except (subprocess.CalledProcessError, IndexError, InvalidVersion):
logger.trace("could not check fish version, defaulting to False")
return False
Fish warning from `dvc/stage/run.py:42-55`:
def _warn_if_fish(executable):
if (
executable is None
or os.path.basename(executable) != "fish"
or _fish_supports_no_config(executable)
):
return
logger.warning(
"DVC detected that you are using a version of fish shell below 3.3.0 "
"Be aware that it might cause problems by overwriting "
"your current environment variables with values defined "
"in 'config.fish', which might affect your command. See "
"https://github.com/treeverse/dvc/issues/1307. "
)
Windows vs Unix shell handling from `dvc/stage/run.py:104,112-113`:
kwargs["shell"] = os.name == "nt"
def get_executable():
return (os.getenv("SHELL") or "/bin/sh") if os.name != "nt" else None