Heuristic:Junyanz Pytorch CycleGAN and pix2pix Test Train Option Consistency
| Knowledge Sources | |
|---|---|
| Domains | Debugging, GAN_Training |
| Last Updated | 2026-02-09 16:00 GMT |
Overview
Explicitly specify `--netG`, `--norm`, `--no_dropout`, and `--dataset_mode` at test time to match training configuration, because different models use different defaults.
Description
A common source of `RuntimeError: Error(s) in loading state_dict` errors is a mismatch between training and test configurations. Different model types (cycle_gan, pix2pix, test) override different default values for generator architecture, normalization, and dropout. If you train with one set of defaults and test with another, the saved weights will not match the constructed network architecture. The codebase modifies defaults per model via `modify_commandline_options()`, so implicit defaults change depending on `--model`.
Usage
Apply this heuristic every time you run test.py. Always explicitly pass the same `--netG`, `--norm`, `--no_dropout` (or not), and `--dataset_mode` flags that were used during training.
The Insight (Rule of Thumb)
- Action: Explicitly specify architecture flags in both training and test scripts.
- Value:
- CycleGAN defaults: `--netG resnet_9blocks --no_dropout --norm instance --dataset_mode unaligned`
- pix2pix defaults: `--netG unet_256 --norm batch --dataset_mode aligned`
- Test model defaults: `--netG resnet_9blocks --norm instance --dataset_mode single`
- Trade-off: Verbose command lines, but prevents cryptic state_dict loading errors.
- Key checks: (1) `--netG` architecture, (2) `--norm` type, (3) `--no_dropout` flag.
Reasoning
Each model type calls `modify_commandline_options()` to set its own defaults. For example, pix2pix sets `--netG unet_256` while CycleGAN keeps `--netG resnet_9blocks`. If you train a pix2pix model and then test with `--model test` (which defaults to `resnet_9blocks`), the network architecture won't match the saved weights. Explicit flags prevent this silent default switching.
From `docs/qa.md`:
"If you get the above errors when loading the generator during test time, you probably have used different network configurations for training and test. [...] Make sure that the flag is the same. [...] To make sure that your training and test follow the same setting, you are encouraged to explicitly specify the --netG, --norm, --dataset_mode, and --no_dropout (or not) in your script."
Code evidence showing pix2pix overrides defaults from `models/pix2pix_model.py:32-36`:
parser.set_defaults(netG="unet_256", dataset_mode="aligned")
# pix2pix uses batch norm by default (different from CycleGAN's instance norm)
parser.set_defaults(norm="batch")
Code evidence showing CycleGAN overrides from `models/cycle_gan_model.py:40`:
parser.set_defaults(no_dropout=True)
# CycleGAN default: --netG resnet_9blocks --norm instance (inherited from base)
InstanceNorm checkpoint patching for legacy compatibility from `models/base_model.py:209-219`:
def __patch_instance_norm_state_dict(self, state_dict, module, keys, i=0):
"""Fix InstanceNorm checkpoints incompatibility (prior to 0.4)"""
key = keys[i]
if i + 1 == len(keys):
if module.__class__.__name__.startswith("InstanceNorm") and \
(key == "running_mean" or key == "running_var"):
if getattr(module, key) is None:
state_dict.pop(".".join(keys))