Implementation:PeterL1n BackgroundMattingV2 MattingRefine

Knowledge Sources	BackgroundMattingV2 Real-Time High-Resolution Background Matting
Domains	Image_Matting, Computer_Vision, Deep_Learning
Last Updated	2026-02-09 00:00 GMT

Overview

Concrete tool for high-resolution matting with selective patch refinement provided by model/model.py.

Description

MattingRefine extends MattingBase by adding a Refiner module that selectively refines error-prone patches at full resolution. The base network processes downsampled inputs (controlled by backbone_scale, default 1/4), and the refiner operates only on patches identified by the error map. This achieves high-resolution output with minimal computational overhead.

Three refinement modes are supported:

full — refine the entire image (highest quality, slowest)
sampling — refine a fixed number of top-error pixels (used for training)
thresholding — refine pixels above an error threshold (used for inference)

Compatibility options for patch operations (patch_crop_method, patch_replace_method) enable deployment to TorchScript and ONNX runtimes.

Usage

Use for high-resolution matting inference (HD, 4K) where fine details like hair strands are important. For training, use sampling mode; for inference, use thresholding mode.

Code Reference

Source Location

Repository: BackgroundMattingV2
File: model/model.py
Lines: 101-196

Signature

class MattingRefine(MattingBase):
    """
    High-resolution matting with selective refinement.
    """
    def __init__(
        self,
        backbone: str,
        backbone_scale: float = 1/4,
        refine_mode: str = 'sampling',
        refine_sample_pixels: int = 80_000,
        refine_threshold: float = 0.1,
        refine_kernel_size: int = 3,
        refine_prevent_oversampling: bool = True,
        refine_patch_crop_method: str = 'unfold',
        refine_patch_replace_method: str = 'scatter_nd'
    ):
        """
        Args:
            backbone: 'resnet50', 'resnet101', or 'mobilenetv2'
            backbone_scale: Downsample factor for backbone (must be <= 0.5)
            refine_mode: 'full', 'sampling', or 'thresholding'
            refine_sample_pixels: Pixels to refine in sampling mode
            refine_threshold: Error threshold for thresholding mode (0-1)
            refine_kernel_size: Refiner conv kernel size (1 or 3)
            refine_prevent_oversampling: Prevent sampling more than needed
            refine_patch_crop_method: 'unfold', 'roi_align', or 'gather'
            refine_patch_replace_method: 'scatter_nd' or 'scatter_element'
        """

    def forward(
        self,
        src: Tensor,  # (B, 3, H, W) source, RGB, 0-1
        bgr: Tensor   # (B, 3, H, W) background, RGB, 0-1
    ) -> Tuple[Tensor, Tensor, Tensor, Tensor, Tensor, Tensor]:
        """
        Returns:
            pha: (B, 1, H, W) refined alpha, 0-1
            fgr: (B, 3, H, W) refined foreground, 0-1
            pha_sm: (B, 1, Hc, Wc) coarse alpha
            fgr_sm: (B, 3, Hc, Wc) coarse foreground
            err_sm: (B, 1, Hc, Wc) coarse error map
            ref_sm: (B, 1, H/4, W/4) refinement selection map
        """

Import

from model import MattingRefine

I/O Contract

Inputs

Name	Type	Required	Description
backbone	str	Yes	Encoder backbone choice
backbone_scale	float	No	Input downsample scale (default 0.25, must be <= 0.5)
refine_mode	str	No	Refinement selection mode (default 'sampling')
refine_sample_pixels	int	No	Fixed pixel count for sampling mode (default 80000)
refine_threshold	float	No	Error threshold for thresholding mode (default 0.1)
src	Tensor[B,3,H,W]	Yes	Source image (H,W must be divisible by 4)
bgr	Tensor[B,3,H,W]	Yes	Background image (same size as src)

Outputs

Name	Type	Description
pha	Tensor[B,1,H,W]	Full-resolution refined alpha matte
fgr	Tensor[B,3,H,W]	Full-resolution refined foreground
pha_sm	Tensor[B,1,Hc,Wc]	Coarse alpha from base network
fgr_sm	Tensor[B,3,Hc,Wc]	Coarse foreground from base network
err_sm	Tensor[B,1,Hc,Wc]	Coarse error map
ref_sm	Tensor[B,1,H/4,W/4]	Refinement selection map (1 = refined patch)

Usage Examples

Training (Sampling Mode)

from model import MattingRefine

model = MattingRefine(
    backbone='resnet50',
    backbone_scale=1/4,
    refine_mode='sampling',
    refine_sample_pixels=80_000
)

pha, fgr, pha_sm, fgr_sm, err_sm, ref_sm = model(src, bgr)

Inference (Thresholding Mode)

model = MattingRefine(
    backbone='resnet50',
    backbone_scale=1/4,
    refine_mode='thresholding',
    refine_threshold=0.1
)
model.load_state_dict(torch.load('checkpoint.pth'), strict=False)
model.eval().cuda()

with torch.no_grad():
    pha, fgr = model(src.cuda(), bgr.cuda())[:2]

Related Pages

Implements Principle

Principle:PeterL1n_BackgroundMattingV2_Selective_refinement_matting

Requires Environment

Environment:PeterL1n_BackgroundMattingV2_PyTorch_CUDA

Uses Heuristics

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment