Implementation:Kornia Kornia Focal Loss

Knowledge Sources	Kornia
Domains	Vision, Loss_Functions
Last Updated	2026-02-09 15:00 GMT

Overview

Focal Loss is a classification loss function designed to address class imbalance by down-weighting easy examples and focusing training on hard negatives.

Description

Focal Loss was introduced by Lin et al. (2018) to address the extreme foreground-background class imbalance encountered in dense object detection. The loss modifies the standard cross-entropy loss by adding a modulating factor that reduces the relative loss for well-classified examples, putting more focus on hard, misclassified examples.

The mathematical formulation is:

$\text{FL}(p_t)=-{\alpha }_t(1-p_t{)}^{\gamma }\log (p_t)$

Where $p_t$ is the model's estimated probability for each class, $\alpha$ is a weighting factor, and $\gamma$ is the focusing parameter. When $\gamma =0$, Focal Loss is equivalent to cross-entropy loss.

This module provides two variants:

• FocalLoss: Multi-class focal loss using softmax.
• BinaryFocalLossWithLogits: Binary focal loss operating on logits using sigmoid.

Usage

Import this loss when training classification models with significant class imbalance, such as object detection networks where background samples vastly outnumber foreground samples. It is also useful for any segmentation or classification task where easy negatives dominate the loss.

Code Reference

Source Location

• Repository: Kornia
• File: kornia/losses/focal.py
• Lines: 1-373

Signature

# Functional API
def focal_loss(
    pred: torch.Tensor,
    target: torch.Tensor,
    alpha: Optional[float],
    gamma: float = 2.0,
    reduction: str = "none",
    weight: Optional[torch.Tensor] = None,
    ignore_index: Optional[int] = -100,
) -> torch.Tensor: ...

# Module API
class FocalLoss(nn.Module):
    def __init__(
        self,
        alpha: Optional[float],
        gamma: float = 2.0,
        reduction: str = "none",
        weight: Optional[torch.Tensor] = None,
        ignore_index: Optional[int] = -100,
    ) -> None: ...
    def forward(self, pred: torch.Tensor, target: torch.Tensor) -> torch.Tensor: ...

# Binary Functional API
def binary_focal_loss_with_logits(
    pred: torch.Tensor,
    target: torch.Tensor,
    alpha: Optional[float] = 0.25,
    gamma: float = 2.0,
    reduction: str = "none",
    pos_weight: Optional[torch.Tensor] = None,
    weight: Optional[torch.Tensor] = None,
    ignore_index: Optional[int] = -100,
) -> torch.Tensor: ...

# Binary Module API
class BinaryFocalLossWithLogits(nn.Module):
    def __init__(
        self,
        alpha: Optional[float],
        gamma: float = 2.0,
        reduction: str = "none",
        pos_weight: Optional[torch.Tensor] = None,
        weight: Optional[torch.Tensor] = None,
        ignore_index: Optional[int] = -100,
    ) -> None: ...
    def forward(self, pred: torch.Tensor, target: torch.Tensor) -> torch.Tensor: ...

Import

from kornia.losses import FocalLoss
from kornia.losses import BinaryFocalLossWithLogits
from kornia.losses import focal_loss
from kornia.losses import binary_focal_loss_with_logits

I/O Contract

Inputs (FocalLoss)

Name	Type	Required	Description
alpha	Optional[float]	Yes	Weighting factor in [0, 1] for balancing classes
gamma	float	No	Focusing parameter (default: 2.0); higher values increase focus on hard examples
reduction	str	No	Reduction mode: 'none', 'mean', or 'sum' (default: 'none')
weight	Optional[torch.Tensor]	No	Per-class weights with shape (num_classes,)
ignore_index	Optional[int]	No	Label value to ignore in loss computation (default: -100)
pred	torch.Tensor	Yes	Logits tensor with shape (N, C, *) where C = number of classes
target	torch.Tensor	Yes	Labels tensor with shape (N, *) with integer class indices

Inputs (BinaryFocalLossWithLogits)

Name	Type	Required	Description
alpha	Optional[float]	No	Weighting factor (default: 0.25)
gamma	float	No	Focusing parameter (default: 2.0)
reduction	str	No	Reduction mode: 'none', 'mean', or 'sum' (default: 'none')
pos_weight	Optional[torch.Tensor]	No	Weight of positive examples with shape (num_classes,)
weight	Optional[torch.Tensor]	No	Per-class weights with shape (num_classes,)
ignore_index	Optional[int]	No	Label value to ignore (default: -100)
pred	torch.Tensor	Yes	Logits tensor with shape (N, C, *)
target	torch.Tensor	Yes	Labels tensor with same shape as pred, values between 0 and 1

Outputs

Name	Type	Description
loss	torch.Tensor	Computed focal loss; shape depends on reduction: scalar for 'mean'/'sum', or same spatial shape as input for 'none'

Usage Examples

import torch
from kornia.losses import FocalLoss, BinaryFocalLossWithLogits

# Multi-class focal loss
C = 5  # num_classes
pred = torch.randn(1, C, 3, 5, requires_grad=True)
target = torch.randint(C, (1, 3, 5))
criterion = FocalLoss(alpha=0.5, gamma=2.0, reduction='mean')
output = criterion(pred, target)
output.backward()

# Binary focal loss
C = 3
pred_bin = torch.randn(1, C, 5, requires_grad=True)
target_bin = torch.randint(2, (1, C, 5))
criterion_bin = BinaryFocalLossWithLogits(alpha=0.25, gamma=2.0, reduction='mean')
output_bin = criterion_bin(pred_bin, target_bin)
output_bin.backward()

Related Pages

• Principle:Kornia_Kornia_Focal_Loss_Function