Implementation:Open compass VLMEvalKit SArena FID

Field	Value
source	VLMEvalKit
domain	Vision, Evaluation, Image Generation, FID

Overview

Calculates Frechet Inception Distance (FID) and FID-C scores for evaluating image generation quality in the SArena benchmark.

Description

The `FIDCalculator` class extends `BaseMetric` to compute FID scores using either InceptionV3 (standard FID) or CLIP ViT-B/32 (FID-C). The `calculate_frechet_distance` method computes the Frechet distance between two multivariate Gaussians derived from feature activations of generated and reference image sets. It uses scipy's matrix square root for stable computation with epsilon correction for singular covariance matrices.

Usage

Called internally by the corresponding dataset class during evaluation.

Code Reference

• Source: vlmeval/dataset/utils/SArena/FID.py, Lines: L1-152
• Import: from vlmeval.dataset.utils.SArena.FID import FIDCalculator

Key Functions:

class FIDCalculator(BaseMetric):
    def calculate_frechet_distance(self, mu1, sigma1, mu2, sigma2, eps=1e-6): ...
    def calculate_score(self, batch, batch_size=64, update=True): ...

I/O Contract

Direction	Description
Inputs	A batch dict with 'pred_im' (predicted images) and 'gt_im' (ground truth images)
Outputs	Float FID score (lower is better) representing distribution distance between generated and real images

Usage Examples

from vlmeval.dataset.utils.SArena.FID import FIDCalculator

calc = FIDCalculator(model_name='InceptionV3')
fid_score = calc.calculate_score(batch)

Related Pages

• Principle:Open_compass_VLMEvalKit_Benchmark_Dataset_Construction