Implementation:Open compass VLMEvalKit SArena LPIPS

Field	Value
source	VLMEvalKit
domain	Vision, Evaluation, Image Generation, Perceptual Similarity

Overview

Calculates LPIPS (Learned Perceptual Image Patch Similarity) scores for evaluating perceptual image quality in the SArena benchmark.

Description

The `LPIPSCalculator` class extends `BaseMetric` to compute perceptual similarity using a VGG-based LPIPS model. It normalizes images with ImageNet statistics and processes them in batches via DataLoader. The LPIPS metric measures perceptual difference between predicted and ground truth images, where lower values indicate higher perceptual similarity.

Usage

Called internally by the corresponding dataset class during evaluation.

Code Reference

• Source: vlmeval/dataset/utils/SArena/LPIPS.py, Lines: L1-53
• Import: from vlmeval.dataset.utils.SArena.LPIPS import LPIPSCalculator

Key Functions:

class LPIPSCalculator(BaseMetric):
    def LPIPS(self, tensor_image1, tensor_image2): ...
    def calculate_score(self, batch, batch_size=8, update=True): ...

I/O Contract

Direction	Description
Inputs	A batch dict with 'gt_im' (ground truth images) and 'pred_im' (predicted images) as PIL Images
Outputs	Tuple of (average_lpips_score, list_of_per_sample_scores); lower scores indicate better perceptual similarity

Usage Examples

from vlmeval.dataset.utils.SArena.LPIPS import LPIPSCalculator

calc = LPIPSCalculator()
avg_score, scores = calc.calculate_score(batch)

Related Pages

• Principle:Open_compass_VLMEvalKit_Benchmark_Dataset_Construction