Implementation:Open compass VLMEvalKit UniSVG Utils
| Field | Value |
|---|---|
| source | VLMEvalKit |
| domain | Vision, Evaluation, SVG Understanding, Multi-metric Scoring |
Overview
Provides multi-metric evaluation utilities for the UniSVG benchmark, using LPIPS, CLIP, Sentence-BERT, and BERTScore for comprehensive SVG quality assessment.
Description
This module implements lazy-loaded deep learning models (_init_models) for multi-dimensional SVG quality evaluation. It initializes LPIPS (using AlexNet backbone) for perceptual distance, CLIP (ViT-Large-Patch14-336) for vision-language similarity, Sentence-BERT (all-MiniLM-L6-v2) for text embedding similarity, and BERTScore for token-level semantic matching. The evaluation pipeline handles SVG-to-image rendering, image preprocessing, and GPU/CPU device management. Models are loaded on demand with global singleton pattern to avoid redundant initialization across evaluation calls.
Usage
Called internally by the UniSVG dataset class during SVG quality evaluation.
Code Reference
- Source:
vlmeval/dataset/utils/uni_svg.py, Lines: L1-621 - Import:
from vlmeval.dataset.utils.uni_svg import _init_models
Key Functions:
def _init_models(): ...
# Global model singletons
lpips_model = None
clip_model = None
clip_processor = None
sbert_model = None
bert_scorer = None
I/O Contract
| Direction | Description |
|---|---|
| Inputs | Generated SVG content strings or rendered images; reference SVG/images for comparison |
| Outputs | Multi-metric score dictionary containing LPIPS distance, CLIP similarity, SBERT similarity, and BERTScore F1 |
Usage Examples
# Internal usage example
from vlmeval.dataset.utils.uni_svg import _init_models
device = _init_models() # Initializes all evaluation models