Implementation:EvolvingLMMs Lab Lmms eval Import Utils

Knowledge Sources	EvolvingLMMs_Lab_Lmms_eval
Domains	Dependency Management, Import System
Last Updated	2026-02-14 00:00 GMT

Overview

Import Utils provides a unified set of utilities for handling optional dependencies in the evaluation framework. It includes functions for checking package availability, importing with fallbacks, requiring dependencies with helpful error messages, and creating lazy module-level imports.

Description

The module centralizes all optional-dependency logic into four complementary utilities: is_package_available for cached availability checks, optional_import for graceful import-with-fallback, require_package for enforcing dependencies with actionable installation instructions, and make_lazy_getattr for deferred module-level imports that improve startup time. Together they let the framework support many optional backends (video, TUI, API) without requiring all of them to be installed.

Usage

Use these utilities at feature entry points, model constructors, and package __init__.py files to gate functionality on available packages and provide clear guidance when a dependency is missing.

Code Reference

Source Location

• Repository: EvolvingLMMs-Lab/lmms-eval
• File: lmms_eval/imports.py
• Lines: 1--115

Key Components

is_package_available

@lru_cache(maxsize=128)
def is_package_available(package_name: str) -> bool:
    """Check if a package is installed (cached)."""
    return importlib.util.find_spec(package_name) is not None

Purpose: Efficiently check whether a package is installed, with results cached for up to 128 packages.

Parameters:

• package_name -- Name of the package to check

Returns: bool indicating whether the package is installed.

optional_import

def optional_import(
    module_name: str,
    attribute: Optional[str] = None,
    fallback: Any = None,
) -> Tuple[Any, bool]:
    try:
        module = importlib.import_module(module_name)
        if attribute is not None:
            return getattr(module, attribute), True
        return module, True
    except (ImportError, AttributeError):
        return fallback, False

Purpose: Import a module or attribute optionally, returning a fallback value if unavailable.

Parameters:

• module_name -- Full module path (e.g., "decord" or "qwen_vl_utils")
• attribute -- Optional attribute to retrieve from the module
• fallback -- Value to return if the import fails (default: None)

Returns: Tuple of (imported_object_or_fallback, is_available).

MissingOptionalDependencyError

class MissingOptionalDependencyError(ImportError):
    def __init__(
        self,
        package: str,
        extras: Optional[str] = None,
        feature: Optional[str] = None,
    ):
        if extras:
            install_cmd = f"pip install lmms_eval[{extras}]"
        else:
            install_cmd = f"pip install {package}"

        feature_msg = f" for {feature}" if feature else ""
        message = (
            f"'{package}' is required{feature_msg} but not installed. "
            f"Install with: {install_cmd}"
        )
        super().__init__(message)

Purpose: Custom exception that provides a helpful installation command when a required optional dependency is missing.

Parameters:

• package -- Name of the missing package
• extras -- Extras group name for installation (e.g., "video", "tui")
• feature -- Feature description for context in the error message

require_package

def require_package(
    package: str,
    extras: Optional[str] = None,
    feature: Optional[str] = None,
) -> None:
    """Require an optional package, raising helpful error if missing."""
    if not is_package_available(package):
        raise MissingOptionalDependencyError(package, extras, feature)

Purpose: Enforce that a package is installed, raising MissingOptionalDependencyError if not.

Parameters:

• package -- Name of the required package
• extras -- Extras group for the installation command
• feature -- Feature description for the error message

Returns: None (only returns if the package is available).

make_lazy_getattr

def make_lazy_getattr(lazy_imports: dict[str, tuple[str, str]]):
    def __getattr__(name: str) -> Any:
        if name in lazy_imports:
            module_path, attr_name = lazy_imports[name]
            module = importlib.import_module(module_path)
            return getattr(module, attr_name)
        raise AttributeError(f"module has no attribute {name!r}")

    return __getattr__

Purpose: Create a module-level __getattr__ function for lazy importing, so heavyweight dependencies are only loaded when first accessed.

Parameters:

• lazy_imports -- Dictionary mapping attribute names to (module_path, attribute_name) tuples

Returns: A __getattr__ function to assign at module level.

I/O Contract

Input	Type	Description
package_name	str	Package name for availability checking
module_name	str	Full module path for optional import
attribute	Optional[str]	Specific attribute to import from module
fallback	Any	Value returned when import fails
extras	Optional[str]	Extras group name for install command
lazy_imports	dict[str, tuple[str, str]]	Mapping of names to (module, attribute) tuples

Output	Type	Description
is_available	bool	Whether the package is installed
(object, flag)	Tuple[Any, bool]	Imported object and availability flag
__getattr__	Callable	Module-level lazy import function

Common Patterns

Feature Detection

from lmms_eval.imports import is_package_available

AVAILABLE_FEATURES = {
    "video": is_package_available("decord"),
    "tui": is_package_available("textual"),
    "api": is_package_available("fastapi"),
}

Model Dependency Gate

from lmms_eval.imports import require_package

class VideoModel:
    def __init__(self):
        require_package("decord", extras="video", feature="video model inference")
        from decord import VideoReader
        self.video_reader_class = VideoReader

Lazy Loading in __init__.py

from lmms_eval.imports import make_lazy_getattr

__getattr__ = make_lazy_getattr({
    "run_tui": ("lmms_eval.tui.cli", "main"),
    "VideoModel": ("lmms_eval.models.video", "VideoModel"),
})

Design Decisions

• LRU cache for availability -- Balances performance with memory; 128 entries is typically sufficient for the set of checked packages.
• Tuple return from optional_import -- Allows callers to both use the imported object and check availability in one call.
• Custom exception class -- Provides structured error messages with actionable installation instructions aligned with PEP 621 extras.
• Lazy loading via __getattr__ -- Standard Python pattern that defers heavyweight imports until first access, improving startup time.
• Broad exception catching in optional_import -- Catches both ImportError and AttributeError for robustness.

Dependencies

• importlib -- Dynamic module importing
• importlib.util -- Package availability checking via find_spec
• functools.lru_cache -- Caching decorator
• typing -- Type annotations

Related Pages

• Principle: EvolvingLMMs_Lab_Lmms_eval_Dependency_Management
• See Also: EvolvingLMMs_Lab_Lmms_eval_Model_Wrapper_Implementation, EvolvingLMMs_Lab_Lmms_eval_Media_Handling