Principle:Huggingface Datasets Dataset Metadata Configuration
| Knowledge Sources | |
|---|---|
| Domains | Data_Engineering, NLP |
| Last Updated | 2026-02-14 18:00 GMT |
Overview
Configuring dataset metadata (description, citation, license, features) provides essential documentation and schema information for dataset discoverability and reproducibility.
Description
Dataset metadata configuration captures all non-data information about a dataset: its description, citation, homepage URL, license, feature schema, split information, size statistics, and version. This metadata serves multiple purposes: it documents the dataset for users, provides schema information for type-safe loading, enables dataset search and discovery on the Hub, and supports reproducibility through version tracking and download checksums. Metadata is persisted as JSON alongside the data files and is also embedded in the dataset card YAML front matter when published to the Hub.
Usage
Use dataset metadata configuration when creating datasets that will be shared, published, or used across projects. Providing accurate metadata is especially important for Hub publishing, where the description, citation, and license are displayed on the dataset card page.
Theoretical Basis
Dataset metadata follows a structured schema with fields that fall into three categories: user-provided documentation (description, citation, homepage, license), automatically computed statistics (download_size, dataset_size, splits), and schema information (features, config_name, version). The metadata object supports serialization to/from JSON and YAML, with a designated subset of fields (_INCLUDED_INFO_IN_YAML) selected for inclusion in the dataset card's YAML front matter. This separation ensures that essential metadata is always visible in the card while detailed statistics remain accessible in the full JSON.