Implementation:Microsoft LoRA WebNLG Dev Dataset

Knowledge Sources	Microsoft_LoRA WebNLG_Challenge
Domains	NLG, Data_to_Text, Benchmarking, RDF_to_Text
Last Updated	2026-02-10 06:00 GMT

Overview

Development split of the WebNLG Challenge 2017 dataset, containing RDF triple-to-text examples organized by DBpedia categories for evaluating natural language generation from structured data.

Description

The WebNLG development set provides 62,319 lines of JSON data containing RDF triples from DBpedia paired with human-written lexicalisations. The data is organized as a dictionary with an entries key containing a list of numbered examples. Each entry includes a category label (e.g., Airport, Astronaut, Building), a set of modified RDF triples with subject/property/object fields, the original triplesets preserving raw DBpedia formatting, and multiple lexicalisations rated as "good" or other quality levels. Categories are divided into "seen" categories (present in training) and "unseen" categories (held out), enabling evaluation of generalization. This dataset is used during LoRA GPT-2 fine-tuning for model selection and hyperparameter tuning on the RDF-to-text task.

Usage

Use this dataset as the validation split when fine-tuning GPT-2 with LoRA for RDF-to-text generation. The format_converting_webnlg.py script converts this JSON into line-delimited context/completion pairs, filtering to only "good" quality lexicalisations. Each converted example also includes a cate boolean indicating whether the category was seen during training, enabling separate evaluation on seen vs. unseen categories. Load during training to monitor model performance and select the best checkpoint.

Code Reference

Source Location

• Repository: Microsoft_LoRA
• File: examples/NLG/data/webnlg_challenge_2017/dev.json
• Converter: examples/NLG/src/format_converting_webnlg.py

Data Schema

{
  "entries": [
    {
      "1": {
        "category": "Airport",
        "dbpedialinks": [],
        "lexicalisations": [
          {
            "comment": "good",
            "lang": "",
            "lex": "The leader of Aarhus is Jacob Bundsgaard.",
            "xml_id": "Id1"
          }
        ],
        "links": [],
        "modifiedtripleset": [
          {
            "subject": "Aarhus",
            "property": "leaderName",
            "object": "Jacob_Bundsgaard"
          }
        ],
        "originaltriplesets": {
          "originaltripleset": [
            [
              {
                "subject": "Aarhus",
                "property": "leaderName",
                "object": "Jacob_Bundsgaard"
              }
            ]
          ]
        },
        "shape": "NA",
        "shape_type": "NA",
        "size": "1",
        "xml_id": "Id1"
      }
    }
  ]
}

Format Conversion

The format_converting_webnlg.py script processes entries by extracting modified triplesets, filtering for "good" lexicalisations, and tracking seen/unseen categories:

# Seen categories used during training
seen = ['Airport', 'Astronaut', 'Building', 'City', 'ComicsCharacter',
        'Food', 'Monument', 'SportsTeam', 'University', 'WrittenWork']

# Conversion produces lines like:
# {"context": "Aarhus : leaderName : Jacob_Bundsgaard",
#  "completion": "The leader of Aarhus is Jacob Bundsgaard.",
#  "cate": true}

Loading

import json

with open("examples/NLG/data/webnlg_challenge_2017/dev.json", "r") as f:
    data = json.load(f)

I/O Contract

Inputs

Name	Type	Required	Description
file_path	str	Yes	Path to the JSON file (e.g., examples/NLG/data/webnlg_challenge_2017/dev.json)

Outputs

Name	Type	Description
data	Dict	Top-level dictionary with an entries key
entries	List[Dict]	List of numbered entry dictionaries
category	str	DBpedia category (e.g., Airport, Astronaut, Building)
modifiedtripleset	List[Dict]	RDF triples with subject, property, object keys
lexicalisations	List[Dict]	Human-written texts with comment (quality), lex (text), lang, and xml_id
originaltriplesets	Dict	Original DBpedia triplesets preserving raw formatting

Usage Examples

Loading and Inspecting WebNLG Dev Data

import json

# Load the development set
with open("examples/NLG/data/webnlg_challenge_2017/dev.json", "r") as f:
    dev_data = json.load(f)

entries = dev_data['entries']
print(f"Total entries: {len(entries)}")

# Inspect first example
first_entry = entries[0]["1"]
print(f"Category: {first_entry['category']}")
print(f"Triples: {first_entry['modifiedtripleset']}")
print(f"Lexicalisations: {len(first_entry['lexicalisations'])}")
for lex in first_entry['lexicalisations']:
    print(f"  [{lex['comment']}] {lex['lex']}")

Analyzing Category Distribution

import json
from collections import Counter

with open("examples/NLG/data/webnlg_challenge_2017/dev.json", "r") as f:
    dev_data = json.load(f)

categories = Counter()
for i, entry in enumerate(dev_data['entries']):
    cat = entry[str(i + 1)]['category']
    categories[cat] += 1

for cat, count in categories.most_common():
    print(f"  {cat}: {count}")

Related Pages

• Environment:Microsoft_LoRA_PyTorch_CUDA_Environment