Principle:Explodinggradients Ragas Knowledge Graph Enrichment
| Knowledge Sources | Domains | Last Updated |
|---|---|---|
| explodinggradients/ragas | LLM Evaluation, Test Data Generation, Knowledge Graphs, Community Detection | 2026-02-10 |
Overview
Description
Knowledge Graph Enrichment is the principle of augmenting a raw knowledge graph with semantic layers -- summaries, embeddings, and cross-node relationships -- to enable diverse and meaningful test data generation. While knowledge graph construction creates the structural skeleton (nodes and parent-child edges), enrichment adds the semantic flesh that makes multi-hop reasoning, thematic clustering, and persona-driven query synthesis possible.
Usage
Knowledge graph enrichment occurs automatically as part of the Ragas test generation pipeline via transforms applied after initial graph construction. It can also be invoked explicitly for custom workflows. The enrichment process involves three main capabilities:
- Indirect Cluster Discovery: The
find_indirect_clusters()method uses the Leiden community detection algorithm to identify groups of nodes that share thematic connections through intermediate nodes, even when not directly connected. These clusters form the basis for multi-hop query scenarios. - Parent/Child Traversal: The
get_child_nodes()andget_parent_nodes()functions traverse the graph hierarchy to assemble multi-level context. Child traversal gathers finer-grained content; parent traversal gathers broader context. - Property Enrichment: Transforms add properties such as
summary(text summaries of node content),summary_embedding(vector embeddings of summaries), and semantic relationship edges to the graph.
Theoretical Basis
Community Detection via Leiden Algorithm: The find_indirect_clusters() method applies the Leiden algorithm (an improvement over Louvain) to identify communities of closely related nodes. The algorithm operates on the adjacency matrix of filtered relationships, grouping nodes into clusters based on connection density. Within each cluster, all simple paths up to a configurable depth limit are enumerated (for small clusters) or sampled via random walks (for large clusters). Each unique path becomes a candidate cluster of related nodes that can serve as the context for a multi-hop question.
Depth-Limited Path Exploration: The depth limit parameter controls the maximum number of edges in a path, which directly determines the complexity of multi-hop questions. A depth of 2 (the minimum) yields 3-node paths (A to B to C); a depth of 3 yields paths up to 4 nodes. The system intelligently switches between exhaustive path enumeration and random sampling based on the estimated number of paths, ensuring efficiency for both small and large graphs.
Hierarchical Traversal for Context Assembly: The get_child_nodes() and get_parent_nodes() functions implement depth-limited depth-first search (DFS) following "child"-type relationships. Child traversal descends from a node to gather its constituent chunks, enabling questions that require synthesizing information from multiple parts of a document. Parent traversal ascends to gather broader context, enabling questions that require understanding the document-level theme.
Embedding-Based Similarity: Node summary embeddings enable cosine similarity comparisons between nodes, which are used for grouping related content during persona generation and for discovering semantic (as opposed to structural) relationships between nodes.
Superset/Subset Deduplication: When collecting indirect clusters, the system removes subset clusters when a superset is found (e.g., if {A, B, C, D} exists, {A, B, C} is removed). This ensures that the most informative context groupings are preserved for downstream query synthesis.
Related Pages
- Implementation:Explodinggradients_Ragas_KnowledgeGraph_Enrichment_Methods
- Principle:Explodinggradients_Ragas_Knowledge_Graph_Construction -- the graph that enrichment operates on
- Principle:Explodinggradients_Ragas_Persona_Generation -- uses enriched embeddings and summaries for persona creation
- Principle:Explodinggradients_Ragas_Test_Query_Synthesis -- uses enriched clusters for multi-hop query generation