Principle:Apache Shardingsphere Versioned Change Persistence
| Knowledge Sources | |
|---|---|
| Domains | Configuration_Management, Distributed_Systems |
| Last Updated | 2026-02-10 00:00 GMT |
Overview
Versioned change persistence is the practice of storing each configuration modification with a monotonically increasing version number, enabling atomic updates and conflict detection across distributed nodes.
Description
When a distributed system must coordinate configuration changes across multiple nodes, simply overwriting the current configuration creates race conditions and makes it impossible to detect whether a node has applied the latest change. Versioned change persistence solves this by decomposing each rule configuration into individual rule node tuples (key-value pairs mapped to repository paths) and persisting each tuple with an explicit version number.
The persistence process works as follows:
- YAML Conversion: Rule configurations are first converted to their YAML representations using a swapper engine, producing YamlRuleConfiguration objects.
- Tuple Decomposition: Each YAML rule configuration is decomposed into granular RuleNodeTuple objects, where each tuple represents a single rule item (e.g., a specific sharding algorithm, a table rule, or a key generator configuration).
- Versioned Write: Each tuple is persisted through a VersionPersistService that assigns the next available version number, writes the content to a versioned path, and atomically switches the active version pointer.
- MetaDataVersion Tracking: The method returns a collection of MetaDataVersion objects that record the node path and the previous active version for each changed tuple, enabling downstream consumers to determine what changed.
This approach provides several guarantees:
- Atomicity at the tuple level: each individual rule item is updated atomically via the version pointer switch.
- Conflict detection: Nodes can compare their local active version against the repository's active version to detect stale reads.
- Auditability: The version history provides a linear record of configuration changes.
- Garbage collection: Old versions are automatically cleaned up after a successful version switch, preventing unbounded storage growth.
Usage
Use versioned change persistence whenever rule configurations need to be written to the shared distributed repository. This applies to:
- Persisting new or altered database-scoped rules (sharding, encryption, read-write splitting).
- Persisting global rule configurations (authority, transaction).
- Deleting rule configurations, which also produces MetaDataVersion records for downstream event processing.
Theoretical Basis
The versioned persistence model follows a version-append-and-switch pattern:
PROCEDURE persist_versioned(database_name, rule_configurations):
result = []
FOR EACH yaml_config IN swap_to_yaml(rule_configurations):
tuples = decompose_to_tuples(database_name, yaml_config)
FOR EACH tuple IN tuples:
version_path = build_version_node_path(tuple.node_path)
next_version = get_next_version(version_path)
// Write content at versioned path
repository.persist(version_path / next_version, tuple.content)
// Atomically switch active version pointer
repository.persist(version_path / "active_version", next_version)
// Clean up old versions
delete_versions_older_than(version_path, next_version)
// Track the change for downstream consumers
previous_version = max(INIT_VERSION, next_version - 1)
result.add(MetaDataVersion(tuple.node_path, previous_version))
RETURN result
END PROCEDURE
The key property of this model is that each tuple persistence is self-contained: it determines its own next version, writes the content, switches the pointer, and cleans up old versions. This means that even if the process fails partway through persisting a multi-tuple rule configuration, the tuples that were successfully persisted are individually consistent. The system relies on eventual consistency and the active version checking mechanism in the listener layer to handle partial writes gracefully.