Workflow:Apache Shardingsphere Dynamic Rule Configuration Change

Property	Value
Workflow Name	Dynamic Rule Configuration Change
Project	Apache ShardingSphere
Domains	Distributed_Database, Configuration_Management, Event_Driven
Source Repository	https://github.com/apache/shardingsphere
Documentation	https://shardingsphere.apache.org/document/current/en/features/distsql/
Last Updated	2026-02-10 12:00 GMT

Overview

The Dynamic Rule Configuration Change workflow describes the end-to-end mechanism by which Apache ShardingSphere propagates configuration changes across all nodes in a cluster. When an operator issues a DistSQL statement or invokes the management API to alter a rule, the change must be durably persisted, version-controlled, broadcast to every cluster member, validated, and finally applied as a live in-memory rule rebuild -- all without downtime or inconsistency.

This workflow is central to ShardingSphere's Cluster mode, where a shared governance registry (typically ZooKeeper) acts as the single source of truth. The design follows an event-driven architecture: the originating node writes the new configuration to the registry with a new version number and atomically switches the active version pointer. Every other node in the cluster detects this write through a registry watcher, receives a DataChangedEvent, validates the version, dispatches the event to the correct handler, and rebuilds the affected rule objects in memory.

The flow applies uniformly to database-level rules (sharding, encryption, shadow, read-write splitting) and global rules (transaction, authority), with the handler layer abstracting over named rule items, unique rule items, and full rule-type changes through a hierarchy of specialized handlers and SPI-based processors.

Description

Architectural Context

ShardingSphere's configuration persistence follows a layered architecture:

Persistence Layer -- ClusterMetaDataManagerPersistService and DatabaseRulePersistService serialize rule configurations into YAML node tuples and write them to the governance registry via VersionPersistService.
Versioning Layer -- VersionPersistService assigns monotonically increasing version numbers. Each configuration item is stored at a versioned path (e.g., /metadata/{db}/rules/{ruleType}/{item}/versions/{n}) and the active version pointer (active_version) is switched atomically after the version content is written.
Event Layer -- The governance registry (ZooKeeper, etcd) fires watch notifications when nodes are created, updated, or deleted. These are delivered to the ShardingSphere cluster as DataChangedEvent objects carrying the changed key path, the new value, and the event type (ADDED, UPDATED, DELETED, or IGNORED).
Listener Layer -- DatabaseMetaDataChangedListener and GlobalMetaDataChangedListener receive raw events and route them through a chain of handlers. The listener first extracts the database name from the event key, checks whether the event's version matches the active version via ActiveVersionChecker, and then dispatches to the first matching handler.
Handler Layer -- Three concrete handler subtypes cover all rule item topologies: NamedRuleItemConfigurationChangedHandler (items identified by name, such as sharding table rules or shadow algorithms), UniqueRuleItemConfigurationChangedHandler (singleton items per rule type, such as a default strategy), and RuleTypeConfigurationChangedHandler (entire rule type deletions). All three extend RuleItemConfigurationChangedHandler, which uses RuleItemChangedNodePathBuilder to parse the event key into a DatabaseRuleNodePath and then delegates to DatabaseRuleItemManager.
Processor Layer -- DatabaseRuleItemManager locates the appropriate RuleItemConfigurationChangedProcessor via the SPI mechanism, keyed by a RuleChangedItemType composed of the rule type and item type (e.g., "shadow" + "shadow_algorithms"). The processor deserializes the YAML content, mutates the current in-memory RuleConfiguration, and hands control to DatabaseRuleConfigurationManager for a full rule rebuild.
Rebuild Layer -- DatabaseRuleConfigurationManager checks whether the affected rule supports partial updates (PartialRuleUpdateSupported). If so, only the delta is applied; otherwise, the entire rule is rebuilt from the updated configuration using DatabaseRulesBuilder, and the MetaDataContexts are atomically swapped. Old rule objects that implement AutoCloseable are closed.
EventBus Layer -- DeliverEventSubscriberRegistry manages cross-cutting subscribers registered on the Guava EventBusContext, enabling additional side effects (e.g., statistics refresh, cache invalidation) to occur after configuration changes.

Key Data Structures

Structure	Purpose
`DataChangedEvent`	Immutable event carrying `key` (registry path), `value` (content or version), and `type` (ADDED / UPDATED / DELETED / IGNORED).
`DatabaseRuleNodePath`	Structured representation of a registry path, decomposed into database name, rule type, and `DatabaseRuleItem` (item type plus optional item name).
`VersionNodePath`	Wraps a `DatabaseRuleNodePath` to derive the `active_version` path and the `versions/{n}` content path.
`MetaDataVersion`	Pairs a `DatabaseRuleNodePath` with a version number, returned from persistence operations to track what was written.
`RuleChangedItemType`	Composite SPI key combining rule type string and item type string, used to locate the correct `RuleItemConfigurationChangedProcessor`.
`RuleNodeTuple`	Pairs a `DatabaseRuleNodePath` with its YAML content string, used during serialization and deserialization.

Concurrency and Consistency

DatabaseRuleItemManager.alter() and DatabaseRuleItemManager.drop() are synchronized on the manager instance, preventing concurrent modifications to the same rule configuration from interleaving.
ActiveVersionChecker.checkSame() compares the event's version value against the registry's current active_version. If they do not match (indicating a stale or superseded event), the event is silently discarded and a warning is logged. This prevents out-of-order event application.
VersionPersistService.switchActiveVersion() writes the active version pointer and then deletes all prior version nodes, ensuring the registry does not accumulate unbounded version history.
ClusterMetaDataManagerPersistService.getReloadedMetaDataContexts() uses a retry loop (up to 30 seconds with 1-second intervals after an initial 3-second wait) to confirm that the local MetaDataContexts reference has been swapped, guarding against event propagation delays.

Usage

This workflow is triggered automatically by ShardingSphere's Cluster mode whenever rule configuration changes occur. Users and operators interact with this workflow indirectly through:

DistSQL statements -- e.g., CREATE SHARDING TABLE RULE, ALTER SHADOW RULE, DROP ENCRYPT RULE.
Management API calls -- Programmatic invocations of MetaDataManagerPersistService.alterRuleConfiguration() or MetaDataManagerPersistService.alterGlobalRuleConfiguration().
Configuration file changes -- Modifications to YAML configuration that are loaded at startup or through configuration refresh operations.

Prerequisites:

ShardingSphere must be running in Cluster mode with a governance registry (ZooKeeper, etcd) configured.
The ClusterPersistRepository must be connected and healthy.
Database listeners must be registered for the target database via ClusterDatabaseListenerPersistCoordinator.

Applicability:

The workflow handles all rule types that implement the RuleItemConfigurationChangedProcessor SPI, including but not limited to: sharding (table rules, key generators, algorithms), read-write splitting (data sources, load balancers), encryption (encryptors, tables), shadow (algorithms, data sources, tables), and global rules (authority, transaction).

Execution Steps

Step 1: Configuration Change Initiated

Attribute	Detail
Trigger	DistSQL statement execution or Management API call
Entry Point	`ClusterMetaDataManagerPersistService.alterRuleConfiguration()` or `ClusterMetaDataManagerPersistService.alterGlobalRuleConfiguration()`
Source File	`mode/type/cluster/core/src/main/java/org/apache/shardingsphere/mode/manager/cluster/persist/service/ClusterMetaDataManagerPersistService.java`

The workflow begins when an operator executes a DistSQL statement (e.g., ALTER SHADOW RULE) or the management API is called programmatically. The ClusterMetaDataManagerPersistService captures a snapshot of the current MetaDataContexts before any mutation occurs:

MetaDataContexts originalMetaDataContexts = new MetaDataContexts(
    metaDataContextManager.getMetaDataContexts().getMetaData(),
    metaDataContextManager.getMetaDataContexts().getStatistics());

This snapshot is used later to detect when the event-driven reload has completed. For database-level rules, the service delegates to metaDataPersistFacade.getDatabaseRuleService().persist(). For global rules, it calls metaDataPersistFacade.getGlobalRuleService().persist(). Null configurations are guarded with an early return.

Step 2: Change Persisted to Repository with New Version

Attribute	Detail
Action	YAML serialization, version increment, content storage
Key Classes	`DatabaseRulePersistService`, `VersionPersistService`
Source Files	`mode/core/src/main/java/org/apache/shardingsphere/mode/metadata/persist/config/database/DatabaseRulePersistService.java`, `mode/core/src/main/java/org/apache/shardingsphere/mode/metadata/persist/version/VersionPersistService.java`

DatabaseRulePersistService.persist() converts each RuleConfiguration to its YAML representation via YamlRuleConfigurationSwapperEngine, then decomposes it into individual RuleNodeTuple objects (one per rule item). For each tuple:

VersionPersistService.getNextVersion() reads existing version children under the item's versions/ path and computes max(existing) + 1, or 0 (the INIT_VERSION) if none exist.
The content is written to /metadata/{db}/rules/{ruleType}/{itemType}/{itemName}/versions/{nextVersion}.
The active version is switched (see Step 3).
A MetaDataVersion is returned recording the node path and the prior version number.

For delete operations, DatabaseRulePersistService.delete() reverses the tuple order (to respect dependency ordering) and removes each item's path from the registry.

Step 3: Active Version Switched Atomically

Attribute	Detail
Action	Active version pointer update, old version cleanup
Key Class	`VersionPersistService`
Source File	`mode/core/src/main/java/org/apache/shardingsphere/mode/metadata/persist/version/VersionPersistService.java`

Immediately after writing the new version content, VersionPersistService.switchActiveVersion() performs two operations:

Pointer update: Writes the new version number to the active_version node at the item's path. This single write is the atomic commit point -- it is the operation that the registry watcher on other nodes detects.
Old version cleanup: Iterates over all version numbers lower than the current one and deletes their content nodes. This prevents unbounded growth of the version history in the registry.

private void switchActiveVersion(final VersionNodePath versionNodePath, final int currentVersion) {
    repository.persist(versionNodePath.getActiveVersionPath(), String.valueOf(currentVersion));
    if (MetaDataVersion.INIT_VERSION != currentVersion) {
        getVersions(versionNodePath.getVersionsPath()).stream()
            .filter(version -> version < currentVersion)
            .forEach(version -> repository.delete(versionNodePath.getVersionPath(version)));
    }
}

If more than two versions are found during version enumeration, a warning is logged indicating a potential configuration anomaly, and the versions are sorted in reverse order to ensure the latest is correctly identified.

Step 4: Data Change Event Emitted by Repository Watcher

Attribute	Detail
Action	Registry watcher detects node change, constructs event
Key Class	`DataChangedEvent`
Source File	`mode/core/src/main/java/org/apache/shardingsphere/mode/event/DataChangedEvent.java`

The governance registry (ZooKeeper, etcd) detects the write to the active_version node and fires a watch notification. The ClusterPersistRepository implementation translates this notification into a DataChangedEvent containing:

key -- The full registry path of the changed node (e.g., /metadata/my_db/rules/shadow/shadow_algorithms/my_algo/active_version).
value -- The new content of the node (for active version changes, this is the version number string such as "1").
type -- One of ADDED, UPDATED, DELETED, or IGNORED.

This event is delivered to all registered DataChangedEventListener instances on the receiving node.

Step 5: Event Listener Receives and Filters the Event

Attribute	Detail
Action	Database/global routing, version validation
Key Classes	`DatabaseMetaDataChangedListener`, `GlobalMetaDataChangedListener`, `ActiveVersionChecker`
Source Files	`mode/type/cluster/core/src/main/java/org/apache/shardingsphere/mode/manager/cluster/dispatch/listener/type/DatabaseMetaDataChangedListener.java`, `mode/type/cluster/core/src/main/java/org/apache/shardingsphere/mode/manager/cluster/dispatch/listener/type/GlobalMetaDataChangedListener.java`, `mode/core/src/main/java/org/apache/shardingsphere/mode/metadata/manager/ActiveVersionChecker.java`

For database-level rule changes, DatabaseMetaDataChangedListener.onChange() executes the following sequence:

Database extraction: Uses NodePathSearcher.find() to extract the database name from the event key. If the key does not match a database metadata path, the event is discarded.
Cache invalidation: Clears the OrderedServicesCache to ensure SPI service resolution reflects any new configuration.
Handler matching: Iterates through the ordered handler chain (SchemaChangedHandler, TableChangedHandler, ViewChangedHandler, StorageUnitChangedHandler, StorageNodeChangedHandler, NamedRuleItemConfigurationChangedHandler, UniqueRuleItemConfigurationChangedHandler, RuleTypeConfigurationChangedHandler). For each handler, the listener checks whether the event key matches the handler's subscribed node path pattern.
Version validation: For ADDED or UPDATED events, ActiveVersionChecker.checkSame() queries the registry for the current active version and compares it to the event's value. If they differ (stale event), the entire event is discarded with a warning log.
Dispatch: The first matching handler's handle() method is invoked and the listener returns immediately (single-dispatch semantics).

For global rule changes, GlobalMetaDataChangedListener.onChange() verifies the event type is in the handler's subscribed types, runs the active version check for GlobalConfigurationChangedHandler instances, clears the ordered services cache, and delegates to the handler.

Step 6: Change Handler Dispatches to Appropriate Processor

Attribute	Detail
Action	Path parsing, SPI processor lookup, configuration mutation
Key Classes	`RuleItemConfigurationChangedHandler`, `RuleItemChangedNodePathBuilder`, `DatabaseRuleItemManager`, `RuleItemConfigurationChangedProcessor` (SPI)
Source Files	`mode/type/cluster/core/src/main/java/org/apache/shardingsphere/mode/manager/cluster/dispatch/handler/database/rule/RuleItemConfigurationChangedHandler.java`, `mode/core/src/main/java/org/apache/shardingsphere/mode/metadata/changed/RuleItemChangedNodePathBuilder.java`, `mode/core/src/main/java/org/apache/shardingsphere/mode/metadata/manager/rule/DatabaseRuleItemManager.java`

RuleItemConfigurationChangedHandler.handle() performs two sub-steps:

6a. Path parsing: RuleItemChangedNodePathBuilder.build() parses the event key to construct a DatabaseRuleNodePath. It first extracts the rule type, then iterates over the rule's named items and unique items (obtained from DatabaseRuleNodeGenerator) to match the specific item. For DELETED events, it checks the raw path structure; for ADDED/UPDATED events, it verifies the path is an active_version path via VersionNodePath.isActiveVersionPath().

6b. SPI dispatch: Based on the event type:

ADDED or UPDATED: Calls DatabaseRuleItemManager.alter(), which:
- Looks up the RuleItemConfigurationChangedProcessor via TypedSPILoader.getService() keyed by RuleChangedItemType(ruleType, itemType).
- Loads the YAML content from the versioned path via metaDataPersistFacade.getVersionService().loadContent().
- Calls processor.swapRuleItemConfiguration() to deserialize the YAML into a typed configuration object.
- Calls processor.changeRuleItemConfiguration() to mutate the current in-memory RuleConfiguration.
- Delegates to DatabaseRuleConfigurationManager.refresh() for rule rebuild.

DELETED: Calls DatabaseRuleItemManager.drop(), which follows a similar SPI lookup and calls processor.dropRuleItemConfiguration() to remove the item from the current configuration. If the resulting configuration is empty (checked via DatabaseRuleConfigurationEmptyChecker), the rule object is removed rather than rebuilt.

Example SPI processors (Shadow rule):

Processor	RuleChangedItemType	Action
`ShadowAlgorithmChangedProcessor`	`("shadow", "shadow_algorithms")`	Adds or replaces an `AlgorithmConfiguration` in `ShadowRuleConfiguration.getShadowAlgorithms()`.
`ShadowDataSourceChangedProcessor`	`("shadow", "data_sources")`	Adds or replaces a `ShadowDataSourceConfiguration` in the data sources collection.
`ShadowTableChangedProcessor`	`("shadow", "tables")`	Puts or removes a `ShadowTableConfiguration` in the tables map.

Step 7: In-Memory Rule Configuration Rebuilt

Attribute	Detail
Action	Rule rebuild, MetaDataContexts swap, old rule closure
Key Classes	`DatabaseRuleConfigurationManager`, `GlobalConfigurationManager`, `DeliverEventSubscriberRegistry`
Source Files	`mode/core/src/main/java/org/apache/shardingsphere/mode/metadata/manager/rule/DatabaseRuleConfigurationManager.java`, `mode/core/src/main/java/org/apache/shardingsphere/mode/metadata/manager/rule/GlobalConfigurationManager.java`, `mode/core/src/main/java/org/apache/shardingsphere/mode/deliver/DeliverEventSubscriberRegistry.java`

For database-level rules, DatabaseRuleConfigurationManager.refresh() (synchronized) executes:

Partial update check: If the existing rule implements PartialRuleUpdateSupported, partialUpdate() is called first to determine whether a full schema refresh is needed, and updateConfiguration() applies the incremental change.
Full rebuild (if needed): Removes the old rule from the rules collection, builds a new rule via DatabaseRulesBuilder.build() using the updated configuration, and adds it back.
MetaDataContexts swap: Creates a new MetaDataContexts via MetaDataContextsFactory.createByAlterRule() and atomically swaps the reference via metaDataContexts.update().
Resource cleanup: Closes old rule objects that implement AutoCloseable.

For global rules, GlobalConfigurationManager.alterGlobalRuleConfiguration() (synchronized) removes the matching rule, rebuilds it via GlobalRulesBuilder.buildSingleRules(), and swaps the ShardingSphereMetaData reference.

Event bus notification: DeliverEventSubscriberRegistry manages subscribers registered on the EventBusContext. After MetaDataContexts are updated, any subscribers (for statistics collection, cache refresh, etc.) are automatically notified through the Guava EventBus mechanism.

Execution Diagram

   Operator / DistSQL                  Originating Node                    Governance Registry             Other Cluster Nodes
         |                                    |                                    |                              |
         |  1. ALTER SHADOW RULE ...          |                                    |                              |
         |----------------------------------->|                                    |                              |
         |                                    |                                    |                              |
         |           2. ClusterMetaDataManagerPersistService                       |                              |
         |              .alterRuleConfiguration()                                  |                              |
         |                                    |                                    |                              |
         |              Snapshot current MetaDataContexts                          |                              |
         |                                    |                                    |                              |
         |           3. DatabaseRulePersistService.persist()                       |                              |
         |              -> YAML serialize to RuleNodeTuples                        |                              |
         |                                    |                                    |                              |
         |           4. VersionPersistService.persist()                            |                              |
         |              -> getNextVersion()   |                                    |                              |
         |              -> write versions/{n} |--------- persist content --------->|                              |
         |              -> switchActiveVersion|--------- persist active_version -->|                              |
         |              -> delete old versions|--------- delete versions/{n-1} -->|                              |
         |                                    |                                    |                              |
         |                                    |        [Watch fires]               |                              |
         |                                    |                                    |--- DataChangedEvent -------->|
         |                                    |                                    |    key: .../active_version   |
         |                                    |                                    |    value: "{n}"              |
         |                                    |                                    |    type: UPDATED             |
         |                                    |                                    |                              |
         |                                    |                                    |     5. DatabaseMetaDataChangedListener
         |                                    |                                    |        .onChange()            |
         |                                    |                                    |        - extract db name     |
         |                                    |                                    |        - match handler       |
         |                                    |                                    |        - ActiveVersionChecker|
         |                                    |                                    |          .checkSame()        |
         |                                    |                                    |<--- query active_version ----|
         |                                    |                                    |---- return "{n}" ----------->|
         |                                    |                                    |        [version matches]     |
         |                                    |                                    |                              |
         |                                    |                                    |     6. RuleItemConfiguration  |
         |                                    |                                    |        ChangedHandler.handle()|
         |                                    |                                    |        -> build NodePath      |
         |                                    |                                    |        -> DatabaseRuleItem    |
         |                                    |                                    |           Manager.alter()    |
         |                                    |                                    |        -> SPI: lookup         |
         |                                    |                                    |           processor by type   |
         |                                    |                                    |<--- load versioned content --|
         |                                    |                                    |---- return YAML ------------>|
         |                                    |                                    |        -> swap + mutate      |
         |                                    |                                    |           RuleConfiguration  |
         |                                    |                                    |                              |
         |                                    |                                    |     7. DatabaseRuleConfig     |
         |                                    |                                    |        Manager.refresh()     |
         |                                    |                                    |        -> rebuild rule       |
         |                                    |                                    |        -> swap MetaDataCtxs  |
         |                                    |                                    |        -> close old rules    |
         |                                    |                                    |        -> EventBus notify    |
         |                                    |                                    |                              |
         |                  [Originating node also receives event and rebuilds]    |                              |
         |                                    |                                    |                              |
         |  Response: Rule altered            |                                    |                              |
         |<-----------------------------------|                                    |                              |

Source File Reference

File	Module Path	Lines	Role
`ClusterMetaDataManagerPersistService.java`	`mode/type/cluster/core`	246	Entry point for cluster-mode configuration persistence; snapshots MetaDataContexts, delegates to rule persist services, and waits for event-driven reload.
`DatabaseRulePersistService.java`	`mode/core`	155	Serializes rule configurations to YAML tuples and persists each item with version tracking.
`VersionPersistService.java`	`mode/core`	91	Manages version numbering, content storage at versioned paths, active version switching, and old version cleanup.
`DataChangedEvent.java`	`mode/core`	43	Immutable event carrying the changed key, value, and type (ADDED, UPDATED, DELETED, IGNORED).
`DatabaseMetaDataChangedListener.java`	`mode/type/cluster/core`	102	Listens for per-database metadata changes; routes events through handler chain with version validation.
`GlobalMetaDataChangedListener.java`	`mode/type/cluster/core`	49	Listens for global configuration changes; validates version for config handlers before dispatch.
`RuleItemConfigurationChangedHandler.java`	`mode/type/cluster/core`	57	Abstract handler that parses event keys via RuleItemChangedNodePathBuilder and dispatches alter/drop to DatabaseRuleItemManager.
`NamedRuleItemConfigurationChangedHandler.java`	`mode/type/cluster/core`	40	Handles named rule items (e.g., specific sharding table rules, shadow algorithms) by subscribing to named-item path patterns.
`UniqueRuleItemConfigurationChangedHandler.java`	`mode/type/cluster/core`	40	Handles unique (singleton) rule items by subscribing to unique-item path patterns.
`RuleTypeConfigurationChangedHandler.java`	`mode/type/cluster/core`	39	Handles entire rule type changes (full rule deletion) by subscribing to rule-type-level path patterns.
`RuleItemChangedNodePathBuilder.java`	`mode/core`	76	Parses a registry path and event type into a structured DatabaseRuleNodePath by matching against named and unique item patterns.
`ActiveVersionChecker.java`	`mode/core`	63	Validates that an event's version matches the current active version in the registry; discards stale events.
`DatabaseRuleConfigurationManager.java`	`mode/core`	99	Rebuilds database-level rules from updated configuration; supports partial updates; swaps MetaDataContexts atomically.
`DatabaseRuleItemManager.java`	`mode/core`	96	Manages individual rule item alter/drop by locating the SPI processor, loading versioned content, and coordinating configuration mutation.
`GlobalConfigurationManager.java`	`mode/core`	101	Manages global rule and properties changes; rebuilds global rules and swaps ShardingSphereMetaData.
`ShadowAlgorithmChangedProcessor.java`	`features/shadow/core`	51	SPI processor for shadow algorithm changes; extends AlgorithmChangedProcessor.
`ShadowDataSourceChangedProcessor.java`	`features/shadow/core`	61	SPI processor for shadow data source changes; deserializes YamlShadowDataSourceConfiguration.
`ShadowTableChangedProcessor.java`	`features/shadow/core`	59	SPI processor for shadow table changes; deserializes YamlShadowTableConfiguration.
`DeliverEventSubscriberRegistry.java`	`mode/core`	41	Registers DeliverEventSubscriber instances on the EventBusContext for cross-cutting event handling.

GitHub URL

Workflow Repository