GBase 8a 9.5.3 New Feature: Event‑Based Data Node Replacement (eventMode)

Starting from certain builds of GBase 8a 9.5.3, a new event‑driven method for replacing failed data nodes is available. Unlike the traditional redistribution‑based replacement, eventMode synchronises change events between nodes to restore data, eliminating the need for a full‑scale rebalancing operation.

Supported Versions

This feature was implemented in 9.5.3.28.18R1_Patch.8 and may be merged into later versions.

Test Cluster Setup

A 3‑node cluster with one management node, one coordinator, and three data nodes. node1 (10.0.2.152) is a pure data node that has been marked UNAVAILABLE due to a failure.

[gbase@rh151 gcinstall]$ gcadmin
...
| node1 | 10.0.2.152 | 1,2 | UNAVAILABLE |
| node2 | 10.0.2.151 | 1,2 | OPEN | OPEN | 0 |
| node3 | 10.0.2.153 | 1,2 | OPEN | OPEN | 0 |

The eventMode Replacement Command

Pass the --feventMode flag to the replace.py script to trigger event‑based replacement.

./replace.py --feventMode \
  --host=10.0.2.152 \
  --dbaUser=gbase \
  --dbaUserPwd=XXXX \
  --generalDBUse=gbase \
  --generalDBPwd=XXXXXX \
  --overwrite \
  --type=data

Prerequisites for eventMode

Before running the command, any ongoing data redistribution must be paused:

• Set gcluster_rebalancing_concurrent_count to 0.
• The gclusterdb.rebalancing_status table must be empty (truncated).

The script checks these conditions automatically and exits if they are not met. After the replacement finishes, a warning is printed reminding you not to restart the rebalance until all dmlstorageevent recoveries are complete.

Execution Walkthrough

The script runs through verification, data packet synchronisation, and metadata recovery. Key log excerpts:

gcluster_rebalancing_concurrent_count value was 0 at coordinator 10.0.2.151
table gclusterdb.rebalancing_status was empty
...
sync dataserver metedata end, spend time 46898 ms
...
restore node state start
...
restore node state end
...
replace nodes spend time: 101831 ms
set dmlstorageevent success
please wait all dmlstorageevent recover success, do not start rebalance before it
Replace gcluster nodes successfully.

Once completed, the gnode status of the replaced node returns to OPEN, and the cluster resumes normal operation.

Pros and Cons

• Advantage: You can perform a node replacement in the middle of a cluster expansion or redistribution, without waiting for the rebalance to finish, thus reducing recovery time.
• Drawback: The event‑based method generates a large number of internal events, making it harder to control the resource impact through parallelism compared to the redistribution method. Brief performance fluctuations may occur during the process.

Summary

eventMode provides a complementary approach to data node replacement in a gbase database cluster. It gives DBAs greater flexibility, especially in large clusters, by decoupling node recovery from long‑running redistribution tasks. Use it when speed is critical and temporary resource spikes are acceptable. GBASE continues to enhance its MPP platform with features that streamline day‑to‑day operations for China‑domestically developed databases.