One question that surfaces regularly in the Percona forums: Does every node in a Percona XtraDB Cluster (PXC) need to have XtraBackup installed? It's a fair question, especially when managing a mixed environment or trying to minimize the software footprint on certain nodes. Here is what the actual mechanics and testing confirm.
The Short Answer (But Read On)
It depends on what you want that node to do. The nuance matters quite a bit here, so it is worth walking through how State Snapshot Transfer (SST) works in PXC and why XtraBackup's presence — or absence — on a given node is significant.
A Quick Refresher on SST in PXC
When a new node joins a Percona XtraDB Cluster, or when an existing node has been down long enough that Incremental State Transfer (IST) is no longer possible, the cluster performs a State Snapshot Transfer (SST). This is essentially a full data copy from a donor node to the joiner node.
PXC supports multiple SST methods, configured in my.cnf:
[mysqld]
wsrep_sst_method = xtrabackup-v2
The available SST methods include:
- xtrabackup-v2 — The recommended method for PXC, using Percona XtraBackup; performs SST without locking the donor for extended periods
- clone — Available in PXC 8.0.22+ using MySQL's built-in Clone Plugin; removes the XtraBackup dependency for SST
- mysqldump — Slower and locks the donor during transfer; not recommended for production
- rsync — Requires the donor to be read-only during transfer, blocking writes; also not recommended for live clusters
The xtrabackup-v2 method has historically been the default approach and remains widely used in existing deployments precisely because it keeps the donor node available for writes during the transfer. The other legacy methods can block writes on the donor, which is generally unacceptable in a production cluster. Note that Percona has been increasingly recommending the clone method for new installations on PXC 8.0.22 and later, as it removes the external tool dependency at the SST layer.
Where Does XtraBackup Need to Be Installed?
When an SST using xtrabackup-v2 is triggered, both the donor and the joiner need XtraBackup installed and accessible. Here is why both sides are involved:
- The donor runs XtraBackup to stream the snapshot data outbound
- The joiner runs XtraBackup — specifically the
xbstreamandxbcryptutilities — to receive and apply that streamed data
If the joiner node does not have XtraBackup installed and you attempt to bring it into the cluster using xtrabackup-v2, the SST will fail. The error log on the joiner will typically show something like this:
[ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_xtrabackup-v2
...
wsrep_sst_xtrabackup-v2: line 522: xbstream: command not found
[ERROR] WSREP: SST failed: 2 (No such file or directory)
That is a clear and unambiguous failure mode. If xbstream is not present on the joiner, the SST will not complete.
What About Nodes That Will Never Be a Joiner?
Technically, if a node will always act as a donor and never needs to rejoin the cluster from scratch, you could argue it only needs XtraBackup in its donor capacity. In practice, however, any node can become a joiner — after a crash, after planned maintenance, or after recovering from a network partition. There is no reliable way to guarantee a node will never need to receive an SST.
The practical guidance here is straightforward: install XtraBackup on every PXC node, without exception. The overhead of having it installed is negligible. The cost of a failed SST during an unplanned outage is not.
The Clone Plugin Alternative (PXC 8.0.22+)
Starting with PXC 8.0.22, Percona added support for the MySQL Clone Plugin as an SST method. This is worth knowing about because it removes the XtraBackup dependency for SST purposes entirely:
[mysqld]
wsrep_sst_method = clone
With the clone method, the Clone Plugin must be loaded on all nodes:
INSTALL PLUGIN clone SONAME 'mysql_clone.so';
SHOW PLUGINS WHERE Name = 'clone';
+-------+--------+-------+----------------+---------+
| Name | Status | Type | Library | License |
+-------+--------+-------+----------------+---------+
| clone | ACTIVE | CLONE | mysql_clone.so | GPL |
+-------+--------+-------+----------------+---------+
The clone method is a solid option for standardizing without XtraBackup as an SST dependency. That said, XtraBackup still has real value for your external backup strategy regardless of which SST method you choose. SST is a cluster synchronization mechanism — it is not a backup, and it should never be treated as one.
Checking Your Current SST Configuration
You can verify your current SST method and Galera-related settings with:
SHOW VARIABLES LIKE 'wsrep_sst_method';
+------------------+---------------+
| Variable_name | Value |
+------------------+---------------+
| wsrep_sst_method | xtrabackup-v2 |
+------------------+---------------+
To check cluster state and confirm which node may be acting as donor:
SHOW STATUS LIKE 'wsrep_local_state_comment';
+---------------------------+--------+
| Variable_name | Value |
+---------------------------+--------+
| wsrep_local_state_comment | Synced |
+---------------------------+--------+
SHOW STATUS LIKE 'wsrep_connected';
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| wsrep_connected | ON |
+-----------------+-------+
SHOW STATUS LIKE 'wsrep_cluster_size';
+--------------------+-------+
| Variable_name | Value |
+--------------------+-------+
| wsrep_cluster_size | 3 |
+--------------------+-------+
Practical Observations
A few things worth noting from working with PXC environments directly:
- The version of XtraBackup must match your PXC version. Using XtraBackup 2.x with PXC 8.0 will cause SST failures. Use Percona XtraBackup 8.0 with PXC 8.0, and confirm version alignment after any upgrade.
- Even if you switch to the
cloneSST method, keep XtraBackup installed for scheduled backups. Your backup strategy and your SST method are separate concerns and should be treated as such. - The
wsrep_sst_donorvariable lets you specify a preferred donor node, which is useful for directing SST away from your busiest or most latency-sensitive member. - If you are running Percona Toolkit alongside PXC, be aware of how DDL replication works in your specific PXC version — Total Order Isolation (TOI) versus Rolling Schema Upgrade (RSU) behavior differs and is worth a dedicated look before running schema changes in production.
Summary
To answer the question directly: if you are using xtrabackup-v2 as your SST method — which remains the default in many existing PXC deployments — then yes, XtraBackup needs to be installed on every cluster member. Any node can be either a donor or a joiner depending on circumstances, and both roles require XtraBackup to be present when using this method.
If you are on PXC 8.0.22 or later and want to eliminate that dependency at the SST layer, the Clone Plugin method is a viable alternative and is increasingly Percona's recommended choice for new deployments. If you are starting fresh, PXC 8.4 LTS is the current long-term support release and the recommended target for new installations. Even when using clone for SST, XtraBackup remains the right tool for your actual backup jobs.
Do not try to save a few megabytes of disk space by skipping XtraBackup on select nodes. The SST failure that eventually results from that decision is not a trade-off worth making.
No comments:
Post a Comment
@AnotherMySQLDBA