Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
10.6(EOL), 10.11, 11.4, 11.8, 12.3
-
None
-
Can result in data loss
-
Q4/2026 Replic. Development
Description
Rsync SST script syncs all filesystems unconditionally:
STATE=$(cat "$FLUSHED")
|
rm "$FLUSHED"
|
|
|
sync
|
|
|
wsrep_log_info "Tables flushed"
|
This can potentially hang the Donor node for a long time. This is most evident in buildbot, where MTR may fail to start the cluster completely because MTR server startup times out before sync completes. Excerpt from builder log:
worker[09] mysql-test-run: WARNING: Waited 120 seconds for /home/buildbot/aarch64-debian-12/build/mysql-test/var/9/run/mysqld.2.pid to be created, still waiting for 60 seconds...
|
worker[10] mysql-test-run: WARNING: Waited 180 seconds for /home/buildbot/aarch64-debian-12/build/mysql-test/var/10/run/mysqld.2.pid to be created, still waiting for 0 seconds...
|
worker[10] mysql-test-run: WARNING: Timeout after mysql-test-run waited 180 seconds for the process [mysqld.2 - pid: 230633, winpid: 230633] to create a pid file.
|
worker[01] mysql-test-run: WARNING: Waited 180 seconds for /home/buildbot/aarch64-debian-12/build/mysql-test/var/1/run/mysqld.2.pid to be created, still waiting for 0 seconds...
|
worker[01] mysql-test-run: WARNING: Timeout after mysql-test-run waited 180 seconds for the process [mysqld.2 - pid: 230647, winpid: 230647] to create a pid file.
|
worker[11] mysql-test-run: WARNING: Waited 180 seconds for /home/buildbot/aarch64-debian-12/build/mysql-test/var/11/run/mysqld.2.pid to be created, still waiting for 0 seconds...
|
Donor log shows that the SST process hangs for a long time:
2026-08-10 22:41:51 0 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'donor' --address '127.0.0.1:19217/rsync_sst' --local-port 19210 --socket '/home/buildbot/aarch64-debian-12/build/mysql-test/var/tmp/10/mysqld.1.sock' --progress 0 --datadir '/home/buildbot/aarch64-debian-12/build/mysql-test/var/10/mysqld.1/data/' --defaults-file '/dev/shm/var_auto_7qqo/10/my.cnf' --defaults-group-suffix '.1' --gtid 'ac3f1a86-950c-11f1-8120-f24accc888dd:2' --gtid-domain-id 0 --binlog 'mysqld-bin' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.1 --defaults-file=/home/buildbot/aarch64-debian-12/build/mysql-test/var/10/my.cnf --log-output=file --log-bin --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --innodb --innodb-cmpmem --innodb-cmp-per-index
|
2026-08-10 22:41:51 1 [Note] WSREP: sst_donor_thread signaled with 0
|
WSREP_SST: [INFO] rsync SST started on donor (20260810 22:41:51.100)
|
2026-08-10 22:41:51 0 [Note] WSREP: Flushing tables for SST...
|
2026-08-10 22:41:51 0 [Note] WSREP: pause
|
2026-08-10 22:41:51 0 [Note] WSREP: Provider paused at ac3f1a86-950c-11f1-8120-f24accc888dd:2 (5)
|
2026-08-10 22:41:51 0 [Note] WSREP: Server paused at: 2
|
2026-08-10 22:41:51 0 [Note] InnoDB: Stopping purge
|
2026-08-10 22:41:51 0 [Note] WSREP: Tables flushed.
|
2026-08-10 22:41:51 0 [Note] WSREP: Donor state reached
|
2026-08-10 22:41:52 0 [Note] WSREP: (ac3eef87-b503, 'tcp://0.0.0.0:19212') turning message relay requesting off
|
2026-08-10 22:44:53 0 [Note] /home/buildbot/aarch64-debian-12/build/sql/mariadbd (initiated by: unknown): Normal shutdown
|
Message
WSREP_SST: [INFO] Tables flushed
|
from SST script gets never logged before the MTR terminates the server process.
Possible fix: Sync only data directories which are transferred during SST, including data directory, binlog dirs etc. Take care of dealing with symlinks.
Attachments
Issue Links
- relates to
-
-
- Open
-
-
MDEV-41744 Stop creating .isl files for DATA DIRECTORY
-
- Open
-
