Details
-
Bug
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
10.11.10
-
None
-
Gentoo linux
mariadb-10.11.10
galera-26.4.21
Description
Hello,
we've encountered a possible bug in mariadb-backup utility, while attempting SST of one of our Galera Cluster nodes.
SST is able to transfer the data to the new node. However when mariadb-backup --prepare is called. It gets stuck and (seemingly) never finishes. During our testing, this state persisted for hours. Only being interrupted by innodb log file being filled causing the SST and Mariadb to shutdown.
However, when we run the same command on that same data again manually it finishes without a problem. We are then able to manually start the node and get it to connect to the cluster.
Following is the precise mariadb-backup command issued:
WSREP_SST: [INFO] Evaluating /usr//bin/mariadb-backup --prepare --no-version-check --use-memory=128G --target-dir='/data/mysql/.sst' --datadir='/data/mysql/.sst' > '/data/mysql/mariabackup.prepare.log' 2>&1 (20250102 04:54:11.307) |
During this stuck phase we can see a lot of errors like this in the mariabackup.prepare.log:
2025-01-14 8:11:37 0 [Note] InnoDB: IO Error: 22 during write of 1536 bytes, for file app_user_664783/generic_collection_item_v001#P#5652409f5ee81cfc8f270185cabaab1b.ibd(644), returned 0 |
2025-01-14 8:11:37 0 [Note] InnoDB: IO Error: 22 during write of 2560 bytes, for file app_user_664783/generic_collection_item_v001#P#5652409f5ee81cfc8f270185cabaab1b.ibd(644), returned 0 |
2025-01-14 8:11:37 0 [Note] InnoDB: IO Error: 22 during write of 2560 bytes, for file app_user_664783/generic_collection_item_v001#P#5652409f5ee81cfc8f270185cabaab1b.ibd(644), returned 0 |
2025-01-14 8:11:37 0 [Note] InnoDB: IO Error: 22 during write of 3072 bytes, for file app_user_664783/generic_collection_item_v001#P#5652409f5ee81cfc8f270185cabaab1b.ibd(644), returned 0 |
2025-01-14 8:11:37 0 [Note] InnoDB: IO Error: 22 during write of 1536 bytes, for file app_user_664783/generic_collection_item_v001#P#5652409f5ee81cfc8f270185cabaab1b.ibd(644), returned 0 |
2025-01-14 8:11:37 0 [Note] InnoDB: IO Error: 22 during write of 1024 bytes, for file app_user_664783/generic_collection_item_v001#P#5652409f5ee81cfc8f270185cabaab1b.ibd(644), returned 0 |
2025-01-14 8:11:37 0 [Note] InnoDB: IO Error: 22 during write of 1536 bytes, for file app_user_499733/generic_collection_item_v001#P#d60680e60bd5c8c08f7c7e5f6a5e3fb0.ibd(643), returned 0 |
2025-01-14 8:11:37 0 [Note] InnoDB: IO Error: 22 during write of 1024 bytes, for file app_user_499733/generic_collection_item_v001#P#d60680e60bd5c8c08f7c7e5f6a5e3fb0.ibd(643), returned 0 |
2025-01-14 8:11:37 0 [Note] InnoDB: IO Error: 22 during write of 1024 bytes, for file app_user_499733/generic_collection_item_v001#P#ecd3e991611e4490e6c0d24be5acd153.ibd(642), returned 0 |
2025-01-14 8:11:37 0 [Note] InnoDB: IO Error: 22 during write of 1024 bytes, for file app_user_499733/generic_collection_item_v001#P#ecd3e991611e4490e6c0d24be5acd153.ibd(642), returned 0 |
2025-01-14 8:11:37 0 [Note] InnoDB: IO Error: 22 during write of 1024 bytes, for file app_user_383118/generic_collection_item_v001#P#46afa26fdeeb6032382866849abdc2a9.ibd(641), returned 0 |
2025-01-14 8:11:37 0 [Note] InnoDB: IO Error: 22 during write of 1024 bytes, for file app_user_383118/generic_collection_item_v001#P#46afa26fdeeb6032382866849abdc2a9.ibd(641), returned 0 |
2025-01-14 8:11:37 0 [Note] InnoDB: IO Error: 22 during write of 2560 bytes, for file app_user_383118/generic_collection_item_v001#P#d5cf2b1863d4bf45f03a38e706ee7058.ibd(640), returned 0 |
2025-01-14 8:11:37 0 [Note] InnoDB: IO Error: 22 during write of 1024 bytes, for file app_user_383118/generic_collection_item_v001#P#d5cf2b1863d4bf45f03a38e706ee7058.ibd(640), returned 0 |
2025-01-14 8:11:37 0 [Note] InnoDB: IO Error: 22 during write of 1024 bytes, for file app_user_383118/generic_collection_item_v001#P#d5cf2b1863d4bf45f03a38e706ee7058.ibd(640), returned 0 |
Do you have an idea what could cause this?
Attachments
Activity
| Transition | Time In Source Status | Execution Times |
|---|
|
2d 16h 41m | 1 |
|
29d 8h | 1 |