Details
Description
cluster got frozen after joiner node hit the following error:
2023-07-10 2:20:50 10 [ERROR] Slave SQL: Could not execute Write_rows_compressed_v1 event on table db.tablex; Deadlock found when trying to get lock; try restarting transaction, Error_code: 1213; handler error HA_ERR_LOCK_DEADLOCK; the event's master log FIRST, end_log_pos 1583, Internal MariaDB error code: 1213 |
2023-07-10 2:20:50 103 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table db.tablex; Deadlock found when trying to get lock; try restarting transaction, Error_code: 1213; handler error HA_ERR_LOCK_DEADLOCK; the event's master log FIRST, end_log_pos 772, Internal MariaDB error code: 1213 |
2023-07-10 2:20:50 10 [Warning] WSREP: Event 3 Write_rows_compressed_v1 apply failed: 149, seqno 45474182196 |
2023-07-10 2:20:50 103 [Warning] WSREP: Event 3 Write_rows_v1 apply failed: 149, seqno 45474182201 |
2023-07-10 2:20:50 0 [Note] WSREP: Member 0(fx114_db3) initiates vote on e4e16f9c-8779-11eb-b356-c3718f5f11e0:45474182196,8b5f5fd24a23838b: Deadlock found when trying to get lock; try restarting transaction, Error_code: 1213; |
2023-07-10 2:20:50 0 [Note] WSREP: Member 2(fx114_db2) responds to vote on e4e16f9c-8779-11eb-b356-c3718f5f11e0:45474182196,0000000000000000: Success |
2023-07-10 2:20:50 0 [Warning] WSREP: Received bogus VOTE message: 45474182196.0, from node f3b6f9e7-1ea8-11ee-bb0a-5bd84ee8291e, expected > 45474389658. Ignoring. |
2023-07-10 2:20:50 0 [Note] WSREP: Member 1(fx114_db1) responds to vote on e4e16f9c-8779-11eb-b356-c3718f5f11e0:45474182196,0000000000000000: Success |
2023-07-10 2:20:50 0 [Warning] WSREP: Received bogus VOTE message: 45474182196.0, from node cc3e6460-1ea6-11ee-88f2-4a01f1fdd917, expected > 45474389637. Ignoring. |
2023-07-10 2:24:49 0 [Note] /usr/local/libexec/mariadbd (initiated by: unknown): Normal shutdown |
2023-07-10 2:24:49 0 [Note] WSREP: Shutdown replication |
2023-07-10 2:24:49 0 [Note] WSREP: Server status change joined -> disconnecting |
An user noted that the following command didn't work in that state and Graceful shutdown executed on each node have no effect. Only possible way to stop the nodes is with kill -9.
SET GLOBAL wsrep_provider_options='pc.bootstrap=YES' ; |
SET GLOBAL wsrep_provider_options="evs.evict=1de97dad-f609-11e5-8a50-ce2e621b0c42"; |
Attachments
Issue Links
- relates to
-
MDEV-32122 Node killing could cause applier to fail on write rows event leading inconsistency
-
- Closed
-
-
-
- Stalled
-