[MDEV-31658] Deadlock found when trying to get lock; try restarting transaction, Error_code: 1213; handler error HA_ERR_LOCK_DEADLOCK; the event's master log FIRST, end_log_pos xxx, Internal MariaDB error code: 1213 Created: 2023-07-10  Updated: 2023-11-17

Status: Open
Project: MariaDB Server
Component/s: Galera
Affects Version/s: 10.6.14
Fix Version/s: N/A

Type: Bug Priority: Critical
Reporter: Allen Lee (Inactive) Assignee: Alexey
Resolution: Unresolved Votes: 1
Labels: None

Issue Links:
Relates
relates to MDEV-32122 Node killing could cause applier to f... Closed

 Description   

cluster got frozen after joiner node hit the following error:

2023-07-10  2:20:50 10 [ERROR] Slave SQL: Could not execute Write_rows_compressed_v1 event on table db.tablex; Deadlock found when trying to get lock; try restarting transaction, Error_code: 1213; handler error HA_ERR_LOCK_DEADLOCK; the event's master log FIRST, end_log_pos 1583, Internal MariaDB error code: 1213
2023-07-10  2:20:50 103 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table db.tablex; Deadlock found when trying to get lock; try restarting transaction, Error_code: 1213; handler error HA_ERR_LOCK_DEADLOCK; the event's master log FIRST, end_log_pos 772, Internal MariaDB error code: 1213
2023-07-10  2:20:50 10 [Warning] WSREP: Event 3 Write_rows_compressed_v1 apply failed: 149, seqno 45474182196
2023-07-10  2:20:50 103 [Warning] WSREP: Event 3 Write_rows_v1 apply failed: 149, seqno 45474182201
2023-07-10  2:20:50 0 [Note] WSREP: Member 0(fx114_db3) initiates vote on e4e16f9c-8779-11eb-b356-c3718f5f11e0:45474182196,8b5f5fd24a23838b:  Deadlock found when trying to get lock; try restarting transaction, Error_code: 1213;
2023-07-10  2:20:50 0 [Note] WSREP: Member 2(fx114_db2) responds to vote on e4e16f9c-8779-11eb-b356-c3718f5f11e0:45474182196,0000000000000000: Success
2023-07-10  2:20:50 0 [Warning] WSREP: Received bogus VOTE message: 45474182196.0, from node f3b6f9e7-1ea8-11ee-bb0a-5bd84ee8291e, expected > 45474389658. Ignoring.
2023-07-10  2:20:50 0 [Note] WSREP: Member 1(fx114_db1) responds to vote on e4e16f9c-8779-11eb-b356-c3718f5f11e0:45474182196,0000000000000000: Success
2023-07-10  2:20:50 0 [Warning] WSREP: Received bogus VOTE message: 45474182196.0, from node cc3e6460-1ea6-11ee-88f2-4a01f1fdd917, expected > 45474389637. Ignoring.
2023-07-10  2:24:49 0 [Note] /usr/local/libexec/mariadbd (initiated by: unknown): Normal shutdown
2023-07-10  2:24:49 0 [Note] WSREP: Shutdown replication
2023-07-10  2:24:49 0 [Note] WSREP: Server status change joined -> disconnecting

An user noted that the following command didn't work in that state and Graceful shutdown executed on each node have no effect. Only possible way to stop the nodes is with kill -9.

SET GLOBAL wsrep_provider_options='pc.bootstrap=YES' ;
SET GLOBAL wsrep_provider_options="evs.evict=1de97dad-f609-11e5-8a50-ce2e621b0c42";



 Comments   
Comment by Alexey [ 2023-07-14 ]

allen.lee@mariadb.com,

Could you please check if there are any tables which might be referencing db.tablex through foreign keys?

Comment by Yakov Kushnirsky [ 2023-10-20 ]

posted DDL, variables and logs to MENT. Will delete and close when no longer needed.

Generated at Thu Feb 08 10:25:31 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.