[MDEV-7124] Slave SQL: Could not execute Delete_rows event on table Created: 2014-11-17  Updated: 2021-12-23  Resolved: 2021-12-23

Status: Closed
Project: MariaDB Server
Component/s: Galera, Replication, Storage Engine - XtraDB
Affects Version/s: 5.5.40-galera
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: j Assignee: Jan Lindström (Inactive)
Resolution: Won't Fix Votes: 1
Labels: galera
Environment:

The environment uses Galera WAN replication through an IPsec VPN with 100ms latency. We have have three nodes, one in each datacenter:

  • MariaDB 5.5.40 in datacenter A, CentOS 6.5.
  • MariaDB 5.5.40 in datacenter B, CentOS 6.5.
  • Galera Arbitrator 25.3.5.rXXXX in datacenter C, AMI Linux.


 Description   

Galera node crashed with data inconsistency from DELETE FROM:

141116 11:19:37 [ERROR] Slave SQL: Could not execute Delete_rows event on table db.tbl; Can't find record in 'tbl', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log FIRST, end_log_pos 12342, Error_code: 1032
141116 11:19:37 [Warning] WSREP: RBR event 2 Delete_rows apply warning: 120, 11183
141116 11:19:37 [Warning] WSREP: Failed to apply app buffer: seqno: 11183, status: 1
141116 11:19:37 [ERROR] Slave SQL: Could not execute Delete_rows event on table db.tbl; Can't find record in 'tbl', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log FIRST, end_log_pos 12342, Error_code: 1032
141116 11:19:37 [Warning] WSREP: RBR event 2 Delete_rows apply warning: 120, 11183
141116 11:19:37 [Warning] WSREP: Failed to apply app buffer: seqno: 11183, status: 1
141116 11:19:37 [ERROR] Slave SQL: Could not execute Delete_rows event on table db.tbl; Can't find record in 'tbl', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log FIRST, end_log_pos 12342, Error_code: 1032
141116 11:19:37 [Warning] WSREP: RBR event 2 Delete_rows apply warning: 120, 11183
141116 11:19:37 [Warning] WSREP: Failed to apply app buffer: seqno: 11183, status: 1
141116 11:19:37 [ERROR] Slave SQL: Could not execute Delete_rows event on table db.tbl; Can't find record in 'tbl', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log FIRST, end_log_pos 12342, Error_code: 1032
141116 11:19:37 [Warning] WSREP: RBR event 2 Delete_rows apply warning: 120, 11183
141116 11:19:37 [ERROR] WSREP: Failed to apply trx: source: adff90d6-6c2b-11e4-9b59-16bcb63464a5 version: 3 local: 0 state: APPLYING flags: 1 conn_id: 2256 trx_id: 400584 seqnos (l: 2116, g: 11183, s: 11181, d: 11182, ts: 4794085928072857)
141116 11:19:37 [ERROR] WSREP: Failed to apply trx 11183 4 times
141116 11:19:37 [ERROR] WSREP: Node consistency compromized, aborting...
141116 11:19:37 [Note] WSREP: Closing send monitor...
141116 11:19:37 [Note] WSREP: Closed send monitor.
141116 11:19:37 [Note] WSREP: gcomm: terminating thread
141116 11:19:37 [Note] WSREP: gcomm: joining thread
141116 11:19:37 [Note] WSREP: gcomm: closing backend
141116 11:19:37 [Note] WSREP: view(view_id(NON_PRIM,18f5b9e2-6c2c-11e4-8201-dbc615e79ec5,101) memb {
141116 11:19:37 [Note] WSREP: view((empty))
141116 11:19:37 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
141116 11:19:37 [Note] WSREP: gcomm: closed
141116 11:19:37 [Note] WSREP: Flow-control interval: [16, 16]
141116 11:19:37 [Note] WSREP: Received NON-PRIMARY.
141116 11:19:37 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 11183)
141116 11:19:37 [Note] WSREP: Received self-leave message.
141116 11:19:37 [Note] WSREP: Flow-control interval: [0, 0]
141116 11:19:37 [Note] WSREP: Received SELF-LEAVE. Closing connection.
141116 11:19:37 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 11183)
141116 11:19:37 [Note] WSREP: RECV thread exiting 0: Success
141116 11:19:37 [Note] WSREP: recv_thread() joined.
141116 11:19:37 [Note] WSREP: Closing replication queue.
141116 11:19:37 [Note] WSREP: Closing slave action queue.
141116 11:19:37 [Note] WSREP: /usr/sbin/mysqld: Terminated.
141116 11:19:37 mysqld_safe Number of processes running now: 0
141116 11:19:37 mysqld_safe WSREP: not restarting wsrep node automatically
141116 11:19:37 mysqld_safe mysqld from pid file /var/run/mysql/mysqld.pid ended1032



 Comments   
Comment by Nirbhay Choubey (Inactive) [ 2014-11-18 ]

Could you share the table structure (SHOW CREATE TABLE tbl) and server configurations (my.cnf)?

Comment by j [ 2014-11-19 ]

Seems to have been fixed in 10.0.6 in this bug: https://mariadb.atlassian.net/browse/MDEV-5217

Comment by Nirbhay Choubey (Inactive) [ 2015-01-05 ]

https://github.com/codership/mysql-wsrep/issues/40

Comment by Jan Lindström (Inactive) [ 2021-12-23 ]

5.5-galera is EOL.

Generated at Thu Feb 08 07:17:11 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.