Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-15252

DELETE with FKs crashes Galera nodes if wsrep_slave_threads>1

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Fixed
    • 10.2.12, 10.2.13, 10.3.5
    • 10.2.15
    • Galera
    • None

    Description

      It just takes a two(with more is the same) nodes Galera cluster with wsrep_slave_threads>1 on the Slave node(s).
      On the Master node import the sql files in sequence and you'll get the crash with error log full of messages of the type:

      2018-02-08 11:41:04 140048415356672 [Note] InnoDB: SQL1: DELETE FROM `CA9176ed2945`/)|Z�'
      2018-02-08 11:41:04 140048415356672 [Note] InnoDB: SQL2:
      2018-02-08 11:41:04 140048415356672 [ERROR] mysqld: Can't find record in 'CA9176ed2945'
      2018-02-08 11:41:04 140048415356672 [ERROR] Slave SQL: Could not execute Delete_rows_v1 event on table c17580.CA9176ed2945; Can't find record in 'CA9176ed2945', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log FIRST, end_log_pos 799009, Internal MariaDB error code: 1032

      Attachments

        1. c17580.tar.gz
          2.07 MB
        2. server1.cnf
          1 kB
        3. server2.cnf
          1 kB
        4. error_log.txt
          33 kB

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MDEV-15611 Due to the failure of foreign key detection, Galera slave node killed himself.

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            View 8 older comments
            sachin.setiya.007 Sachin Setiya (Inactive) added a comment -

            it is independent of sst_method and innodb_autoinc_lock_mode.

            sachin.setiya.007 Sachin Setiya (Inactive) added a comment - it is independent of sst_method and innodb_autoinc_lock_mode.
            sachin.setiya.007 Sachin Setiya (Inactive) added a comment -

            Patch for mdev-15611 also solves this (test case is passing without failing http://lists.askmonty.org/pipermail/commits/2018-April/012274.html )

            sachin.setiya.007 Sachin Setiya (Inactive) added a comment - Patch for mdev-15611 also solves this (test case is passing without failing http://lists.askmonty.org/pipermail/commits/2018-April/012274.html )
            winstone Zdravelina Sokolovska (Inactive) added a comment -

            appears also on 10.3.5 

            2018-04-04 17:30:33 2 [Note] InnoDB:  SQL1: DELETE FROM  `CA9176ed2945`{▒▒Z SQL2: DELETE FROM  `VA13db63946f`▒▒▒Z
            		 
            2018-04-04 17:30:33 2 [ERROR] mysqld: Can't find record in 'CA9176ed2945'
            		 
            2018-04-04 17:30:33 2 [ERROR] Slave SQL: Could not execute Delete_rows_v1 event on table md_15252.CA9176ed2945; Can't find record in 'CA9176ed2945', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log FIRST, end_log_pos 2251485, Internal MariaDB error code: 1032
            		 
            2018-04-04 17:30:34 2 [Warning] WSREP: RBR event 281 Delete_rows_v1 apply warning: 120, 257581
            		 
            2018-04-04 17:30:34 2 [ERROR] WSREP: Failed to apply trx: source: 6557e5a0-3743-11e8-8941-fe255bc5c908 version: 3 local: 0 state: APPLYING flags: 1 conn_id: 105705 trx_id: 697453 seqnos (l: 110076, g: 257581, s: 257580, d: 257580, ts: 89926014190718)
            		 
            2018-04-04 17:30:34 2 [ERROR] WSREP: Failed to apply trx 257581 4 times
            		 
            2018-04-04 17:30:34 2 [ERROR] WSREP: Node consistency compromised, aborting...
            		 
            2018-04-04 17:30:34 2 [Note] WSREP: Closing send monitor...
            		 
            2018-04-04 17:30:34 2 [Note] WSREP: Closed send monitor.
            		 
            2018-04-04 17:30:34 2 [Note] WSREP: gcomm: terminating thread
            		 
            2018-04-04 17:30:34 2 [Note] WSREP: gcomm: joining thread
            		 
            2018-04-04 17:30:34 2 [Note] WSREP: gcomm: closing backend

            winstone Zdravelina Sokolovska (Inactive) added a comment - appears also on 10.3.5 2018-04-04 17:30:33 2 [Note] InnoDB: SQL1: DELETE FROM `CA9176ed2945`{▒▒Z SQL2: DELETE FROM `VA13db63946f`▒▒▒Z 2018-04-04 17:30:33 2 [ERROR] mysqld: Can't find record in 'CA9176ed2945' 2018-04-04 17:30:33 2 [ERROR] Slave SQL: Could not execute Delete_rows_v1 event on table md_15252.CA9176ed2945; Can't find record in 'CA9176ed2945', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log FIRST, end_log_pos 2251485, Internal MariaDB error code: 1032 2018-04-04 17:30:34 2 [Warning] WSREP: RBR event 281 Delete_rows_v1 apply warning: 120, 257581 2018-04-04 17:30:34 2 [ERROR] WSREP: Failed to apply trx: source: 6557e5a0-3743-11e8-8941-fe255bc5c908 version: 3 local: 0 state: APPLYING flags: 1 conn_id: 105705 trx_id: 697453 seqnos (l: 110076, g: 257581, s: 257580, d: 257580, ts: 89926014190718) 2018-04-04 17:30:34 2 [ERROR] WSREP: Failed to apply trx 257581 4 times 2018-04-04 17:30:34 2 [ERROR] WSREP: Node consistency compromised, aborting... 2018-04-04 17:30:34 2 [Note] WSREP: Closing send monitor... 2018-04-04 17:30:34 2 [Note] WSREP: Closed send monitor. 2018-04-04 17:30:34 2 [Note] WSREP: gcomm: terminating thread 2018-04-04 17:30:34 2 [Note] WSREP: gcomm: joining thread 2018-04-04 17:30:34 2 [Note] WSREP: gcomm: closing backend
            sachin.setiya.007 Sachin Setiya (Inactive) added a comment -

            patch for mdev-15611 solves this

            sachin.setiya.007 Sachin Setiya (Inactive) added a comment - patch for mdev-15611 solves this
            sachin.setiya.007 Sachin Setiya (Inactive) added a comment -

            Solved by mdev-15611

            sachin.setiya.007 Sachin Setiya (Inactive) added a comment - Solved by mdev-15611

            People

              sachin.setiya.007 Sachin Setiya (Inactive)
              claudio.nanni Claudio Nanni
              Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.