Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-20543

Latching order violation during B-tree operations

    Details

      Description

      It turns out that there are more causes to MDEV-14637 than the too small secondary index record size estimates that were fixed in MySQL 5.7.23.

      With the following patch (extracted from MDEV-19514) applied to MySQL 5.7 or MariaDB Server 10.2 or later, the bug is repeatable:

      diff --git a/storage/innobase/buf/buf0buf.cc b/storage/innobase/buf/buf0buf.cc
      index 4f9ecf10ec7..ecd90053d68 100644
      --- a/storage/innobase/buf/buf0buf.cc
      +++ b/storage/innobase/buf/buf0buf.cc
      @@ -4617,6 +4617,8 @@ got_block:
       		fix_block->made_dirty_with_no_latch = dirty_with_no_latch;
       	}
       
      +	rw_lock_x_lock(&block->lock);
      +	rw_lock_x_unlock(&block->lock);
       	mtr_memo_type_t	fix_type;
       
       	switch (rw_latch) {
      

      ./mtr --big-test --mysqld=--innodb-page-size=4k innodb.innodb_bulk_create_index
      

      mysql-5.7.27

      CURRENT_TEST: innodb.innodb_bulk_create_index
      mysqltest: In included file ./suite/innodb/include/innodb_bulk_create_index.inc at line 84:
      included from /mariadb/mysql-server/mysql-test/suite/innodb/t/innodb_bulk_create_index.test at line 41:
      At line 84: query 'SELECT * FROM t1 WHERE title = 'a3000'' failed: 2013: Lost connection to MySQL server during query
      Version: '5.7.27-debug-log'  socket: '/dev/shm/5.7/mysql-test/var/tmp/mysqld.1.sock'  port: 13000  Source distribution
      ########################################
      DEADLOCK of threads detected!
      rw-lock 0x7f188e748968 --Thread 139743545079552 has waited at buf0buf.cc line 4620 for 0  seconds the semaphore:
      X-lock on RW-latch at 0x7f188e748968 created in file buf0buf.cc line 1460
      a writer (thread id 139743127922432) has reserved it in mode  exclusive
      number of readers 0, waiters flag 1, lock_word: 0
      Last time read locked in file row0row.cc line 1075
      Last time write locked in file /mariadb/mysql-server/storage/innobase/include/mtr0mtr.ic line 153
      Locked: thread 139743127922432 file mtr0mtr.ic line 153  X-LOCK
      Locked: thread 139743127922432 file mtr0mtr.ic line 153  X-LOCK
      rw-lock 0x7f188e72a030 --Thread 139743127922432 has waited at buf0buf.cc line 4620 for 0  seconds the semaphore:
      X-lock (wait_ex) on RW-latch at 0x7f188e72a030 created in file buf0buf.cc line 1460
      a writer (thread id 139743127922432) has reserved it in mode  wait exclusive
      number of readers 1, waiters flag 0, lock_word: ffffffffefffffff
      Last time read locked in file btr0cur.cc line 5736
      Last time write locked in file /mariadb/mysql-server/storage/innobase/include/mtr0mtr.ic line 117
      Locked: thread 139743545079552 file btr0cur.cc line 5736  S-LOCK
      Locked: thread 139743545079552 file btr0cur.cc line 5736  S-LOCK
      2019-09-10T07:31:49.289423Z 0 [ERROR] [FATAL] InnoDB: ######################################## Deadlock Detected!
      

      My original work-around from MDEV-14637 (always acquire the index->lock in X mode for BTR_MODIFY_TREE operations) prevents the crash, but would introduce a very prominent performance regression.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                thiru Thirunarayanan Balathandayuthapani
                Reporter:
                marko Marko Mäkelä
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: