Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-18104

MyRocks-Gap-Lock: range locking bounds are incorrect for multi-part keys

Details

    Description

      create table ten(a int);
      insert into ten values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9);
      create table t4 (
        kp1 int not null,
        kp2 int not null,
        a int,
        primary key(kp1, kp2)
      ) engine=rocksdb;
       
      insert into t4 select 1,a, 1234 from ten;
      insert into t4 values (2, 3, 1234);
      insert into t4 values (2, 5, 1234);
      insert into t4 values (2, 7, 1234);
      insert into t4 select 3,a, 1234 from ten;
      

      First, a good example (when it does work):

      Connection1:

      mysql> begin;
      Query OK, 0 rows affected (0.00 sec)
       
      mysql> select * from t4 where kp1=2 and kp2 between 1 and 10 for update;
      +-----+-----+------+
      | kp1 | kp2 | a    |
      +-----+-----+------+
      |   2 |   3 | 1234 |
      |   2 |   5 | 1234 |
      |   2 |   7 | 1234 |
      +-----+-----+------+
      3 rows in set (0.01 sec)
       
      mysql> select * from information_schema.rocksdb_locks;
      +------------------+----------------+-----------------------------------------------------+------+
      | COLUMN_FAMILY_ID | TRANSACTION_ID | KEY                                                 | MODE |
      +------------------+----------------+-----------------------------------------------------+------+
      |                0 |             26 | 000001068000000280000001 - 00000106800000028000000a | X    |
      +------------------+----------------+-----------------------------------------------------+------+
      1 row in set (0.00 sec)
      

      Connection2:

      mysql> insert into t4 values ( 2,9,9999);
      ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction: Timeout on index: test.t4.PRIMARY
      

      This is expected.
      Now, bad example:
      Connection 1:

      mysql> begin;
      Query OK, 0 rows affected (0.00 sec)
       
      mysql> select * from t4 where kp1=2 for update;
      +-----+-----+------+
      | kp1 | kp2 | a    |
      +-----+-----+------+
      |   2 |   3 | 1234 |
      |   2 |   5 | 1234 |
      |   2 |   7 | 1234 |
      +-----+-----+------+
      3 rows in set (0.00 sec)
      

      mysql> select * from information_schema.rocksdb_locks;
      +------------------+----------------+--------------------------+------+
      | COLUMN_FAMILY_ID | TRANSACTION_ID | KEY                      | MODE |
      +------------------+----------------+--------------------------+------+
      |                0 |             28 | 000001068000000280000007 | X    |
      |                0 |             28 | 000001068000000280000005 | X    |
      |                0 |             28 | 000001068000000280000003 | X    |
      |                0 |             28 | 0000010680000002         | X    |
      +------------------+----------------+--------------------------+------+
      4 rows in set (0.00 sec)
      

      Note the last line. Is this the range lock ??

      Indeed, its bounds are wrong and it does not inhibit the INSERT:
      Connection2:

      mysql> insert into t4 values ( 2,9,9999);
      Query OK, 1 row affected (0.01 sec)
      

      Attachments

        Issue Links

          Activity

            psergei Sergei Petrunia added a comment - - edited

            Keys are compared here:

              #0  toku_keycompare (key1=0x7fffcc022530, key1len=12, key2=0x7fffd8004fb0, key2len=12) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/standalone_port.cc:128
              #1  0x0000555556fc013f in toku_builtin_compare_fun (db=0x5555581dc3c0, a=0x5555581dd2f8, b=0x7ffff42cf330) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/standalone_port.cc:143
              #2  0x0000555557094934 in toku::comparator::operator() (this=0x5555581dd118, a=0x5555581dd2f8, b=0x7ffff42cf330) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/ft/comparator.h:126
              #3  0x0000555557094497 in toku::keyrange::compare (this=0x5555581dd2e0, cmp=..., range=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/keyrange.cc:103
              #4  0x00005555570945a1 in toku::keyrange::overlaps (this=0x5555581dd2e0, cmp=..., range=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/keyrange.cc:118
              #5  0x0000555556fb3e7c in toku::treenode::range_overlaps (this=0x5555581dd2b0, range=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/treenode.cc:95
              #6  0x0000555556fb4e6b in toku::concurrent_tree::locked_keyrange::acquire (this=0x7ffff42cf170, range=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/concurrent_tree.cc:72
              #7  0x0000555556fb5e47 in toku::locktree::acquire_lock_consolidated (this=0x5555581dd100, prepared_lkr=0x7ffff42cf170, txnid=32, left_key=0x7ffff42cf330, right_key=0x7ffff42cf350, conflicts=0x7ffff42cf2a0) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/locktree.cc:329
              #8  0x0000555556fb612c in toku::locktree::acquire_lock (this=0x5555581dd100, is_write_request=true, txnid=32, left_key=0x7ffff42cf330, right_key=0x7ffff42cf350, conflicts=0x7ffff42cf2a0) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/locktree.cc:382
              #9  0x0000555556fb61d2 in toku::locktree::try_acquire_lock (this=0x5555581dd100, is_write_request=true, txnid=32, left_key=0x7ffff42cf330, right_key=0x7ffff42cf350, conflicts=0x7ffff42cf2a0, big_txn=false) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/locktree.cc:399
              #10 0x0000555556fb6278 in toku::locktree::acquire_write_lock (this=0x5555581dd100, txnid=32, left_key=0x7ffff42cf330, right_key=0x7ffff42cf350, conflicts=0x7ffff42cf2a0, big_txn=false) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/locktree.cc:412
              #11 0x0000555556fb0d73 in toku::lock_request::start (this=0x7ffff42cf410) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/lock_request.cc:165
              #12 0x0000555556f721ad in rocksdb::RangeLockMgr::TryRangeLock (this=0x55555820d1a0, txn=0x7fffd800ba90, column_family_id=0, start_key=..., end_key=..., exclusive=false) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/rocksdb/utilities/transactions/transaction_lock_mgr.cc:376
              #13 0x0000555556f60722 in rocksdb::PessimisticTransactionDB::TryRangeLock (this=0x555558205c00, txn=0x7fffd800ba90, cfh_id=0, start_key=..., end_key=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/rocksdb/utilities/transactions/pessimistic_transaction_db.cc:400
              #14 0x0000555557071072 in rocksdb::PessimisticTransaction::GetRangeLock (this=0x7fffd800ba90, column_family=0x5555581d8f80, start_key=..., end_key=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/rocksdb/utilities/transactions/pessimistic_transaction.cc:601
              #15 0x0000555556b41cd3 in myrocks::Rdb_transaction_impl::lock_range (this=0x7fffd800e2d0, cf=0x5555581d8f80, start=..., end=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/ha_rocksdb.cc:2736
              #16 0x0000555556b1ddfe in myrocks::ha_rocksdb::set_range_lock (this=0x7fffd812f190, tx=0x7fffd800e2d0, kd=..., find_flag=@0x7ffff42d0ebc: HA_READ_KEY_OR_NEXT, slice=..., end_key=0x7fffd812f410) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/ha_rocksdb.cc:8177
              #17 0x0000555556b1e475 in myrocks::ha_rocksdb::index_read_map_impl (this=0x7fffd812f190, buf=0x7fffd812d100 "\377\002", key=0x7fffd80b2aa8 "\002", keypart_map=3, find_flag=HA_READ_KEY_OR_NEXT, end_key=0x7fffd812f410) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/ha_rocksdb.cc:8303
              #18 0x0000555556b1da0d in myrocks::ha_rocksdb::read_range_first (this=0x7fffd812f190, start_key=0x7fffd812f3f0, end_key=0x7fffd812f410, eq_range_arg=false, sorted=false) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/ha_rocksdb.cc:8091
              #19 0x0000555556232be8 in handler::multi_range_read_next (this=0x7fffd812f190, range_info=0x7ffff42d1078) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/sql/handler.cc:6392
            

            toku_keycompare is basically "memcmp(), and longer keys are greater than shorter ones".

            psergei Sergei Petrunia added a comment - - edited Keys are compared here: #0 toku_keycompare (key1=0x7fffcc022530, key1len=12, key2=0x7fffd8004fb0, key2len=12) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/standalone_port.cc:128 #1 0x0000555556fc013f in toku_builtin_compare_fun (db=0x5555581dc3c0, a=0x5555581dd2f8, b=0x7ffff42cf330) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/standalone_port.cc:143 #2 0x0000555557094934 in toku::comparator::operator() (this=0x5555581dd118, a=0x5555581dd2f8, b=0x7ffff42cf330) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/ft/comparator.h:126 #3 0x0000555557094497 in toku::keyrange::compare (this=0x5555581dd2e0, cmp=..., range=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/keyrange.cc:103 #4 0x00005555570945a1 in toku::keyrange::overlaps (this=0x5555581dd2e0, cmp=..., range=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/keyrange.cc:118 #5 0x0000555556fb3e7c in toku::treenode::range_overlaps (this=0x5555581dd2b0, range=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/treenode.cc:95 #6 0x0000555556fb4e6b in toku::concurrent_tree::locked_keyrange::acquire (this=0x7ffff42cf170, range=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/concurrent_tree.cc:72 #7 0x0000555556fb5e47 in toku::locktree::acquire_lock_consolidated (this=0x5555581dd100, prepared_lkr=0x7ffff42cf170, txnid=32, left_key=0x7ffff42cf330, right_key=0x7ffff42cf350, conflicts=0x7ffff42cf2a0) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/locktree.cc:329 #8 0x0000555556fb612c in toku::locktree::acquire_lock (this=0x5555581dd100, is_write_request=true, txnid=32, left_key=0x7ffff42cf330, right_key=0x7ffff42cf350, conflicts=0x7ffff42cf2a0) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/locktree.cc:382 #9 0x0000555556fb61d2 in toku::locktree::try_acquire_lock (this=0x5555581dd100, is_write_request=true, txnid=32, left_key=0x7ffff42cf330, right_key=0x7ffff42cf350, conflicts=0x7ffff42cf2a0, big_txn=false) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/locktree.cc:399 #10 0x0000555556fb6278 in toku::locktree::acquire_write_lock (this=0x5555581dd100, txnid=32, left_key=0x7ffff42cf330, right_key=0x7ffff42cf350, conflicts=0x7ffff42cf2a0, big_txn=false) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/locktree.cc:412 #11 0x0000555556fb0d73 in toku::lock_request::start (this=0x7ffff42cf410) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/lock_request.cc:165 #12 0x0000555556f721ad in rocksdb::RangeLockMgr::TryRangeLock (this=0x55555820d1a0, txn=0x7fffd800ba90, column_family_id=0, start_key=..., end_key=..., exclusive=false) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/rocksdb/utilities/transactions/transaction_lock_mgr.cc:376 #13 0x0000555556f60722 in rocksdb::PessimisticTransactionDB::TryRangeLock (this=0x555558205c00, txn=0x7fffd800ba90, cfh_id=0, start_key=..., end_key=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/rocksdb/utilities/transactions/pessimistic_transaction_db.cc:400 #14 0x0000555557071072 in rocksdb::PessimisticTransaction::GetRangeLock (this=0x7fffd800ba90, column_family=0x5555581d8f80, start_key=..., end_key=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/rocksdb/utilities/transactions/pessimistic_transaction.cc:601 #15 0x0000555556b41cd3 in myrocks::Rdb_transaction_impl::lock_range (this=0x7fffd800e2d0, cf=0x5555581d8f80, start=..., end=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/ha_rocksdb.cc:2736 #16 0x0000555556b1ddfe in myrocks::ha_rocksdb::set_range_lock (this=0x7fffd812f190, tx=0x7fffd800e2d0, kd=..., find_flag=@0x7ffff42d0ebc: HA_READ_KEY_OR_NEXT, slice=..., end_key=0x7fffd812f410) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/ha_rocksdb.cc:8177 #17 0x0000555556b1e475 in myrocks::ha_rocksdb::index_read_map_impl (this=0x7fffd812f190, buf=0x7fffd812d100 "\377\002", key=0x7fffd80b2aa8 "\002", keypart_map=3, find_flag=HA_READ_KEY_OR_NEXT, end_key=0x7fffd812f410) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/ha_rocksdb.cc:8303 #18 0x0000555556b1da0d in myrocks::ha_rocksdb::read_range_first (this=0x7fffd812f190, start_key=0x7fffd812f3f0, end_key=0x7fffd812f410, eq_range_arg=false, sorted=false) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/ha_rocksdb.cc:8091 #19 0x0000555556232be8 in handler::multi_range_read_next (this=0x7fffd812f190, range_info=0x7ffff42d1078) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/sql/handler.cc:6392 toku_keycompare is basically "memcmp(), and longer keys are greater than shorter ones".

            Where is TokuDB's processing for different kinds range bounds ( [strictly] less than, etc?).
            Well, it's not there :

            create table t10 (
              pk int primary key ,
              a int
            ) engine=tokudb;
            insert into t10 select a,a from one_k;
             
            mysql> begin;
            Query OK, 0 rows affected (0.00 sec)
             
            mysql> select * from t10 where pk > 5 and pk < 10 for update;
            +----+------+
            | pk | a    |
            +----+------+
            |  6 |    6 |
            |  7 |    7 |
            |  8 |    8 |
            |  9 |    9 |
            +----+------+
            4 rows in set (0.01 sec)
            

            Connection 2:

            mysql> begin;
            Query OK, 0 rows affected (0.00 sec)
             
            mysql> select * from t10 where pk >= 10 and pk <= 15 for update;
            ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
            

            psergei Sergei Petrunia added a comment - Where is TokuDB's processing for different kinds range bounds ( [strictly] less than, etc?). Well, it's not there : create table t10 ( pk int primary key , a int ) engine=tokudb; insert into t10 select a,a from one_k;   mysql> begin ; Query OK, 0 rows affected (0.00 sec)   mysql> select * from t10 where pk > 5 and pk < 10 for update ; + ----+------+ | pk | a | + ----+------+ | 6 | 6 | | 7 | 7 | | 8 | 8 | | 9 | 9 | + ----+------+ 4 rows in set (0.01 sec) Connection 2: mysql> begin ; Query OK, 0 rows affected (0.00 sec)   mysql> select * from t10 where pk >= 10 and pk <= 15 for update ; ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction

            Fix pushed into the tree

            psergei Sergei Petrunia added a comment - Fix pushed into the tree

            People

              psergei Sergei Petrunia
              psergei Sergei Petrunia
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.