[MDEV-18104] MyRocks-Gap-Lock: range locking bounds are incorrect for multi-part keys Created: 2018-12-29  Updated: 2019-01-08  Resolved: 2019-01-06

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - RocksDB
Fix Version/s: N/A

Type: Task Priority: Major
Reporter: Sergei Petrunia Assignee: Sergei Petrunia
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
PartOf
is part of MDEV-15603 Gap Lock support in MyRocks Stalled

 Description    

create table ten(a int);
 
insert into ten values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9);
 
create table t4 (
 
  kp1 int not null,
 
  kp2 int not null,
 
  a int,
 
  primary key(kp1, kp2)
 
) engine=rocksdb;
 
 
 
insert into t4 select 1,a, 1234 from ten;
 
insert into t4 values (2, 3, 1234);
 
insert into t4 values (2, 5, 1234);
 
insert into t4 values (2, 7, 1234);
 
insert into t4 select 3,a, 1234 from ten;

First, a good example (when it does work):

Connection1:

mysql> begin;
 
Query OK, 0 rows affected (0.00 sec)
 
 
 
mysql> select * from t4 where kp1=2 and kp2 between 1 and 10 for update;
 
+-----+-----+------+
 
| kp1 | kp2 | a    |
 
+-----+-----+------+
 
|   2 |   3 | 1234 |
 
|   2 |   5 | 1234 |
 
|   2 |   7 | 1234 |
 
+-----+-----+------+
 
3 rows in set (0.01 sec)
 
 
 
mysql> select * from information_schema.rocksdb_locks;
 
+------------------+----------------+-----------------------------------------------------+------+
 
| COLUMN_FAMILY_ID | TRANSACTION_ID | KEY                                                 | MODE |
 
+------------------+----------------+-----------------------------------------------------+------+
 
|                0 |             26 | 000001068000000280000001 - 00000106800000028000000a | X    |
 
+------------------+----------------+-----------------------------------------------------+------+
 
1 row in set (0.00 sec)

Connection2:

mysql> insert into t4 values ( 2,9,9999);
 
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction: Timeout on index: test.t4.PRIMARY

This is expected.
Now, bad example:
Connection 1: 

mysql> begin;
 
Query OK, 0 rows affected (0.00 sec)
 
 
 
mysql> select * from t4 where kp1=2 for update;
 
+-----+-----+------+
 
| kp1 | kp2 | a    |
 
+-----+-----+------+
 
|   2 |   3 | 1234 |
 
|   2 |   5 | 1234 |
 
|   2 |   7 | 1234 |
 
+-----+-----+------+
 
3 rows in set (0.00 sec)


mysql> select * from information_schema.rocksdb_locks;
 
+------------------+----------------+--------------------------+------+
 
| COLUMN_FAMILY_ID | TRANSACTION_ID | KEY                      | MODE |
 
+------------------+----------------+--------------------------+------+
 
|                0 |             28 | 000001068000000280000007 | X    |
 
|                0 |             28 | 000001068000000280000005 | X    |
 
|                0 |             28 | 000001068000000280000003 | X    |
 
|                0 |             28 | 0000010680000002         | X    |
 
+------------------+----------------+--------------------------+------+
 
4 rows in set (0.00 sec)

Note the last line. Is this the range lock ??

Indeed, its bounds are wrong and it does not inhibit the INSERT:
Connection2:

mysql> insert into t4 values ( 2,9,9999);
 
Query OK, 1 row affected (0.01 sec)



 Comments   
Comment by Sergei Petrunia [ 2018-12-29 ]

Keys are compared here:

  #0  toku_keycompare (key1=0x7fffcc022530, key1len=12, key2=0x7fffd8004fb0, key2len=12) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/standalone_port.cc:128
 
  #1  0x0000555556fc013f in toku_builtin_compare_fun (db=0x5555581dc3c0, a=0x5555581dd2f8, b=0x7ffff42cf330) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/standalone_port.cc:143
 
  #2  0x0000555557094934 in toku::comparator::operator() (this=0x5555581dd118, a=0x5555581dd2f8, b=0x7ffff42cf330) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/ft/comparator.h:126
 
  #3  0x0000555557094497 in toku::keyrange::compare (this=0x5555581dd2e0, cmp=..., range=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/keyrange.cc:103
 
  #4  0x00005555570945a1 in toku::keyrange::overlaps (this=0x5555581dd2e0, cmp=..., range=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/keyrange.cc:118
 
  #5  0x0000555556fb3e7c in toku::treenode::range_overlaps (this=0x5555581dd2b0, range=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/treenode.cc:95
 
  #6  0x0000555556fb4e6b in toku::concurrent_tree::locked_keyrange::acquire (this=0x7ffff42cf170, range=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/concurrent_tree.cc:72
 
  #7  0x0000555556fb5e47 in toku::locktree::acquire_lock_consolidated (this=0x5555581dd100, prepared_lkr=0x7ffff42cf170, txnid=32, left_key=0x7ffff42cf330, right_key=0x7ffff42cf350, conflicts=0x7ffff42cf2a0) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/locktree.cc:329
 
  #8  0x0000555556fb612c in toku::locktree::acquire_lock (this=0x5555581dd100, is_write_request=true, txnid=32, left_key=0x7ffff42cf330, right_key=0x7ffff42cf350, conflicts=0x7ffff42cf2a0) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/locktree.cc:382
 
  #9  0x0000555556fb61d2 in toku::locktree::try_acquire_lock (this=0x5555581dd100, is_write_request=true, txnid=32, left_key=0x7ffff42cf330, right_key=0x7ffff42cf350, conflicts=0x7ffff42cf2a0, big_txn=false) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/locktree.cc:399
 
  #10 0x0000555556fb6278 in toku::locktree::acquire_write_lock (this=0x5555581dd100, txnid=32, left_key=0x7ffff42cf330, right_key=0x7ffff42cf350, conflicts=0x7ffff42cf2a0, big_txn=false) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/locktree.cc:412
 
  #11 0x0000555556fb0d73 in toku::lock_request::start (this=0x7ffff42cf410) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/range_locking/locktree/lock_request.cc:165
 
  #12 0x0000555556f721ad in rocksdb::RangeLockMgr::TryRangeLock (this=0x55555820d1a0, txn=0x7fffd800ba90, column_family_id=0, start_key=..., end_key=..., exclusive=false) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/rocksdb/utilities/transactions/transaction_lock_mgr.cc:376
 
  #13 0x0000555556f60722 in rocksdb::PessimisticTransactionDB::TryRangeLock (this=0x555558205c00, txn=0x7fffd800ba90, cfh_id=0, start_key=..., end_key=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/rocksdb/utilities/transactions/pessimistic_transaction_db.cc:400
 
  #14 0x0000555557071072 in rocksdb::PessimisticTransaction::GetRangeLock (this=0x7fffd800ba90, column_family=0x5555581d8f80, start_key=..., end_key=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/rocksdb/utilities/transactions/pessimistic_transaction.cc:601
 
  #15 0x0000555556b41cd3 in myrocks::Rdb_transaction_impl::lock_range (this=0x7fffd800e2d0, cf=0x5555581d8f80, start=..., end=...) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/ha_rocksdb.cc:2736
 
  #16 0x0000555556b1ddfe in myrocks::ha_rocksdb::set_range_lock (this=0x7fffd812f190, tx=0x7fffd800e2d0, kd=..., find_flag=@0x7ffff42d0ebc: HA_READ_KEY_OR_NEXT, slice=..., end_key=0x7fffd812f410) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/ha_rocksdb.cc:8177
 
  #17 0x0000555556b1e475 in myrocks::ha_rocksdb::index_read_map_impl (this=0x7fffd812f190, buf=0x7fffd812d100 "\377\002", key=0x7fffd80b2aa8 "\002", keypart_map=3, find_flag=HA_READ_KEY_OR_NEXT, end_key=0x7fffd812f410) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/ha_rocksdb.cc:8303
 
  #18 0x0000555556b1da0d in myrocks::ha_rocksdb::read_range_first (this=0x7fffd812f190, start_key=0x7fffd812f3f0, end_key=0x7fffd812f410, eq_range_arg=false, sorted=false) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/storage/rocksdb/ha_rocksdb.cc:8091
 
  #19 0x0000555556232be8 in handler::multi_range_read_next (this=0x7fffd812f190, range_info=0x7ffff42d1078) at /home/psergey/dev-git/mysql-5.6-rangelocking2-rebase1/sql/handler.cc:6392

toku_keycompare is basically "memcmp(), and longer keys are greater than shorter ones".

Comment by Sergei Petrunia [ 2018-12-29 ]

Where is TokuDB's processing for different kinds range bounds ( [strictly] less than, etc?).
Well, it's not there :

create table t10 (
 
  pk int primary key ,
 
  a int
 
) engine=tokudb;
 
insert into t10 select a,a from one_k;
 
 
 
mysql> begin;
 
Query OK, 0 rows affected (0.00 sec)
 
 
 
mysql> select * from t10 where pk > 5 and pk < 10 for update;
 
+----+------+
 
| pk | a    |
 
+----+------+
 
|  6 |    6 |
 
|  7 |    7 |
 
|  8 |    8 |
 
|  9 |    9 |
 
+----+------+
 
4 rows in set (0.01 sec)

Connection 2:

mysql> begin;
 
Query OK, 0 rows affected (0.00 sec)
 
 
 
mysql> select * from t10 where pk >= 10 and pk <= 15 for update;
 
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction

Comment by Sergei Petrunia [ 2019-01-06 ]

Fix pushed into the tree

Generated at Thu Feb 08 08:41:32 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.