Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-17874

MyRocks-Gap-Lock: Lock memory overhead

Details

    Description

      Several MyRocks tests currently fail, because they reach rocksdb_max_lock_memory and lock escalation is unable to reduce memory usage.

      One example is autoinc_vars_thread_2. AUTO_INCREMENT values are generated by each thread using auto_increment_offset which causes all adjacent row locks to belong to different transactions.

      Attachments

        Issue Links

          is part of

          Task - A task that needs to be done. MDEV-15603 Gap Lock support in MyRocks

          • Major - Major loss of function.
          • Stalled

          Activity

            Ascending order - Click to sort in descending order
            psergei Sergei Petrunia created issue -
            psergei Sergei Petrunia made changes -
            Field Original Value New Value
            psergei Sergei Petrunia added a comment -

            in spetrunia/mysql-5.6-rangelocking 

            toku::concurrent_tree::get_insertion_memory_overhead()=176

            in Percona Server:

             
            toku::concurrent_tree::get_insertion_memory_overhead()=192

            Add to that keyrange::get_memory_size() which is (sizeof(key_range)=72) + left_key_size +right_key_size (they are intentionally counted twice).

            In total, for pk INT PRIMARY KEY the lock memory usage is 264 bytes in 5.6-rangelocking (debug build) and 274 in debug build of Percona Server.

            psergei Sergei Petrunia added a comment - in spetrunia/mysql-5.6-rangelocking toku::concurrent_tree::get_insertion_memory_overhead()=176 in Percona Server: toku::concurrent_tree::get_insertion_memory_overhead()=192 Add to that keyrange::get_memory_size() which is (sizeof(key_range)=72) + left_key_size +right_key_size (they are intentionally counted twice). In total, for pk INT PRIMARY KEY the lock memory usage is 264 bytes in 5.6-rangelocking (debug build) and 274 in debug build of Percona Server.
            psergei Sergei Petrunia added a comment -

            In Percona Server's MTR environment: tokudb_max_lock_memory is not set, so tokudb_init_func sets it to 1/8 th of tokudb-cache-size which is 512M, so we get 64M.

            In MariaDB, the default value is tokudb_max_lock_memory = 1 035 604 992= 1G

            psergei Sergei Petrunia added a comment - In Percona Server's MTR environment: tokudb_max_lock_memory is not set, so tokudb_init_func sets it to 1/8 th of tokudb-cache-size which is 512M, so we get 64M. In MariaDB, the default value is tokudb_max_lock_memory = 1 035 604 992= 1G
            elenst Elena Stepanova made changes -
            Component/s Locking [ 10900 ]
            Component/s Storage Engine - RocksDB [ 13901 ]
            Component/s Tests [ 10800 ]
            Component/s OTHER [ 10125 ]
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.3 [ 22126 ]
            Affects Version/s 10.2 [ 14601 ]
            Affects Version/s 10.3 [ 22126 ]
            psergei Sergei Petrunia made changes -
            Affects Version/s N/A [ 14700 ]
            Affects Version/s 10.2 [ 14601 ]
            Affects Version/s 10.3 [ 22126 ]
            psergei Sergei Petrunia made changes -
            Fix Version/s N/A [ 14700 ]
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.3 [ 22126 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s N/A [ 14700 ]
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 90991 ] MariaDB v4 [ 141035 ]
            psergei Sergei Petrunia made changes -
            Fix Version/s N/A [ 14700 ]
            Fix Version/s 10.2 [ 14601 ]
            Resolution Won't Fix [ 2 ]
            Status Open [ 1 ] Closed [ 6 ]

            People

              psergei Sergei Petrunia
              psergei Sergei Petrunia
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.