Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-27805

tpcc workload shows regression with MDB-10.6

Details

    Description

      Steps to reproduce:

      1. Load TPCC data set (for the experiment I have used --tables=10 --scale=40) given my 24 cores VM.

      2. Run the TPCC workload.

      Retry (1) and (2) with 10.5 and 10.6. You will observe the significant performance difference.
      check the graph for reference.

      If you try to do (1) with 10.5 (loading with 10.5) and (2) with 10.6 you would see that performance is restored back to the original level as set by 10.5.

      What is going wrong?
      1. there have been multiple changes in the recent 10.6.5 timeframe (#25919) around stats recalculation. It has been observed that with 10.6.5 during/post-loading background stats calculation is not taking place. (This could be confirmed by querying mysql.innodb_table_stats post loading).

      2. Behavior is intermittent and also could also get affected by the configuration of load and machine though there is no set pattern observed. [I was able to reproduce it majority of the times (9 out of 10 times). One suggestion to improve reproducibility: load the machine enough to keep CPU busy with loading of the data].

      Attachments

        1. arm - tpcc workload.png
          19 kB
          Krunal Bauskar
        2. x86 - tpcc workload.png
          17 kB
          Krunal Bauskar

        Issue Links

          is caused by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-25919 InnoDB reports misleading lock wait timeout on DDL operations

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27629 Performance problems due to incorrect table statistics after database imports

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-28327 InnoDB persistent statistics fail to update after bulk insert

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            krunalbauskar Krunal Bauskar added a comment -

            Work-around for the said problem: run Analyze table and re-run the queries things should be back to normal.
            (if you are upgrading old data-directory to use new MDB version things you will not observe the problem).

            krunalbauskar Krunal Bauskar added a comment - Work-around for the said problem: run Analyze table and re-run the queries things should be back to normal. (if you are upgrading old data-directory to use new MDB version things you will not observe the problem).
            krunalbauskar Krunal Bauskar added a comment - - edited

            MDEV-27805: tpcc workload shows regression with MDB-10.6

            • regression got revealed while running tpcc workload.
            • as part of MDEV-25919 changes logic for statistics computation was revamped.
            • if the table has changed to certain threshold then table is added to
              statistics recomputation queue (dict_stats_recalc_pool_add)
            • after the table is added to queue the background statistics thread is
              notified
            • during revamp the condition to notify background statistics threads was
              wrongly updated to check if the queue/vector is empty when it should
              check if there is queue/vector has entries to process.
            • vec.begin() == vec.end() : only when vector is empty
            • also accessing these iterator outside the parallely changing vector is not
              safe
            • fix now tend to notify background statistics thread if the logic adds
              an entry to the queue/vector.

            (patch: https://github.com/MariaDB/server/pull/2014)

            krunalbauskar Krunal Bauskar added a comment - - edited MDEV-27805 : tpcc workload shows regression with MDB-10.6 regression got revealed while running tpcc workload. as part of MDEV-25919 changes logic for statistics computation was revamped. if the table has changed to certain threshold then table is added to statistics recomputation queue (dict_stats_recalc_pool_add) after the table is added to queue the background statistics thread is notified during revamp the condition to notify background statistics threads was wrongly updated to check if the queue/vector is empty when it should check if there is queue/vector has entries to process. vec.begin() == vec.end() : only when vector is empty also accessing these iterator outside the parallely changing vector is not safe fix now tend to notify background statistics thread if the logic adds an entry to the queue/vector. (patch: https://github.com/MariaDB/server/pull/2014 )

            People

              marko Marko Mäkelä
              krunalbauskar Krunal Bauskar
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.