Details

    Description

      Run some pre-release performance checks to make sure MyRocks build is adequate.

      Attachments

        1. run1-results.ods
          36 kB
        2. run2-results.ods
          21 kB
        3. screenshot-1.png
          screenshot-1.png
          101 kB
        4. screenshot-2.png
          screenshot-2.png
          74 kB
        5. screenshot-3.png
          screenshot-3.png
          49 kB
        6. screenshot-4.png
          screenshot-4.png
          59 kB
        7. screenshot-5.png
          screenshot-5.png
          91 kB
        8. sysbench-oltp-rw-simple-run1.tgz
          6 kB

        Issue Links

          Activity

            Debugging InnoDB on 10.2

            Master

            sysbench: 151 676 transactions
            Binlog_commits += 151 676
            Binlog_commit_groups += 95 717

            Slave (debug build):

            threads=20
            log_write_up_to calls = 375 744
            catchup time: 624.609522 sec.

            Slave without the tracing code (for comparison)

            sysbench on master: 145,089 transactions
            catchup time: 592.940083

            There are many more log_write_up_to calls than transactions

            psergei Sergei Petrunia added a comment - Debugging InnoDB on 10.2 Master sysbench: 151 676 transactions Binlog_commits += 151 676 Binlog_commit_groups += 95 717 Slave (debug build): threads=20 log_write_up_to calls = 375 744 catchup time: 624.609522 sec. Slave without the tracing code (for comparison) sysbench on master: 145,089 transactions catchup time: 592.940083 There are many more log_write_up_to calls than transactions

            Now slave catch-up takes ~595 sec,
            number of commits matches the expected one and we see that commits do happen
            in parallel:

            counter_value=145,158
            n_active histogram= {
              0, 0
              1, 36464
              2, 29865
              3, 21621
              4, 15590
              5, 11351
              6, 8336
              7, 6121
              8, 4456
              9, 3220
              10, 2371
              11, 1685
              12, 1231
              13, 852
              14, 624
              15, 436
              16, 308
              17, 231
              18, 177
              19, 143
              20, 76
              21, 0
              22, 0
              23, 0
              24, 0
              25, 0
              26, 0
              27, 0
              28, 0
              29, 0
              30, 0
              31, 0
            }
            

            psergei Sergei Petrunia added a comment - Now slave catch-up takes ~595 sec, number of commits matches the expected one and we see that commits do happen in parallel: counter_value=145,158 n_active histogram= { 0, 0 1, 36464 2, 29865 3, 21621 4, 15590 5, 11351 6, 8336 7, 6121 8, 4456 9, 3220 10, 2371 11, 1685 12, 1231 13, 852 14, 624 15, 436 16, 308 17, 231 18, 177 19, 143 20, 76 21, 0 22, 0 23, 0 24, 0 25, 0 26, 0 27, 0 28, 0 29, 0 30, 0 31, 0 }
            psergei Sergei Petrunia added a comment - - edited

            Using the same test setup as before (145K transactions, slave with slave_parallel_workers=20, _mode=conservative, NO log-slave-updates) to look at which code in innobase_commit executes in parallel:

            This piece of code is executed by one thread at a time :

                if (commit_trx
                        || (!thd_test_options(thd, OPTION_NOT_AUTOCOMMIT | OPTION_BEGIN))) {
             
                            DBUG_EXECUTE_IF("crash_innodb_before_commit",
                                            DBUG_SUICIDE(););
             
             
                            /* Run the fast part of commit if we did not already. */
                            if (!trx_is_active_commit_ordered(trx)) {
                                    innobase_commit_ordered_2(trx, thd);
             
                            }
             
                            /* We were instructed to commit the whole transaction, or
                            this is an SQL statement end and autocommit is on */
             
                            /* At this point commit order is fixed and transaction is
                            visible to others. So we can wakeup other commits waiting for
                            this one, to allow then to group commit with us. */
            

            histogram:

             n_active histogram= {
               0, 0
               1, 145158
               2, 0
               3, 0
            

            ...
            while this part of code runs in parallel, most likely due to the first call it is making:

                            thd_wakeup_subsequent_commits(thd, 0);
             
             
                            /* Now do a write + flush of logs. */
                            trx_commit_complete_for_mysql(trx);
             
                            trx_deregister_from_2pc(trx);
                    } else {
            

            histogram:

            n_active histogram= {
              0, 0
              1, 39488
              2, 29460
              3, 20922
              4, 14941
              5, 10823
              6, 7922
              7, 5884
              8, 4360
              9, 3136
              10, 2266
              11, 1659
              12, 1234
              13, 880
              14, 632
              15, 479
              16, 360
              17, 275
              18, 212
              19, 150
              20, 75
              21, 0
              22, 0
              23, 0
            ...
            }
            

            psergei Sergei Petrunia added a comment - - edited Using the same test setup as before (145K transactions, slave with slave_parallel_workers=20, _mode=conservative, NO log-slave-updates ) to look at which code in innobase_commit executes in parallel: This piece of code is executed by one thread at a time : if (commit_trx || (!thd_test_options(thd, OPTION_NOT_AUTOCOMMIT | OPTION_BEGIN))) {   DBUG_EXECUTE_IF( "crash_innodb_before_commit" , DBUG_SUICIDE(););     /* Run the fast part of commit if we did not already. */ if (!trx_is_active_commit_ordered(trx)) { innobase_commit_ordered_2(trx, thd);   }   /* We were instructed to commit the whole transaction, or this is an SQL statement end and autocommit is on */   /* At this point commit order is fixed and transaction is visible to others. So we can wakeup other commits waiting for this one, to allow then to group commit with us. */ histogram: n_active histogram= { 0, 0 1, 145158 2, 0 3, 0 ... while this part of code runs in parallel, most likely due to the first call it is making: thd_wakeup_subsequent_commits(thd, 0);     /* Now do a write + flush of logs. */ trx_commit_complete_for_mysql(trx);   trx_deregister_from_2pc(trx); } else { histogram: n_active histogram= { 0, 0 1, 39488 2, 29460 3, 20922 4, 14941 5, 10823 6, 7922 7, 5884 8, 4360 9, 3136 10, 2266 11, 1659 12, 1234 13, 880 14, 632 15, 479 16, 360 17, 275 18, 212 19, 150 20, 75 21, 0 22, 0 23, 0 ... }
            psergei Sergei Petrunia added a comment - - edited

            Filed MDEV-15372: Parallel slave speedup very limited when log_slave_updates=OFF, now testing a fix for it.

            psergei Sergei Petrunia added a comment - - edited Filed MDEV-15372 : Parallel slave speedup very limited when log_slave_updates=OFF , now testing a fix for it.

            RC release was made so closing this task

            psergei Sergei Petrunia added a comment - RC release was made so closing this task

            People

              psergei Sergei Petrunia
              psergei Sergei Petrunia
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.