Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-31273

Optimize write to binlog

Details

    Description

      The current "legacy" method of writing transactions to binlog involves per-event actions, dealing with changes in the event header, which can be optimized away.

      In particular the event checksum can be computed as early as at ctor and the binlog write be done with one cache-to-cache copy.

      Expected performance gain may vary, and may need to estimated before taking on implementation, but intuitively it looks to be greater for small event size transactions.

      Attachments

        Activity

          Ascending order - Click to sort in descending order
          View 15 older comments
          Roel Roel Van de Paar added a comment -

          As no performance benefits were found from the implementation, returning this to Elkin.
          serg Elkin Please confirm if we will still proceed with implementing this patch given the findings.

          Roel Roel Van de Paar added a comment - As no performance benefits were found from the implementation, returning this to Elkin . serg Elkin Please confirm if we will still proceed with implementing this patch given the findings.
          knielsen Kristian Nielsen added a comment -

          Erm, no.

          As was already explained, we don't necessarily expect any measurable throughput improvement from this patch.

          I have attached a simple test script test_binlog_checksum_precompute_stall.pl that measures the time to commit a large transaction. On fast disk, I measure the stall from commit of large transaction with --binlog-legacy-event-pos being 30% longer than without it:

          legacy:
          		 
           
          		 
          Table rows: 200002
          		 
          Time for all commits: 0.171386
          		 
          Time for big commits: 0.171134
          		 
           
          		 
          precomputed:
          		 
           
          		 
          Table rows: 200002
          		 
          Time for all commits: 0.131941
          		 
          Time for big commits: 0.13176

          knielsen Kristian Nielsen added a comment - Erm, no. As was already explained, we don't necessarily expect any measurable throughput improvement from this patch. I have attached a simple test script test_binlog_checksum_precompute_stall.pl that measures the time to commit a large transaction. On fast disk, I measure the stall from commit of large transaction with --binlog-legacy-event-pos being 30% longer than without it: legacy:   Table rows: 200002 Time for all commits: 0.171386 Time for big commits: 0.171134   precomputed:   Table rows: 200002 Time for all commits: 0.131941 Time for big commits: 0.13176
          axel Axel Schwenke added a comment -

          I agree with knielsen. If it reduces bloat or removes redundant information from the binlog, then we should do this change in the binlog format. Regardless of performance improvements. Only counter argument would be a performance regression (but I see no chance for that)

          I just now discover that the tests on g5 were run with the default InnoDB redo log size of 4M. I meant to run them with a huge (16G) log, but somehow the trailing 'G' was lost from my.cnf. I will repeat the benchmark on g5.

          axel Axel Schwenke added a comment - I agree with knielsen . If it reduces bloat or removes redundant information from the binlog, then we should do this change in the binlog format. Regardless of performance improvements. Only counter argument would be a performance regression (but I see no chance for that) I just now discover that the tests on g5 were run with the default InnoDB redo log size of 4M. I meant to run them with a huge (16G) log, but somehow the trailing 'G' was lost from my.cnf. I will repeat the benchmark on g5.
          axel Axel Schwenke added a comment -

          I've rerun the OLTP tests on g5 with 16G InnoDB redo log and now I see no differences between new and legacy binlog: OLTP-MDEV-31273-g5.2.pdf

          The differences are no bigger than the usual fluctuations.

          axel Axel Schwenke added a comment - I've rerun the OLTP tests on g5 with 16G InnoDB redo log and now I see no differences between new and legacy binlog: OLTP-MDEV-31273-g5.2.pdf The differences are no bigger than the usual fluctuations.
          Roel Roel Van de Paar added a comment -

          Testing completed. OK to push.

          Roel Roel Van de Paar added a comment - Testing completed. OK to push.

          People

            knielsen Kristian Nielsen
            Elkin Andrei Elkin
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.