Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-26356

Performance regression after dict_sys.mutex removal

Details

    Description

      Users can configure the number of purge threads to use.

      Currently, the existing logic looks at the increasing history length and accordingly increases the purge threads (respecting the user-set threshold). User active user workload causes history length to increase that, in turn, leads to more purge threads to get scheduled.

      Purge operation generates redo-records in addition to user workload redo-records. This dual redo generations cause intense pressure on redo-log which can easily cause redo-log to hit the threshold and there-by-causing jitter in overall throughput (including furious flushing).
      In order to improve the user workload performance and it is important to ensure background activity like a purge is not overdone while ensuring history length is kept in check to avoid an increase in select latency.

      The proposed patch explores an adaptive purge thread scheduling approach based on the redo log fill factor. Logic tends to dynamically increase or decrease the purge threads based on how much redo log is filled, leaving enough space/resources/flush bandwidth for the user workload.

      Testing done so far has revealed quite encouraging results especially with slower disk where-in flush is unable to catch up with redo log generation rate. Increasing in history length doesn't tend to have a regressing effect on the queries.

      Attachments

        1. adaptive_purge.gnumeric
          25 kB
        2. adaptive_purge.tar.gz
          63 kB
        3. arm.pt16.8.png
          arm.pt16.8.png
          147 kB
        4. arm.pt4.slowdisk.png
          arm.pt4.slowdisk.png
          165 kB
        5. arm.pt8.slowdisk.png
          arm.pt8.slowdisk.png
          166 kB
        6. patched2-ssd40.txt
          57 kB
        7. patched2-ssd40-80.txt
          57 kB
        8. patched2-ssd40-80-32.txt
          57 kB
        9. purge-thread=8 + nvme + cpu bound.png
          purge-thread=8 + nvme + cpu bound.png
          38 kB
        10. purge-thread=8 + nvme disk + iobound workload.png
          purge-thread=8 + nvme disk + iobound workload.png
          38 kB
        11. x86.pt16.8.png
          x86.pt16.8.png
          153 kB
        12. x86.pt4.slowdisk.png
          x86.pt4.slowdisk.png
          161 kB
        13. x86.pt8.slowdisk.png
          x86.pt8.slowdisk.png
          166 kB

        Issue Links

          Activity

            krunalbauskar Krunal Bauskar created issue -
            krunalbauskar Krunal Bauskar made changes -
            Field Original Value New Value
            Attachment x86.pt16.8.png [ 58537 ]
            Attachment x86.pt8.slowdisk.png [ 58538 ]
            Attachment x86.pt4.slowdisk.png [ 58539 ]
            Attachment arm.pt16.8.png [ 58540 ]
            Attachment arm.pt8.slowdisk.png [ 58541 ]
            Attachment arm.pt4.slowdisk.png [ 58542 ]
            marko Marko Mäkelä made changes -
            Assignee Marko Mäkelä [ marko ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            Issue Type Task [ 3 ] Bug [ 1 ]
            marko Marko Mäkelä made changes -
            Fix Version/s 10.6 [ 24028 ]
            Fix Version/s 10.7 [ 24805 ]
            Affects Version/s 10.6 [ 24028 ]
            Affects Version/s 10.7 [ 24805 ]
            Labels performance purge
            Priority Major [ 3 ] Critical [ 2 ]
            Summary Adaptive purge threads scheduling based on redo log fill factor Performance regression after dict_sys.mutex removal
            krunalbauskar Krunal Bauskar made changes -
            krunalbauskar Krunal Bauskar made changes -
            marko Marko Mäkelä made changes -
            Attachment adaptive_purge.tar.gz [ 59024 ]
            Attachment adaptive_purge.gnumeric [ 59025 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            issue.field.resolutiondate 2021-09-16 17:47:13.0 2021-09-16 17:47:13.103
            marko Marko Mäkelä made changes -
            Fix Version/s 10.6.5 [ 26034 ]
            Fix Version/s 10.7.0 [ 26072 ]
            Fix Version/s 10.6 [ 24028 ]
            Fix Version/s 10.7 [ 24805 ]
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Closed [ 6 ]
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 124273 ] MariaDB v4 [ 159578 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            Attachment patched2-ssd40.txt [ 72131 ]
            marko Marko Mäkelä made changes -
            Attachment patched2-ssd40-80.txt [ 72132 ]
            marko Marko Mäkelä made changes -
            Attachment patched2-ssd40-80-32.txt [ 72133 ]

            People

              marko Marko Mäkelä
              krunalbauskar Krunal Bauskar
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.