Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-26356

Performance regression after dict_sys.mutex removal

    XMLWordPrintable

    Details

      Description

      Users can configure the number of purge threads to use.

      Currently, the existing logic looks at the increasing history length and accordingly increases the purge threads (respecting the user-set threshold). User active user workload causes history length to increase that, in turn, leads to more purge threads to get scheduled.

      Purge operation generates redo-records in addition to user workload redo-records. This dual redo generations cause intense pressure on redo-log which can easily cause redo-log to hit the threshold and there-by-causing jitter in overall throughput (including furious flushing).
      In order to improve the user workload performance and it is important to ensure background activity like a purge is not overdone while ensuring history length is kept in check to avoid an increase in select latency.

      The proposed patch explores an adaptive purge thread scheduling approach based on the redo log fill factor. Logic tends to dynamically increase or decrease the purge threads based on how much redo log is filled, leaving enough space/resources/flush bandwidth for the user workload.

      Testing done so far has revealed quite encouraging results especially with slower disk where-in flush is unable to catch up with redo log generation rate. Increasing in history length doesn't tend to have a regressing effect on the queries.

        Attachments

        1. adaptive_purge.gnumeric
          25 kB
        2. adaptive_purge.tar.gz
          63 kB
        3. arm.pt16.8.png
          arm.pt16.8.png
          147 kB
        4. arm.pt4.slowdisk.png
          arm.pt4.slowdisk.png
          165 kB
        5. arm.pt8.slowdisk.png
          arm.pt8.slowdisk.png
          166 kB
        6. purge-thread=8 + nvme + cpu bound.png
          purge-thread=8 + nvme + cpu bound.png
          38 kB
        7. purge-thread=8 + nvme disk + iobound workload.png
          purge-thread=8 + nvme disk + iobound workload.png
          38 kB
        8. x86.pt16.8.png
          x86.pt16.8.png
          153 kB
        9. x86.pt4.slowdisk.png
          x86.pt4.slowdisk.png
          161 kB
        10. x86.pt8.slowdisk.png
          x86.pt8.slowdisk.png
          166 kB

          Issue Links

            Activity

              People

              Assignee:
              marko Marko Mäkelä
              Reporter:
              krunalbauskar Krunal Bauskar
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Git Integration