Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-30458

Consolidate Serial Replica to Parallel Replica with 1 Worker Thread

    XMLWordPrintable

Details

    • Task
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • None
    • Replication
    • None

    Description

      To reduce code duplication and behavioral inconsistencies, the serial slave's SQL thread can be replaced with the parallel replication thread pool, and only using a single thread. Some of these inconsistencies can be seen by MDEV-17516 and MDEV-29639, which showcase different and inaccurate Seconds_Behind_Master behaviors between the serial/parallel versions. Treating the serial replica as slave_parallel_threads=1 will allow for both consistent and accurate Seconds_Behind_Master values, as well as reduce code bloat.
      A user setting slave_parallel_threads=0 should automatically change to 1, and issue a warning to the user.

      The following items are additional improvements that can be made:
      1. We can reduce the server memory footprint and improve load balancing by removing the parallel worker queues being in memory. Instead, we can treat the relay log file as a single worker queue which all worker threads pull. The workers would know where to pull from by introducing a new status variable to track the GTID state of queued events, and in-combination with MDEV-4991 (GTID Binlog Indexing), will know where to read in the relay log for the next events. Additional analysis needs to be done here to quantify these benefits.
      2. Set the default slave_parallel_threads value to 2 to increase default concurrency, and set warnings when slave_parallel_threads is 1 or 0.

      Attachments

        1. repl_benchmark.png
          15 kB
          Brandon Nesterenko
        2. repl_benchmark-no_lsu.png
          14 kB
          Brandon Nesterenko

        Issue Links

          Activity

            People

              bnestere Brandon Nesterenko
              bnestere Brandon Nesterenko
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.