Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-30458

Consolidate Serial Replica to Parallel Replica with 1 Worker Thread

Details

    • Task
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • None
    • Replication
    • None

    Description

      To reduce code duplication and behavioral inconsistencies, the serial slave's SQL thread can be replaced with the parallel replication thread pool, and only using a single thread. Some of these inconsistencies can be seen by MDEV-17516 and MDEV-29639, which showcase different and inaccurate Seconds_Behind_Master behaviors between the serial/parallel versions. Treating the serial replica as slave_parallel_threads=1 will allow for both consistent and accurate Seconds_Behind_Master values, as well as reduce code bloat.
      A user setting slave_parallel_threads=0 should automatically change to 1, and issue a warning to the user.

      The following items are additional improvements that can be made:
      1. We can reduce the server memory footprint and improve load balancing by removing the parallel worker queues being in memory. Instead, we can treat the relay log file as a single worker queue which all worker threads pull. The workers would know where to pull from by introducing a new status variable to track the GTID state of queued events, and in-combination with MDEV-4991 (GTID Binlog Indexing), will know where to read in the relay log for the next events. Additional analysis needs to be done here to quantify these benefits.
      2. Set the default slave_parallel_threads value to 2 to increase default concurrency, and set warnings when slave_parallel_threads is 1 or 0.

      Attachments

        Issue Links

          Activity

            Performed a quick benchmark to compare a serial slave, against a parallel replica with one worker thread. The workload of the benchmark consisted of updates to a table with a single row. The first graph uses log-slave-updates, and the second is without log-slave-updates. There doesn't seem to be a concrete performance difference between the two methods.

            bnestere Brandon Nesterenko added a comment - Performed a quick benchmark to compare a serial slave, against a parallel replica with one worker thread. The workload of the benchmark consisted of updates to a table with a single row. The first graph uses log-slave-updates, and the second is without log-slave-updates. There doesn't seem to be a concrete performance difference between the two methods.

            People

              bnestere Brandon Nesterenko
              bnestere Brandon Nesterenko
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.