Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-7502

Automatic provisioning of slave

    XMLWordPrintable

Details

    Description

      Idea

      The purpose of this task is to create an easy-to-use facility for setting up a
      new MariaDB replication slave.

      Setting up a new slave currently involves: 1) installing MariaDB with initial
      database; 2) point the slave to the master with CHANGE MASTER TO; 3) copying
      initial data from the master to the slave; and 4) starting the slave with
      START SLAVE. The idea is to automate step (3), which currently needs to be
      done manually.

      The syntax could be something as simple as

      LOAD DATA FROM MASTER

      This would then connect to the master that is currently configured. It will
      load a snapshot of all the data on the master, and leave the slave position at
      the point of the snapshot, ready for START SLAVE to continue replication from
      that point.

      Implementation:

      The idea is to do this non-blocking on the master, in a way that works for any
      storage engine. It will rely on row-based replication to be used between the
      master and the slave.

      At the start of LOAD DATA FROM MASTER, the slave will enter a special
      provisioning mode. It will start replicating events from the master at the
      master's current position.

      The master dump thread will send binlog events to the slave as normal. But in
      addition, it will interleave a dump of all the data on the master contained in
      tables, views, or stored functions. Whenever the dump thread would normally go
      to sleep waiting for more data to arrive in the binlog, the dump thread will
      instead send another chunk of data in the binlog stream for the slave to apply.

      A "chunk of data" can be:

      • A CREATE OR REPLACE TABLE / VIEW / PROCEDURE / FUNCTION
      • A range of N rows (N=100, for example). Each successive chunk will do a
        range scan on the primary key from the end position of the last chunk.

      Sending data in small chunks avoids the need for long-lived table locks or
      transactions that could adversely affect master performance.

      The slave will connect in GTID mode. The master will send dumped chunks in a
      separate domain id, allowing the slave to process chunks in parallel with
      normal data.

      During the provisioning, all normal replication events from the master will
      arrive on the slave, and the slave will attempt to apply them locally. Some of
      these events will fail to apply, since the affected table or row may not yet
      have been loaded. In the provisioning mode, all such errors will be silently
      ignored. Proper locking (isolation mode, eg.) must be used on the master when
      fetching chunks, to ensure that updates for any row will always be applied
      correctly on the slave, either in a chunk, or in a later row event.

      In order to make the first version of this feature feasible to implement in a
      reasonable amount of time, it should set a number of reasonable restrictions
      (which could be relaxed in a later version of the feature):

      • Give up with an error if the slave is not configured for GTID mode
        (MASTER_USE_GTID != NO).
      • Give up with error if the slave receives any event in statement-based
        binlogging (so the master must be running in row-based replication mode,
        and no DDL must be done while the provisioning is running).
      • Give up with an error if the master has a table without primary key.
      • Secondary indexes will be enabled during the provisioning; this means that
        tables with large secondary indexes could be expensive to provision.

      Attachments

        Issue Links

          Activity

            People

              knielsen Kristian Nielsen
              knielsen Kristian Nielsen
              Votes:
              30 Vote for this issue
              Watchers:
              38 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.