Idea

The purpose of this task is to create an easy-to-use facility for setting up a
new MariaDB replication slave.

Setting up a new slave currently involves: 1) installing MariaDB with initial
database; 2) point the slave to the master with CHANGE MASTER TO; 3) copying
initial data from the master to the slave; and 4) starting the slave with
START SLAVE. The idea is to automate step (3), which currently needs to be
done manually.

The syntax could be something as simple as

LOAD DATA FROM MASTER

This would then connect to the master that is currently configured. It will
load a snapshot of all the data on the master, and leave the slave position at
the point of the snapshot, ready for START SLAVE to continue replication from
that point.

Implementation:

The idea is to do this non-blocking on the master, in a way that works for any
storage engine. It will rely on row-based replication to be used between the
master and the slave.

At the start of LOAD DATA FROM MASTER, the slave will enter a special
provisioning mode. It will start replicating events from the master at the
master's current position.

The master dump thread will send binlog events to the slave as normal. But in
addition, it will interleave a dump of all the data on the master contained in
tables, views, or stored functions. Whenever the dump thread would normally go
to sleep waiting for more data to arrive in the binlog, the dump thread will
instead send another chunk of data in the binlog stream for the slave to apply.

A "chunk of data" can be:

• A CREATE OR REPLACE TABLE / VIEW / PROCEDURE / FUNCTION

• A range of N rows (N=100, for example). Each successive chunk will do a
  range scan on the primary key from the end position of the last chunk.

Sending data in small chunks avoids the need for long-lived table locks or
transactions that could adversely affect master performance.

The slave will connect in GTID mode. The master will send dumped chunks in a
separate domain id, allowing the slave to process chunks in parallel with
normal data.

During the provisioning, all normal replication events from the master will
arrive on the slave, and the slave will attempt to apply them locally. Some of
these events will fail to apply, since the affected table or row may not yet
have been loaded. In the provisioning mode, all such errors will be silently
ignored. Proper locking (isolation mode, eg.) must be used on the master when
fetching chunks, to ensure that updates for any row will always be applied
correctly on the slave, either in a chunk, or in a later row event.

In order to make the first version of this feature feasible to implement in a
reasonable amount of time, it should set a number of reasonable restrictions
(which could be relaxed in a later version of the feature):

• Give up with an error if the slave is not configured for GTID mode
  (MASTER_USE_GTID != NO).

• Give up with error if the slave receives any event in statement-based
  binlogging (so the master must be running in row-based replication mode,
  and no DDL must be done while the provisioning is running).

• Give up with an error if the master has a table without primary key.

• Secondary indexes will be enabled during the provisioning; this means that
  tables with large secondary indexes could be expensive to provision.