Details
-
New Feature
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
Description
Idea
The purpose of this task is to create an easy-to-use facility for setting up a
new MariaDB replication slave.
Setting up a new slave currently involves: 1) installing MariaDB with initial
database; 2) point the slave to the master with CHANGE MASTER TO; 3) copying
initial data from the master to the slave; and 4) starting the slave with
START SLAVE. The idea is to automate step (3), which currently needs to be
done manually.
The syntax could be something as simple as
LOAD DATA FROM MASTER
This would then connect to the master that is currently configured. It will
load a snapshot of all the data on the master, and leave the slave position at
the point of the snapshot, ready for START SLAVE to continue replication from
that point.
Implementation:
The idea is to do this non-blocking on the master, in a way that works for any
storage engine. It will rely on row-based replication to be used between the
master and the slave.
At the start of LOAD DATA FROM MASTER, the slave will enter a special
provisioning mode. It will start replicating events from the master at the
master's current position.
The master dump thread will send binlog events to the slave as normal. But in
addition, it will interleave a dump of all the data on the master contained in
tables, views, or stored functions. Whenever the dump thread would normally go
to sleep waiting for more data to arrive in the binlog, the dump thread will
instead send another chunk of data in the binlog stream for the slave to apply.
A "chunk of data" can be:
- A CREATE OR REPLACE TABLE / VIEW / PROCEDURE / FUNCTION
- A range of N rows (N=100, for example). Each successive chunk will do a
range scan on the primary key from the end position of the last chunk.
Sending data in small chunks avoids the need for long-lived table locks or
transactions that could adversely affect master performance.
The slave will connect in GTID mode. The master will send dumped chunks in a
separate domain id, allowing the slave to process chunks in parallel with
normal data.
During the provisioning, all normal replication events from the master will
arrive on the slave, and the slave will attempt to apply them locally. Some of
these events will fail to apply, since the affected table or row may not yet
have been loaded. In the provisioning mode, all such errors will be silently
ignored. Proper locking (isolation mode, eg.) must be used on the master when
fetching chunks, to ensure that updates for any row will always be applied
correctly on the slave, either in a chunk, or in a later row event.
In order to make the first version of this feature feasible to implement in a
reasonable amount of time, it should set a number of reasonable restrictions
(which could be relaxed in a later version of the feature):
- Give up with an error if the slave is not configured for GTID mode
(MASTER_USE_GTID != NO).
- Give up with error if the slave receives any event in statement-based
binlogging (so the master must be running in row-based replication mode,
and no DDL must be done while the provisioning is running).
- Give up with an error if the master has a table without primary key.
- Secondary indexes will be enabled during the provisioning; this means that
tables with large secondary indexes could be expensive to provision.
Attachments
Issue Links
- blocks
-
MDEV-20106 Restart slave with memory table break replication in strict-mode
- Open
- relates to
-
MDEV-8925 Extending protocol to support binary provisioning
- Open
-
MDEV-11675 Lag Free Alter On Slave
- Closed
-
MDEV-14992 BACKUP: in-server backup
- Open
-
MDEV-21106 Port clone plugin from MySQL
- Open
-
MXS-2542 Add rebuild server to MariaDB Monitor
- Closed
-
MDEV-15610 Add SST for asynchronous slaves
- Closed
- links to