Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
10.4.3, 10.1.38, 10.2.22, 10.3.13
-
None
Description
Mariabackup inherited a xtrabackup_binlog_pos_innodb from Percona XtraBackup. However, this file is apparently only safe to use with Percona Server versions that have a special lockless binary log feature:
Starting with Percona Server for MySQL 5.6.26-74.0 LOCK TABLES FOR BACKUP flushes the current binary log coordinates to InnoDB. Thus, under active LOCK TABLES FOR BACKUP, the binary log coordinates in InnoDB are consistent with its redo log and any non-transactional updates (as the latter are blocked by LOCK TABLES FOR BACKUP). It is planned that this change will enable Percona XtraBackup to avoid issuing the more invasive LOCK BINLOG FOR BACKUP command under some circumstances.
When the Lockless binary log information feature is available [1] on the server, Percona XtraBackup can trust binary log information stored in the InnoDB system header and avoid executing LOCK BINLOG FOR BACKUP (and thus, blocking commits for the duration of finalizing the REDO log copy) under a number of circumstances:
- when the server is not a GTID-enabled Galera cluster node
- when the replication I/O thread information should not be stored as a part of the backup (i.e. when the xtrabackup --slave-info option is not specified)
If all of the above conditions hold, Percona XtraBackup does not execute the SHOW MASTER STATUS as a part of the backup procedure, does not create the xtrabackup_binlog_info file on backup. Instead, that information is retrieved and the file is created after preparing the backup, along with creating xtrabackup_binlog_pos_innodb, which in this case contains exactly the same information as in xtrabackup_binlog_info and is thus redundant.
https://www.percona.com/doc/percona-xtrabackup/2.4/advanced/lockless_bin-log.html
Maybe Mariabackup shouldn't create it at all? It can lead users to make dangerous assumptions. For example, some users seem to think they can use this position when they run Mariabackup with --no-lock.
For example, if we take a backup with --no-lock:
$ sudo mariabackup --backup --no-lock \
|
> --target-dir=/home/ec2-user/backup/ \
|
> --user=mariabackup --password=mypassword
|
[00] 2019-03-13 18:43:56 Connecting to MySQL server host: localhost, user: mariabackup, password: set, port: not set, socket: not set
|
[00] 2019-03-13 18:43:56 Using server version 10.3.13-MariaDB-log
|
[00] 2019-03-13 18:43:56 mariabackup based on MariaDB server 10.3.13-MariaDB Linux (x86_64)
|
[00] 2019-03-13 18:43:56 uses posix_fadvise().
|
[00] 2019-03-13 18:43:56 cd to /var/lib/mysql/
|
...
|
[00] 2019-03-13 18:43:58 completed OK!
|
And then prepare it:
$ sudo mariabackup --prepare \
|
> --target-dir=/home/ec2-user/backup/
|
[00] 2019-03-13 18:44:38 mariabackup based on MariaDB server 10.3.13-MariaDB Linux (x86_64)
|
[00] 2019-03-13 18:44:38 cd to /home/ec2-user/backup/
|
...
|
[00] 2019-03-13 18:44:39 Last binlog file ./mariadb-bin.000007, position 632
|
[00] 2019-03-13 18:44:39 completed OK!
|
Then this file is present:
$ ls -l /home/ec2-user/backup/xtrabackup_binlog*
|
-rw-r----- 1 root root 38 Mar 13 18:43 /home/ec2-user/backup/xtrabackup_binlog_info
|
-rw-r--r-- 1 root root 25 Mar 13 18:44 /home/ec2-user/backup/xtrabackup_binlog_pos_innodb
|
Attachments
Issue Links
- relates to
-
MDEV-5336 Implement BACKUP STAGE for safe external backups
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Stalled
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
MDEV-20552 Remove innobackupex mode from Mariabackup
-
- Stalled
-
-
-
- Closed
-
-
-
- Closed
-
Activity
Hi wlad,
OK, thanks. I thought I'd ask because I noticed that MDEV-5336 duplicated MDEV-8436, which is about porting Percona's "LOCK TABLES FOR BACKUP". That statement seems to be how Percona implemented "lockless binlog" in Percona Server and XtraBackup.
MDEV-5336, if you read its current synopsis, is very very different from porting LOCK TABLES FOR BACKUP.
Thanks, wlad. I do see in the issue description of MDEV-5336 that 10.4's "LOCK FOR BACKUP" does explicitly lock the binlog during "BACKUP STAGE BLOCK_COMMIT":
BACKUP STAGE BLOCK_COMMIT
- Lock the binary log and commit/rollback to ensure that no changes are
committed to any tables. If there are active committ's or data to be copied to
the binary log this will be allowed to finish before the lock is granted.
You're right that the actual implementation looks much different than Percona's "LOCK TABLES FOR BACKUP".
The "BLOCK_COMMIT" stage is, with relatively small differences, the same as FTWRL, not really "lockless".
Removing this functionality is a misunderstanding, and seems a serious regression in a GA release.
It is correct that obtaining the binlog position lockless in this way has some limitations. But these limitations have always been there, and while it requires some care, it does not make the given position useless. In contrast, it is the only way to get a consistent position in a fully lockless way.
The limitation is that the binlog position is the position of the last DML against an InnoDB table that is included in the backup. This means that the user must avoid doing DDL or updates to the mysql.* system tables (eg. GRANT) during the backup. If that precation is taken, the position is correct.
So this fix should be reverted to restore the functionality and fix the regression in a GA release. And the documentation could be updated to explain what the limitation is.
MDEV-5336introduced "LOCK FOR BACKUP" in MariaDB 10.4.2, so I'm not sure if that implementation changes anything about the usefulness of xtrabackup_binlog_pos_innodb.