Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
10.2(EOL), 10.3(EOL), 10.4(EOL), 10.5, 10.6, 10.7(EOL), 10.8(EOL), 10.9(EOL)
-
None
Description
If a new schema with at least one table is created after a full backup is taken, but before the incremental is taken, the backup cannot be restored because incremental prepare fails.
mariabackup prepare fails with : Error on rename ..... (errno: 20 "Not a directory" )
The issue seems to be that mariabackup prepare expects the folder for the new schema to be already there, but it isn't because the schema is created after the full is taken, so can't be found in the full backup.
This is critical as it breaks backup restores in presence of incrementals when the above condition is met (which is all but rare).
How to repeat:
1. take a full backup
2. create a new schema and table
3. take an incremental backup
4. restore the full backup and then the incremental backup; prepare will crash
Attachments
Issue Links
- relates to
-
MDEV-23335 MariaBackup Incremental Does Not Reflect Dropped/Created Databases
- Closed
Activity
This MTR test demonstrates the problem:
--source include/have_innodb.inc
|
|
call mtr.add_suppression("InnoDB: New log files created"); |
|
--let basedir=$MYSQLTEST_VARDIR/tmp/backup
|
--let incremental_dir=$MYSQLTEST_VARDIR/tmp/backup_inc1
|
|
CREATE TABLE t1 (i INT) ENGINE=InnoDB; |
INSERT INTO t1 VALUES(0); |
|
--disable_result_log
|
--exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --backup --parallel=10 --target-dir=$basedir
|
--enable_result_log
|
|
CREATE DATABASE test1; |
CREATE TABLE test1.t1 (a INT) ENGINE=InnoDB; |
INSERT INTO test1.t1 VALUES (1000); |
|
--disable_result_log
|
--exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --backup --parallel=2 --ftwrl-wait-timeout=5 --ftwrl-wait-threshold=300 --ftwrl-wai
|
t-query-type=all --target-dir=$incremental_dir --incremental-basedir=$basedir |
--exec $XTRABACKUP --prepare --target-dir=$basedir
|
--exec $XTRABACKUP --prepare --target-dir=$basedir --incremental-dir=$incremental_dir
|
--enable_result_log
|
|
echo # Restore and check results; |
--let $targetdir=$basedir
|
--source include/restart_and_restore.inc
|
--enable_result_log
|
|
SELECT * FROM t1; |
SELECT * FROM test1.t1; |
|
DROP TABLE t1; |
DROP DATABASE test1; |
--rmdir $basedir
|
--rmdir $incremental_dir |
The output is:
==============================================================================
|
|
TEST RESULT TIME (ms) or COMMENT
|
--------------------------------------------------------------------------
|
|
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
|
call mtr.add_suppression("InnoDB: New log files created");
|
CREATE TABLE t1 (i INT) ENGINE=InnoDB;
|
INSERT INTO t1 VALUES(0);
|
CREATE DATABASE test1;
|
CREATE TABLE test1.t1 (a INT) ENGINE=InnoDB;
|
INSERT INTO test1.t1 VALUES (1000);
|
mariabackup.incremental_backup_newdb_before_inc 'innodb' [ fail ]
|
Test ended at 2022-04-29 17:22:01
|
|
CURRENT_TEST: mariabackup.incremental_backup_newdb_before_inc
|
mysqltest: At line 22: exec of '/home/bar/maria-git/server.10.6/BUILD-DEB-ASAN/extra/mariabackup/mariabackup --prepare --target-dir=/home/bar/maria-git/server.10.6/BUILD-DEB-ASAN/mysql-test/var/tmp/backup --incremental-dir=/home/bar/maria-git/server.10.6/BUILD-DEB-ASAN/mysql-test/var/tmp/backup_inc1 2>&1' failed, error: 256, status: 1, errno: 11
|
Output from before failure:
|
/home/bar/maria-git/server.10.6/BUILD-DEB-ASAN/extra/mariabackup/mariabackup based on MariaDB server 10.6.8-MariaDB Linux (x86_64)
|
[00] 2022-04-29 17:22:00 incremental backup from 44203 is enabled.
|
[00] 2022-04-29 17:22:00 cd to /home/bar/maria-git/server.10.6/BUILD-DEB-ASAN/mysql-test/var/tmp/backup/
|
[00] 2022-04-29 17:22:00 open files limit requested 0, set to 1024
|
[00] 2022-04-29 17:22:00 Removing ./test/t1.ibd
|
[00] 2022-04-29 17:22:00 Renaming /home/bar/maria-git/server.10.6/BUILD-DEB-ASAN/mysql-test/var/tmp/backup_inc1//test/t1.new to ./test/t1.ibd
|
|
[00] 2022-04-29 17:22:00 Renaming /home/bar/maria-git/server.10.6/BUILD-DEB-ASAN/mysql-test/var/tmp/backup_inc1//test1/t1.new to ./test1/t1.ibd
|
|
/home/bar/maria-git/server.10.6/BUILD-DEB-ASAN/extra/mariabackup/mariabackup: Error on rename of '/home/bar/maria-git/server.10.6/BUILD-DEB-ASAN/mysql-test/var/tmp/backup_inc1//test1/t1.new' to './test1/t1.ibd' (errno: 20 "Not a directory")
|
[00] FATAL ERROR: 2022-04-29 17:22:00 Can't rename /home/bar/maria-git/server.10.6/BUILD-DEB-ASAN/mysql-test/var/tmp/backup_inc1//test1/t1.new to ./test1/t1.ibd errno 2
|
|
- saving '/home/bar/maria-git/server.10.6/BUILD-DEB-ASAN/mysql-test/var/log/mariabackup.incremental_backup_newdb_before_inc-innodb/' to '/home/bar/maria-git/server.10.6/BUILD-DEB-ASAN/mysql-test/var/log/mariabackup.incremental_backup_newdb_before_inc-innodb/'
|
--------------------------------------------------------------------------
|
The servers were restarted 0 times
|
Spent 0.000 of 8 seconds executing testcases
|
|
Failure: Failed 1/1 tests, 0.00% were successful.
|
|
Failing test(s): mariabackup.incremental_backup_newdb_before_inc
|
serg, please review a patch:
https://github.com/MariaDB/server/commit/fda1ccea3e1710ff35549431429872676699e578
Thanks.
Repeatable with 10.2 with a different test:
--source include/have_innodb.inc
|
--source include/have_debug.inc
|
|
call mtr.add_suppression("InnoDB: New log files created"); |
|
--echo #
|
--echo # MDEV-28446 mariabackup prepare fails for incrementals if a new schema is created after full backup is taken
|
--echo #
|
|
--let $basedir=$MYSQLTEST_VARDIR/tmp/backup
|
--let $incremental_dir=$MYSQLTEST_VARDIR/tmp/backup_inc1
|
|
CREATE TABLE t1(i INT PRIMARY KEY) ENGINE INNODB; |
|
--disable_result_log
|
--exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --backup --target-dir=$basedir
|
--enable_result_log
|
|
--let after_load_tablespaces=BEGIN NOT ATOMIC CREATE DATABASE test1; CREATE TABLE test1.t1 ENGINE=INNODB SELECT UUID() from test.seq_1_to_10000; END
|
|
--exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --backup --target-dir=$incremental_dir --incremental-basedir=$basedir --dbug=+d,mariabackup_events,mari
|
--let after_load_tablespaces=
|
--disable_result_log
|
--echo # Prepare full backup, apply incremental one
|
--exec $XTRABACKUP --apply-log-only --prepare --target-dir=$basedir
|
--exec $XTRABACKUP --prepare --target-dir=$basedir --incremental-dir=$incremental_dir
|
--enable_result_log
|
|
--let $targetdir=$basedir
|
--source include/restart_and_restore.inc
|
--enable_result_log
|
|
SELECT COUNT(*) FROM test.t1; |
SELECT COUNT(*) FROM test1.t1; |
|
DROP TABLE t1; |
DROP DATABASE test1; |
--rmdir $basedir
|
--rmdir $incremental_dir |
....
|
[00] 2022-04-30 08:35:32 Redo log (from LSN 1631081 to 2536974) was copied.
|
[00] 2022-04-30 08:35:32 completed OK!
|
mysqltest: At line 26: exec of '/home/bar/maria-git/server.10.2/BUILD-DEB/extra/mariabackup/mariabackup --prepare --target-dir=/home/bar/maria-git/server.10.2/BUILD-DEB/mysql-test/var/tmp/backup --incremental-dir=/home/bar/maria-git/server.10.2/BUILD-DEB/mysql-test/var/tmp/backup_inc1 2>&1' failed, error: 256, status: 1, errno: 11
|
Output from before failure:
|
/home/bar/maria-git/server.10.2/BUILD-DEB/extra/mariabackup/mariabackup based on MariaDB server 10.2.44-MariaDB Linux (x86_64)
|
[00] 2022-04-30 08:35:33 incremental backup from 1631081 is enabled.
|
[00] 2022-04-30 08:35:33 cd to /home/bar/maria-git/server.10.2/BUILD-DEB/mysql-test/var/tmp/backup/
|
[00] 2022-04-30 08:35:33 open files limit requested 0, set to 1024
|
[00] 2022-04-30 08:35:33 Renaming /home/bar/maria-git/server.10.2/BUILD-DEB/mysql-test/var/tmp/backup_inc1//test1/t1.new to ./test1/t1.ibd
|
|
/home/bar/maria-git/server.10.2/BUILD-DEB/extra/mariabackup/mariabackup: Error on rename of '/home/bar/maria-git/server.10.2/BUILD-DEB/mysql-test/var/tmp/backup_inc1//test1/t1.new' to './test1/t1.ibd' (errno: 20 "Not a directory")
|
[00] FATAL ERROR: 2022-04-30 08:35:33 Can't rename /home/bar/maria-git/server.10.2/BUILD-DEB/mysql-test/var/tmp/backup_inc1//test1/t1.new to ./test1/t1.ibd errno 2
|
|
- saving '/home/bar/maria-git/server.10.2/BUILD-DEB/mysql-test/var/log/mariabackup.AAA-innodb/' to '/home/bar/maria-git/server.10.2/BUILD-DEB/mysql-test/var/log/mariabackup.AAA-innodb/'
|
--------------------------------------------------------------------------
|
The servers were restarted 0 times
|
Spent 0.000 of 9 seconds executing testcases
|
|
Failure: Failed 1/1 tests, 0.00% were successful.
|
serg, this is a patch for 10.2:
https://github.com/MariaDB/server/commit/5d1ae4f4c71743ce60e7ab218b33deecec4c4050
The problem is not repeatable with versions between 10.2 and 10.5.
Stably repeatable with 10.6 and 10.9.
(Did not check 10.7 and 10.8).