Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
10.1.23, 10.1.34, 10.2(EOL)
-
None
Description
E.g. backup of initial database in 10.1.23 with
innodb-data-file-path=ibdata1:10M;ibdata2:5M:autoextend
is showing corruption :
170506 09:06:04 [01] Copying ./ibdata1 to /dev/shm/__var0/var1/backup/full/ibdata1
|
170506 09:06:05 [01] ...done
|
170506 09:06:05 [01] Copying ./ibdata2 to /dev/shm/__var0/var1/backup/full/ibdata2
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
Full output:
bash: ===> 10.1.23/bin/mariabackup --defaults-file=/dev/shm/__var0/var1/my.cnf --backup --target-dir=/dev/shm/__var0/var1/backup/
|
170506 09:13:48 Connecting to MySQL server host: localhost, user: root, password: not set, port: not set, socket: /dev/shm/__var0/tmp/mysql.sock.X8lno8
|
Using server version 10.1.23-MariaDB
|
10.1.23/bin/mariabackup based on MariaDB server 10.1.23-MariaDB Linux (x86_64)
|
xtrabackup: uses posix_fadvise().
|
xtrabackup: cd to /dev/shm/__var0/var1/data
|
xtrabackup: open files limit requested 0, set to 1024
|
xtrabackup: using the following InnoDB configuration:
|
xtrabackup: innodb_data_home_dir = ./
|
xtrabackup: innodb_data_file_path = ibdata1:10M;ibdata2:5M:autoextend
|
xtrabackup: innodb_log_group_home_dir = ./
|
xtrabackup: innodb_log_files_in_group = 2
|
xtrabackup: innodb_log_file_size = 50331648
|
170506 09:13:48 >> log scanned up to (1616829)
|
xtrabackup: Generating a list of tablespaces
|
170506 09:13:49 [01] Copying ./ibdata1 to /dev/shm/__var0/var1/backup/ibdata1
|
170506 09:13:49 [01] ...done
|
170506 09:13:49 [01] Copying ./ibdata2 to /dev/shm/__var0/var1/backup/ibdata2
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
170506 09:13:49 >> log scanned up to (1616829)
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Error: failed to read page after 10 retries. File ./ibdata2 seems to be corrupted.
|
[01] xtrabackup: Error: xtrabackup_copy_datafile() failed.
|
[01] xtrabackup: Error: failed to copy datafile.
|
2017-05-06 09:13:49: bash: ===> mariabackup failed with exit code 1
|
Attachments
Issue Links
- causes
-
-
- Closed
-
-
-
- Closed
-
- is blocked by
-
-
- Closed
-
- relates to
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
(2 relates to)
Activity
I attempted to repeat this bug in 10.1.34 but failed. I hope that the issue will be repeatable by slightly modifying my attempts below.
Note: MariaDB before 10.1.25 (MDEV-12113), 10.2.2 as well as MySQL before 5.7 used to write the FIL_PAGE_FILE_FLUSH_LSN field of the first page in each data file of the system tablespace, on shutdown. Later, the write to anything else but the first file was removed, because the same field was repurposed: in MySQL 5.7 and MariaDB 10.2.2 for the Split Sequence Number (SSN) of the SPATIAL INDEX R-tree pages, and in MariaDB 10.1 for the encryption key version and checksum. This field is not covered by any page checksum (until MDEV-12026 and MDEV-18644 implemented innodb_checksum_algorithm=full_crc32 in MariaDB 10.4), and the write of page 0 at shutdown is not even covered by the doublewrite buffer.
The first part of my patch is attempting to rewrite some dummy value to the FIL_PAGE_FILE_FLUSH_LSN field of the first page of the second data file. The test does not fail with mariadb-10.1.34, like I would have expected. The second part of the patch is writing that field in the clustered index root page of an .ibd file, and also that test fails to fail.
diff --git a/mysql-test/suite/mariabackup/absolute_ibdata_paths.test b/mysql-test/suite/mariabackup/absolute_ibdata_paths.test
|
index 6717f16d199..f73ae559584 100644
|
--- a/mysql-test/suite/mariabackup/absolute_ibdata_paths.test
|
+++ b/mysql-test/suite/mariabackup/absolute_ibdata_paths.test
|
@@ -2,6 +2,27 @@
|
# Innodb system tablespace is specified with absolute path in the .opt file
|
CREATE TABLE t(i INT) ENGINE INNODB;
|
INSERT INTO t VALUES(1);
|
+
|
+let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
|
+let MYSQLD_DATADIR= `SELECT @@datadir`;
|
+--source include/shutdown_mysqld.inc
|
+perl;
|
+my $file= "$ENV{MYSQLD_DATADIR}/ibdata_second";
|
+open(FILE, "+<", $file) or die "Unable to open $file\n";
|
+binmode FILE;
|
+my $ps= $ENV{INNODB_PAGE_SIZE};
|
+my $page;
|
+my $pos = $ps * 0;
|
+sysseek(FILE, $pos, 0) || die "Unable to seek $file\n";
|
+die "Unable to read $file" unless sysread(FILE, $page, $ps) == $ps;
|
+substr($page,26,8) = pack("NN", 1, 1);
|
+sysseek(FILE, $pos, 0) || die "Unable to rewind $file\n";
|
+syswrite(FILE, $page, $ps)==$ps || die "Unable to write $file\n";
|
+close(FILE) || die "Unable to close $file\n";
|
+EOF
|
+
|
+--source include/start_mysqld.inc
|
+
|
echo # xtrabackup backup;
|
|
let $targetdir=$MYSQLTEST_VARDIR/tmp/backup;
|
diff --git a/mysql-test/suite/mariabackup/mdev-12711.test b/mysql-test/suite/mariabackup/mdev-12711.test
|
new file mode 100644
|
index 00000000000..1da5cc57315
|
--- /dev/null
|
+++ b/mysql-test/suite/mariabackup/mdev-12711.test
|
@@ -0,0 +1,44 @@
|
+--source include/have_innodb.inc
|
+--source include/not_embedded.inc
|
+
|
+SET GLOBAL innodb_file_per_table=1;
|
+CREATE TABLE t1(a INT PRIMARY KEY) ENGINE=InnoDB;
|
+INSERT INTO t1 VALUES(1);
|
+
|
+let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
|
+let MYSQLD_DATADIR= `SELECT @@datadir`;
|
+--source include/shutdown_mysqld.inc
|
+perl;
|
+my $file= "$ENV{MYSQLD_DATADIR}/test/t1.ibd";
|
+open(FILE, "+<", $file) or die "Unable to open $file\n";
|
+binmode FILE;
|
+my $ps= $ENV{INNODB_PAGE_SIZE};
|
+my $page;
|
+sysseek(FILE, 3*$ps, 0) || die "Unable to seek $file\n";
|
+die "Unable to read $file" unless sysread(FILE, $page, $ps) == $ps;
|
+substr($page,26,8) = pack("NN", 1, 1);
|
+sysseek(FILE, 3*$ps, 0) || die "Unable to rewind $file\n";
|
+syswrite(FILE, $page, $ps)==$ps || die "Unable to write $file\n";
|
+close(FILE) || die "Unable to close $file\n";
|
+EOF
|
+
|
+--source include/start_mysqld.inc
|
+SELECT * FROM t1;
|
+
|
+let $targetdir=$MYSQLTEST_VARDIR/tmp/backup;
|
+
|
+--disable_result_log
|
+exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --backup --target-dir=$targetdir;
|
+--enable_result_log
|
+exec $XTRABACKUP --prepare --target-dir=$targetdir;
|
+
|
+--source include/shutdown_mysqld.inc
|
+rmdir $MYSQLD_DATADIR;
|
+exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --copy-back --datadir=$MYSQLD_DATADIR --target-dir=$targetdir;
|
+echo # restart server;
|
+--source include/start_mysqld.inc
|
+--enable_result_log
|
+
|
+SELECT * from t1;
|
+DROP TABLE t1;
|
+rmdir $targetdir; |
I think that we may have to analyze strace output to determine which exact page is being rejected, to rule out misleading diagnostic messages.
I can repeat the problem. I checked out and build mariadb-10.1.23 release build. I run mysql_install_db script to empty datadir with my.cnf containing only innodb-data-file-path=ibdata1:10M;ibdata2:5M:autoextend (so no encryption configured or enabled). I checkout out mariadb-10.1.34 and build it release build. I started mysqld on datadir and mariabackup and received similar error messages as described above. I emptied backup directory and checket out latest 10.1 and build again release build. I see still :
jan@jan-laptop-asus:~$ rm -rf xtrabackup_backupfiles
|
jan@jan-laptop-asus:~$ /usr/local/mysql/bin/mariabackup --defaults-file=/home/jan/my.cnf --datadir=/home/jan/newdb --backup
|
Info: Using unique option prefix 'backup' is error-prone and can break in the future. Please use the full name 'backup_encrypted' instead.
|
190326 11:53:51 Connecting to MySQL server host: localhost, user: not set, password: not set, port: not set, socket: not set
|
Using server version 10.1.39-MariaDB
|
/usr/local/mysql/bin/mariabackup based on MariaDB server 10.1.39-MariaDB Linux (x86_64)
|
mariabackup: uses posix_fadvise().
|
mariabackup: cd to /home/jan/newdb/
|
mariabackup: open files limit requested 0, set to 1024
|
mariabackup: using the following InnoDB configuration:
|
mariabackup: innodb_data_home_dir =
|
mariabackup: innodb_data_file_path = ibdata1:10M;ibdata2:5M:autoextend
|
mariabackup: innodb_log_group_home_dir = ./
|
mariabackup: innodb_log_files_in_group = 2
|
mariabackup: innodb_log_file_size = 50331648
|
190326 11:53:51 >> log scanned up to (1616839)
|
mariabackup: Generating a list of tablespaces
|
190326 11:53:51 [01] Copying ibdata1 to /home/jan/xtrabackup_backupfiles/ibdata1
|
190326 11:53:51 [01] ...done
|
190326 11:53:51 [01] Copying ibdata2 to /home/jan/xtrabackup_backupfiles/ibdata2
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
190326 11:53:52 >> log scanned up to (1616839)
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Error: failed to read page after 10 retries. File ibdata2 seems to be corrupted.
|
2019-03-26 11:53:52 7f87bd47f700 InnoDB: Page dump in ascii and hex (16384 bytes):
|
len 16384; hex 0000000000000000000000000000000000000000000000000000000000000018abbd000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000; asc ;
|
InnoDB: End of page dump
|
2019-03-26 11:53:52 7f87bd47f700 InnoDB: Compressed page type (0); stored checksum in field1 0; calculated checksums for field1: crc32 985610738, innodb 0, none 3735928559; page LSN 0; page number (if stored to page already) 0; space id (if stored to page already) 0
|
InnoDB: Page may be a freshly allocated page
|
[01] mariabackup: Error: xtrabackup_copy_datafile() failed.
|
[01] mariabackup: Error: failed to copy datafile.
|
Note that ibdata2 file can't be encrypted if there is no encryption metadata on ibdata1 file page 0.
I can repeat this by ensuring that the first page of the second data file will be all zero:
diff --git a/mysql-test/suite/mariabackup/absolute_ibdata_paths.opt b/mysql-test/suite/mariabackup/absolute_ibdata_paths.opt
|
index 52b6b743ac8..28848b9b086 100644
|
--- a/mysql-test/suite/mariabackup/absolute_ibdata_paths.opt
|
+++ b/mysql-test/suite/mariabackup/absolute_ibdata_paths.opt
|
@@ -1 +1 @@
|
---innodb --innodb-data-home-dir= --innodb-data-file-path=$MYSQLTEST_VARDIR/tmp/absolute_path_ibdata1:3M;ibdata_second:1M:autoextend
|
\ No newline at end of file
|
+--innodb --innodb-data-home-dir= --innodb-data-file-path=$MYSQLTEST_VARDIR/tmp/absolute_path_ibdata1:6M;ibdata_second:1M:autoextend
|
diff --git a/mysql-test/suite/mariabackup/absolute_ibdata_paths.test b/mysql-test/suite/mariabackup/absolute_ibdata_paths.test
|
index 6717f16d199..f73ae559584 100644
|
--- a/mysql-test/suite/mariabackup/absolute_ibdata_paths.test
|
+++ b/mysql-test/suite/mariabackup/absolute_ibdata_paths.test
|
@@ -2,6 +2,27 @@
|
# Innodb system tablespace is specified with absolute path in the .opt file
|
CREATE TABLE t(i INT) ENGINE INNODB;
|
INSERT INTO t VALUES(1);
|
+
|
+let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
|
+let MYSQLD_DATADIR= `SELECT @@datadir`;
|
+--source include/shutdown_mysqld.inc
|
+perl;
|
+my $file= "$ENV{MYSQLD_DATADIR}/ibdata_second";
|
+open(FILE, "+<", $file) or die "Unable to open $file\n";
|
+binmode FILE;
|
+my $ps= $ENV{INNODB_PAGE_SIZE};
|
+my $page;
|
+my $pos = $ps * 0;
|
+sysseek(FILE, $pos, 0) || die "Unable to seek $file\n";
|
+die "Unable to read $file" unless sysread(FILE, $page, $ps) == $ps;
|
+substr($page,26,8) = pack("NN", 1, 1);
|
+sysseek(FILE, $pos, 0) || die "Unable to rewind $file\n";
|
+syswrite(FILE, $page, $ps)==$ps || die "Unable to write $file\n";
|
+close(FILE) || die "Unable to close $file\n";
|
+EOF
|
+
|
+--source include/start_mysqld.inc
|
+
|
echo # xtrabackup backup;
|
|
let $targetdir=$MYSQLTEST_VARDIR/tmp/backup; |
A newer Mariabackup would actually dump the corrupted page:
|
10.1 ed643f4bb31d8e5a9c0bc161583b8c9b2a08c26b |
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Error: failed to read page after 10 retries. File ibdata_second seems to be corrupted.
|
2019-03-26 12:42:18 7ff37ad78700 InnoDB: Page dump in ascii and hex (16384 bytes):
|
len 16384; hex 0000000000000000000000000000000000000000000000000000000000010000000100000…
|
Was this bug not fixed in Mariabackup 10.1.38 and 10.2.22 by
MDEV-12112andMDEV-18129? (Note: the parameter that was introduced inMDEV-18129was later renamed to --encrypted-backup.)