Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
10.1.23, 10.1.34, 10.2(EOL)
-
None
Description
E.g. backup of initial database in 10.1.23 with
innodb-data-file-path=ibdata1:10M;ibdata2:5M:autoextend
is showing corruption :
170506 09:06:04 [01] Copying ./ibdata1 to /dev/shm/__var0/var1/backup/full/ibdata1
|
170506 09:06:05 [01] ...done
|
170506 09:06:05 [01] Copying ./ibdata2 to /dev/shm/__var0/var1/backup/full/ibdata2
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
Full output:
bash: ===> 10.1.23/bin/mariabackup --defaults-file=/dev/shm/__var0/var1/my.cnf --backup --target-dir=/dev/shm/__var0/var1/backup/
|
170506 09:13:48 Connecting to MySQL server host: localhost, user: root, password: not set, port: not set, socket: /dev/shm/__var0/tmp/mysql.sock.X8lno8
|
Using server version 10.1.23-MariaDB
|
10.1.23/bin/mariabackup based on MariaDB server 10.1.23-MariaDB Linux (x86_64)
|
xtrabackup: uses posix_fadvise().
|
xtrabackup: cd to /dev/shm/__var0/var1/data
|
xtrabackup: open files limit requested 0, set to 1024
|
xtrabackup: using the following InnoDB configuration:
|
xtrabackup: innodb_data_home_dir = ./
|
xtrabackup: innodb_data_file_path = ibdata1:10M;ibdata2:5M:autoextend
|
xtrabackup: innodb_log_group_home_dir = ./
|
xtrabackup: innodb_log_files_in_group = 2
|
xtrabackup: innodb_log_file_size = 50331648
|
170506 09:13:48 >> log scanned up to (1616829)
|
xtrabackup: Generating a list of tablespaces
|
170506 09:13:49 [01] Copying ./ibdata1 to /dev/shm/__var0/var1/backup/ibdata1
|
170506 09:13:49 [01] ...done
|
170506 09:13:49 [01] Copying ./ibdata2 to /dev/shm/__var0/var1/backup/ibdata2
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
170506 09:13:49 >> log scanned up to (1616829)
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Error: failed to read page after 10 retries. File ./ibdata2 seems to be corrupted.
|
[01] xtrabackup: Error: xtrabackup_copy_datafile() failed.
|
[01] xtrabackup: Error: failed to copy datafile.
|
2017-05-06 09:13:49: bash: ===> mariabackup failed with exit code 1
|
Attachments
Issue Links
- causes
-
-
- Closed
-
-
-
- Closed
-
- is blocked by
-
-
- Closed
-
- relates to
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
(2 relates to)
Activity
| Field | Original Value | New Value |
|---|---|---|
| Link |
This issue is blocked by |
| Attachment | __var0.zip [ 43620 ] |
| Fix Version/s | 10.1 [ 16100 ] |
| Assignee | Marko Mäkelä [ marko ] | Jan Lindström [ jplindst ] |
| Link |
This issue relates to |
| Link |
This issue relates to |
Just a note that this is still valid bug and with fix of MDEV-12709 it is easy to verify with mtr :
> ./mtr full_backup --mysqld="--innodb_data_file_path=ibdata1:10M;ibdata2:5M:autoextend"
|
Logging: /home/a/spd/_depot/m-branch/10.1/mysql-test/mysql-test-run.pl full_backup --mysqld=--innodb_data_file_path=ibdata1:10M;ibdata2:5M:autoextend
|
vardir: /home/a/spd/m0-10.1/build/mysql-test/var
|
Checking leftover processes...
|
Removing old var directory...
|
Creating var directory '/home/a/spd/m0-10.1/build/mysql-test/var'...
|
Checking supported features...
|
MariaDB Version 10.1.26-MariaDB
|
- SSL connections supported
|
Sphinx 'indexer' binary not found, sphinx suite will be skipped
|
Collecting tests...
|
Installing system database...
|
|
|
==============================================================================
|
|
|
TEST RESULT TIME (ms) or COMMENT
|
--------------------------------------------------------------------------
|
|
|
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
|
mariabackup.full_backup [ fail ]
|
Test ended at 2017-07-28 23:29:48
|
|
|
CURRENT_TEST: mariabackup.full_backup
|
mysqltest: At line 7: exec of '/home/a/spd/m0-10.1/build/extra/mariabackup/mariabackup --defaults-file=/home/a/spd/m0-10.1/build/mysql-test/var/my.cnf --backup --target-dir=/home/a/spd/m0-10.1/build/mysql-test/var/tmp/backup 2>&1' failed, error: 256, status: 1, errno: 0
|
Output from before failure:
|
170728 23:29:47 Connecting to MySQL server host: localhost, user: root, password: set, port: 16000, socket: /home/a/spd/m0-10.1/build/mysql-test/var/tmp/mysqld.1.sock
|
Using server version 10.1.26-MariaDB
|
/home/a/spd/m0-10.1/build/extra/mariabackup/mariabackup based on MariaDB server 10.1.26-MariaDB Linux (x86_64)
|
xtrabackup: uses posix_fadvise().
|
xtrabackup: cd to /home/a/spd/m0-10.1/build/mysql-test/var/mysqld.1/data/
|
xtrabackup: open files limit requested 1024, set to 1024
|
xtrabackup: using the following InnoDB configuration:
|
xtrabackup: innodb_data_home_dir = ./
|
xtrabackup: innodb_data_file_path = ibdata1:10M;ibdata2:5M:autoextend
|
xtrabackup: innodb_log_group_home_dir = ./
|
xtrabackup: innodb_log_files_in_group = 2
|
xtrabackup: innodb_log_file_size = 5242880
|
170728 23:29:47 >> log scanned up to (1632711)
|
xtrabackup: Generating a list of tablespaces
|
170728 23:29:47 [01] Copying ./ibdata1 to /home/a/spd/m0-10.1/build/mysql-test/var/tmp/backup/ibdata1
|
170728 23:29:47 [01] ...done
|
170728 23:29:47 [01] Copying ./ibdata2 to /home/a/spd/m0-10.1/build/mysql-test/var/tmp/backup/ibdata2
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
170728 23:29:48 >> log scanned up to (1632711)
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Database page corruption detected at page 0, retrying...
|
InnoDB: Checksum fields zero but page is not empty.
|
[01] xtrabackup: Error: failed to read page after 10 retries. File ./ibdata2 seems to be corrupted.
|
[01] xtrabackup: Error: xtrabackup_copy_datafile() failed.
|
[01] xtrabackup: Error: failed to copy datafile.
|
|
|
|
|
|
|
The result from queries just before the failure was:
|
CREATE TABLE t(i INT) ENGINE INNODB;
|
INSERT INTO t VALUES(1);
|
# xtrabackup backup
|
|
|
- saving '/home/a/spd/m0-10.1/build/mysql-test/var/log/mariabackup.full_backup/' to '/home/a/spd/m0-10.1/build/mysql-test/var/log/mariabackup.full_backup/'
|
--------------------------------------------------------------------------
|
The servers were restarted 0 times
|
Spent 0.000 of 6 seconds executing testcases
|
|
|
Failure: Failed 1/1 tests, 0.00% were successful.
|
|
|
Failing test(s): mariabackup.full_backup
|
|
|
The log files in var/log may give you some hint of what went wrong.
|
|
|
If you want to report this error, please read first the documentation
|
at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html
|
|
|
mysql-test-run: *** ERROR: there were failing test cases
|
10.2 passes for innodb_page_size=16k , but innodb_page_size=32k fails with this error message (maybe related to MDEV-11556 . at least message is the same ):
2017-07-28 23:43:47 139647445629120 [ERROR] InnoDB: Tablespace size stored in header is 2688 pages, but the sum of data file sizes is 480 pages
|
Let me know if this needs another bug report for such output in 10.2 or it will be fixed as part of this bug:
> ./mtr full_backup --mysqld="--innodb_data_file_path=ibdata1:10M;ibdata2:5M:autoextend" --force
|
Logging: /home/a/spd/_depot/m-branch/10.2/mysql-test/mysql-test-run.pl full_backup --mysqld=--innodb_data_file_path=ibdata1:10M;ibdata2:5M:autoextend --force
|
vardir: /home/a/spd/m1-10.2/build/mysql-test/var
|
Checking leftover processes...
|
Removing old var directory...
|
Creating var directory '/home/a/spd/m1-10.2/build/mysql-test/var'...
|
Checking supported features...
|
MariaDB Version 10.2.8-MariaDB
|
- SSL connections supported
|
Collecting tests...
|
Installing system database...
|
|
|
==============================================================================
|
|
|
TEST RESULT TIME (ms) or COMMENT
|
--------------------------------------------------------------------------
|
|
|
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
|
mariabackup.full_backup '32k,innodb' [ fail ]
|
Test ended at 2017-07-28 23:47:06
|
|
|
CURRENT_TEST: mariabackup.full_backup
|
mysqltest: At line 17: exec of '/home/a/spd/m1-10.2/build/extra/mariabackup/mariabackup --prepare --target-dir=/home/a/spd/m1-10.2/build/mysql-test/var/tmp/backup 2>&1' failed, error: 256, status: 1, errno: 11
|
Output from before failure:
|
/home/a/spd/m1-10.2/build/extra/mariabackup/mariabackup based on MariaDB server 10.2.8-MariaDB Linux (x86_64)
|
xtrabackup: cd to /home/a/spd/m1-10.2/build/mysql-test/var/tmp/backup/
|
xtrabackup: This target seems to be not prepared yet.
|
InnoDB: The universal page size of the database is set to 32768.
|
xtrabackup: using the following InnoDB configuration for recovery:
|
xtrabackup: innodb_data_home_dir = .
|
xtrabackup: innodb_data_file_path = ibdata1:10M;ibdata2:5M:autoextend
|
xtrabackup: innodb_log_group_home_dir = .
|
xtrabackup: Starting InnoDB instance for recovery.
|
xtrabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter)
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Uses event mutexes
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Compressed tables use zlib 1.2.7
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Number of pools: 1
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Using SSE2 crc32 instructions
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Initializing buffer pool, total size = 100M, instances = 1, chunk size = 100M
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Completed initialization of buffer pool
|
2017-07-28 23:47:05 139739991480064 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Need to create a new innodb_system data file 'ibdata2'.
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Setting file './ibdata2' size to 5 MB. Physically writing the file full; Please wait ...
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: File './ibdata2' size is now 5 MB.
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Highest supported file format is Barracuda.
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Starting crash recovery from checkpoint LSN=3116833
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Recovered page [page id: space=0, page number=320] from the doublewrite buffer.
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Recovered page [page id: space=0, page number=322] from the doublewrite buffer.
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Recovered page [page id: space=0, page number=321] from the doublewrite buffer.
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Recovered page [page id: space=0, page number=323] from the doublewrite buffer.
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Recovered page [page id: space=0, page number=324] from the doublewrite buffer.
|
2017-07-28 23:47:05 139740263680192 [Note] InnoDB: Starting final batch to recover 9 pages from redo log.
|
2017-07-28 23:47:05 139740263680192 [ERROR] InnoDB: Tablespace size stored in header is 2688 pages, but the sum of data file sizes is 480 pages
|
2017-07-28 23:47:05 139740263680192 [ERROR] InnoDB: Cannot start InnoDB. The tail of the system tablespace is missing. Have you edited innodb_data_file_path in my.cnf in an inappropriate way, removing data files from there? You can set innodb_force_recovery=1 in my.cnf to force a startup if you are trying to recover a badly corrupt database.
|
2017-07-28 23:47:05 139740263680192 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
|
xtrabackup: innodb_init() returned 11 (Generic error).
|
|
|
|
|
|
|
The result from queries just before the failure was:
|
CREATE TABLE t(i INT) ENGINE INNODB;
|
INSERT INTO t VALUES(1);
|
# xtrabackup backup
|
INSERT INTO t VALUES(2);
|
# xtrabackup prepare
|
|
|
- saving '/home/a/spd/m1-10.2/build/mysql-test/var/log/mariabackup.full_backup-32k,innodb/' to '/home/a/spd/m1-10.2/build/mysql-test/var/log/mariabackup.full_backup-32k,innodb/'
|
mariabackup.full_backup '64k,innodb' [ skipped ] Test requires InnoDB.
|
line
|
2017-07-28 23:47:07 139809037383872 [Warning] InnoDB: Are you sure you are using the right ib_logfiles to start up the database? Log sequence number in the ib_logfiles is 8716, less than the log sequence number in the first system tablespace file header, 130988.
|
2017-07-28 23:47:08 139809037383872 [ERROR] InnoDB: Cannot create doublewrite buffer: the first file in innodb_data_file_path must be at least 12M.
|
2017-07-28 23:47:08 139809037383872 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
|
2017-07-28 23:47:08 139809037383872 [ERROR] Plugin 'InnoDB' init function returned error.
|
2017-07-28 23:47:08 139809037383872 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
|
^ Found warnings in /home/a/spd/m1-10.2/build/mysql-test/var/log/mysqld.1.err
|
ok
|
|
|
mariabackup.full_backup '16k,innodb' [ pass ] 5727
|
mariabackup.full_backup '4k,innodb' [ pass ] 5767
|
mariabackup.full_backup '8k,innodb' [ pass ] 5751
|
--------------------------------------------------------------------------
|
The servers were restarted 4 times
|
Spent 17.245 of 44 seconds executing testcases
|
mysql-test-run: WARNING: Got errors/warnings while running tests, please examine '/home/a/spd/m1-10.2/build/mysql-test/var/log/warnings' for details.
|
|
|
Completed: Failed 1/4 tests, 75.00% were successful.
|
|
|
Failing test(s): mariabackup.full_backup
|
|
|
The log files in var/log may give you some hint of what went wrong.
|
|
|
If you want to report this error, please read first the documentation
|
at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html
|
|
|
1 tests were skipped, 1 by the test itself.
|
|
|
mysql-test-run: *** ERROR: there were failing test cases
|
| Affects Version/s | 10.2 [ 14601 ] |
| Fix Version/s | 10.2 [ 14601 ] |
This should be fixed by MDEV-12113: install_db shows corruption for rest encryption with innodb_data_file_path=ibdata1:3M;
| Fix Version/s | 10.1.25 [ 22542 ] | |
| Fix Version/s | 10.2 [ 14601 ] | |
| Fix Version/s | 10.1 [ 16100 ] | |
| Resolution | Fixed [ 1 ] | |
| Status | Open [ 1 ] | Closed [ 6 ] |
| Resolution | Fixed [ 1 ] | |
| Status | Closed [ 6 ] | Stalled [ 10000 ] |
| Fix Version/s | 10.1 [ 16100 ] | |
| Fix Version/s | 10.1.25 [ 22542 ] |
| Fix Version/s | 10.2 [ 14601 ] |
I've reopened MDEV-12113 as well. Let me know if this bug needs to be re-verified once MDEV-12113 is closed.
| Affects Version/s | 10.1.34 [ 23100 ] |
| Component/s | mariabackup [ 14500 ] | |
| Support case ID | 27137 | |
| Priority | Major [ 3 ] | Critical [ 2 ] |
| Link |
This issue relates to |
Was this bug not fixed in Mariabackup 10.1.38 and 10.2.22 by MDEV-12112 and MDEV-18129? (Note: the parameter that was introduced in MDEV-18129 was later renamed to --encrypted-backup.)
| Link |
This issue relates to |
I attempted to repeat this bug in 10.1.34 but failed. I hope that the issue will be repeatable by slightly modifying my attempts below.
Note: MariaDB before 10.1.25 (MDEV-12113), 10.2.2 as well as MySQL before 5.7 used to write the FIL_PAGE_FILE_FLUSH_LSN field of the first page in each data file of the system tablespace, on shutdown. Later, the write to anything else but the first file was removed, because the same field was repurposed: in MySQL 5.7 and MariaDB 10.2.2 for the Split Sequence Number (SSN) of the SPATIAL INDEX R-tree pages, and in MariaDB 10.1 for the encryption key version and checksum. This field is not covered by any page checksum (until MDEV-12026 and MDEV-18644 implemented innodb_checksum_algorithm=full_crc32 in MariaDB 10.4), and the write of page 0 at shutdown is not even covered by the doublewrite buffer.
The first part of my patch is attempting to rewrite some dummy value to the FIL_PAGE_FILE_FLUSH_LSN field of the first page of the second data file. The test does not fail with mariadb-10.1.34, like I would have expected. The second part of the patch is writing that field in the clustered index root page of an .ibd file, and also that test fails to fail.
diff --git a/mysql-test/suite/mariabackup/absolute_ibdata_paths.test b/mysql-test/suite/mariabackup/absolute_ibdata_paths.test
|
index 6717f16d199..f73ae559584 100644
|
--- a/mysql-test/suite/mariabackup/absolute_ibdata_paths.test
|
+++ b/mysql-test/suite/mariabackup/absolute_ibdata_paths.test
|
@@ -2,6 +2,27 @@
|
# Innodb system tablespace is specified with absolute path in the .opt file
|
CREATE TABLE t(i INT) ENGINE INNODB;
|
INSERT INTO t VALUES(1);
|
+
|
+let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
|
+let MYSQLD_DATADIR= `SELECT @@datadir`;
|
+--source include/shutdown_mysqld.inc
|
+perl;
|
+my $file= "$ENV{MYSQLD_DATADIR}/ibdata_second";
|
+open(FILE, "+<", $file) or die "Unable to open $file\n";
|
+binmode FILE;
|
+my $ps= $ENV{INNODB_PAGE_SIZE};
|
+my $page;
|
+my $pos = $ps * 0;
|
+sysseek(FILE, $pos, 0) || die "Unable to seek $file\n";
|
+die "Unable to read $file" unless sysread(FILE, $page, $ps) == $ps;
|
+substr($page,26,8) = pack("NN", 1, 1);
|
+sysseek(FILE, $pos, 0) || die "Unable to rewind $file\n";
|
+syswrite(FILE, $page, $ps)==$ps || die "Unable to write $file\n";
|
+close(FILE) || die "Unable to close $file\n";
|
+EOF
|
+
|
+--source include/start_mysqld.inc
|
+
|
echo # xtrabackup backup;
|
|
let $targetdir=$MYSQLTEST_VARDIR/tmp/backup;
|
diff --git a/mysql-test/suite/mariabackup/mdev-12711.test b/mysql-test/suite/mariabackup/mdev-12711.test
|
new file mode 100644
|
index 00000000000..1da5cc57315
|
--- /dev/null
|
+++ b/mysql-test/suite/mariabackup/mdev-12711.test
|
@@ -0,0 +1,44 @@
|
+--source include/have_innodb.inc
|
+--source include/not_embedded.inc
|
+
|
+SET GLOBAL innodb_file_per_table=1;
|
+CREATE TABLE t1(a INT PRIMARY KEY) ENGINE=InnoDB;
|
+INSERT INTO t1 VALUES(1);
|
+
|
+let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
|
+let MYSQLD_DATADIR= `SELECT @@datadir`;
|
+--source include/shutdown_mysqld.inc
|
+perl;
|
+my $file= "$ENV{MYSQLD_DATADIR}/test/t1.ibd";
|
+open(FILE, "+<", $file) or die "Unable to open $file\n";
|
+binmode FILE;
|
+my $ps= $ENV{INNODB_PAGE_SIZE};
|
+my $page;
|
+sysseek(FILE, 3*$ps, 0) || die "Unable to seek $file\n";
|
+die "Unable to read $file" unless sysread(FILE, $page, $ps) == $ps;
|
+substr($page,26,8) = pack("NN", 1, 1);
|
+sysseek(FILE, 3*$ps, 0) || die "Unable to rewind $file\n";
|
+syswrite(FILE, $page, $ps)==$ps || die "Unable to write $file\n";
|
+close(FILE) || die "Unable to close $file\n";
|
+EOF
|
+
|
+--source include/start_mysqld.inc
|
+SELECT * FROM t1;
|
+
|
+let $targetdir=$MYSQLTEST_VARDIR/tmp/backup;
|
+
|
+--disable_result_log
|
+exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --backup --target-dir=$targetdir;
|
+--enable_result_log
|
+exec $XTRABACKUP --prepare --target-dir=$targetdir;
|
+
|
+--source include/shutdown_mysqld.inc
|
+rmdir $MYSQLD_DATADIR;
|
+exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --copy-back --datadir=$MYSQLD_DATADIR --target-dir=$targetdir;
|
+echo # restart server;
|
+--source include/start_mysqld.inc
|
+--enable_result_log
|
+
|
+SELECT * from t1;
|
+DROP TABLE t1;
|
+rmdir $targetdir; |
I think that we may have to analyze strace output to determine which exact page is being rejected, to rule out misleading diagnostic messages.
I can repeat the problem. I checked out and build mariadb-10.1.23 release build. I run mysql_install_db script to empty datadir with my.cnf containing only innodb-data-file-path=ibdata1:10M;ibdata2:5M:autoextend (so no encryption configured or enabled). I checkout out mariadb-10.1.34 and build it release build. I started mysqld on datadir and mariabackup and received similar error messages as described above. I emptied backup directory and checket out latest 10.1 and build again release build. I see still :
jan@jan-laptop-asus:~$ rm -rf xtrabackup_backupfiles
|
jan@jan-laptop-asus:~$ /usr/local/mysql/bin/mariabackup --defaults-file=/home/jan/my.cnf --datadir=/home/jan/newdb --backup
|
Info: Using unique option prefix 'backup' is error-prone and can break in the future. Please use the full name 'backup_encrypted' instead.
|
190326 11:53:51 Connecting to MySQL server host: localhost, user: not set, password: not set, port: not set, socket: not set
|
Using server version 10.1.39-MariaDB
|
/usr/local/mysql/bin/mariabackup based on MariaDB server 10.1.39-MariaDB Linux (x86_64)
|
mariabackup: uses posix_fadvise().
|
mariabackup: cd to /home/jan/newdb/
|
mariabackup: open files limit requested 0, set to 1024
|
mariabackup: using the following InnoDB configuration:
|
mariabackup: innodb_data_home_dir =
|
mariabackup: innodb_data_file_path = ibdata1:10M;ibdata2:5M:autoextend
|
mariabackup: innodb_log_group_home_dir = ./
|
mariabackup: innodb_log_files_in_group = 2
|
mariabackup: innodb_log_file_size = 50331648
|
190326 11:53:51 >> log scanned up to (1616839)
|
mariabackup: Generating a list of tablespaces
|
190326 11:53:51 [01] Copying ibdata1 to /home/jan/xtrabackup_backupfiles/ibdata1
|
190326 11:53:51 [01] ...done
|
190326 11:53:51 [01] Copying ibdata2 to /home/jan/xtrabackup_backupfiles/ibdata2
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
190326 11:53:52 >> log scanned up to (1616839)
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Error: failed to read page after 10 retries. File ibdata2 seems to be corrupted.
|
2019-03-26 11:53:52 7f87bd47f700 InnoDB: Page dump in ascii and hex (16384 bytes):
|
len 16384; hex 0000000000000000000000000000000000000000000000000000000000000018abbd000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000; asc ;
|
InnoDB: End of page dump
|
2019-03-26 11:53:52 7f87bd47f700 InnoDB: Compressed page type (0); stored checksum in field1 0; calculated checksums for field1: crc32 985610738, innodb 0, none 3735928559; page LSN 0; page number (if stored to page already) 0; space id (if stored to page already) 0
|
InnoDB: Page may be a freshly allocated page
|
[01] mariabackup: Error: xtrabackup_copy_datafile() failed.
|
[01] mariabackup: Error: failed to copy datafile.
|
Note that ibdata2 file can't be encrypted if there is no encryption metadata on ibdata1 file page 0.
| Attachment | dbfiles.tgz [ 47877 ] |
| Assignee | Jan Lindström [ jplindst ] | Vladislav Vaintroub [ wlad ] |
I can repeat this by ensuring that the first page of the second data file will be all zero:
diff --git a/mysql-test/suite/mariabackup/absolute_ibdata_paths.opt b/mysql-test/suite/mariabackup/absolute_ibdata_paths.opt
|
index 52b6b743ac8..28848b9b086 100644
|
--- a/mysql-test/suite/mariabackup/absolute_ibdata_paths.opt
|
+++ b/mysql-test/suite/mariabackup/absolute_ibdata_paths.opt
|
@@ -1 +1 @@
|
---innodb --innodb-data-home-dir= --innodb-data-file-path=$MYSQLTEST_VARDIR/tmp/absolute_path_ibdata1:3M;ibdata_second:1M:autoextend
|
\ No newline at end of file
|
+--innodb --innodb-data-home-dir= --innodb-data-file-path=$MYSQLTEST_VARDIR/tmp/absolute_path_ibdata1:6M;ibdata_second:1M:autoextend
|
diff --git a/mysql-test/suite/mariabackup/absolute_ibdata_paths.test b/mysql-test/suite/mariabackup/absolute_ibdata_paths.test
|
index 6717f16d199..f73ae559584 100644
|
--- a/mysql-test/suite/mariabackup/absolute_ibdata_paths.test
|
+++ b/mysql-test/suite/mariabackup/absolute_ibdata_paths.test
|
@@ -2,6 +2,27 @@
|
# Innodb system tablespace is specified with absolute path in the .opt file
|
CREATE TABLE t(i INT) ENGINE INNODB;
|
INSERT INTO t VALUES(1);
|
+
|
+let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
|
+let MYSQLD_DATADIR= `SELECT @@datadir`;
|
+--source include/shutdown_mysqld.inc
|
+perl;
|
+my $file= "$ENV{MYSQLD_DATADIR}/ibdata_second";
|
+open(FILE, "+<", $file) or die "Unable to open $file\n";
|
+binmode FILE;
|
+my $ps= $ENV{INNODB_PAGE_SIZE};
|
+my $page;
|
+my $pos = $ps * 0;
|
+sysseek(FILE, $pos, 0) || die "Unable to seek $file\n";
|
+die "Unable to read $file" unless sysread(FILE, $page, $ps) == $ps;
|
+substr($page,26,8) = pack("NN", 1, 1);
|
+sysseek(FILE, $pos, 0) || die "Unable to rewind $file\n";
|
+syswrite(FILE, $page, $ps)==$ps || die "Unable to write $file\n";
|
+close(FILE) || die "Unable to close $file\n";
|
+EOF
|
+
|
+--source include/start_mysqld.inc
|
+
|
echo # xtrabackup backup;
|
|
let $targetdir=$MYSQLTEST_VARDIR/tmp/backup; |
A newer Mariabackup would actually dump the corrupted page:
|
10.1 ed643f4bb31d8e5a9c0bc161583b8c9b2a08c26b |
[01] mariabackup: Database page corruption detected at page 0, retrying...
|
[01] mariabackup: Error: failed to read page after 10 retries. File ibdata_second seems to be corrupted.
|
2019-03-26 12:42:18 7ff37ad78700 InnoDB: Page dump in ascii and hex (16384 bytes):
|
len 16384; hex 0000000000000000000000000000000000000000000000000000000000010000000100000…
|
| Assignee | Vladislav Vaintroub [ wlad ] | Marko Mäkelä [ marko ] |
| Status | Stalled [ 10000 ] | In Progress [ 3 ] |
| issue.field.resolutiondate | 2019-03-26 12:04:15.0 | 2019-03-26 12:04:15.539 |
| Fix Version/s | 10.1.39 [ 23305 ] | |
| Fix Version/s | 10.2.24 [ 23308 ] | |
| Fix Version/s | 10.3.14 [ 23216 ] | |
| Fix Version/s | 10.4.4 [ 23310 ] | |
| Fix Version/s | 10.2 [ 14601 ] | |
| Fix Version/s | 10.1 [ 16100 ] | |
| Resolution | Fixed [ 1 ] | |
| Status | In Progress [ 3 ] | Closed [ 6 ] |
| Link |
This issue causes |
| Link |
This issue causes |
| Workflow | MariaDB v3 [ 80632 ] | MariaDB v4 [ 152104 ] |
| Zendesk Related Tickets | 173648 |
I think that this is a design problem of the MariaDB encryption.
MariaDB encryption is reusing the field FIL_PAGE_FILE_FLUSH_LSN which used to be 0 in all pages except the first page of each data file of the InnoDB system tablespace.
In MySQL 5.7, the field was repurposed for FIL_RTREE_SPLIT_SEQ_NUM. This is the reason why MariaDB 10.2 will not support encryption in SPATIAL INDEX (
MDEV-11974).The problem here appears to be that xtrabackup is incorrectly mistaking the first page of the second data file of the system tablespace for an encrypted one, because the FIL_PAGE_FILE_FLUSH_LSN field is not 0.
MDEV-11939was filed for a similar issue in the innochecksum utility.In xtrabackup, we should add a check that FIL_PAGE_FILE_FLUSH_LSN can be nonzero in an unencrypted system tablespace. Also, MariaDB 10.1 should probably stop writing FIL_PAGE_FILE_FLUSH_LSN at shutdown to the first page of other than the first system tablespace file (10.2 already stopped this because of the SPATIAL INDEX mentioned above), so that it will not make an encrypted page unaccessible by corrupting the encryption key ID.