[MDEV-12966] mariabackup fails with non-default innodb_page_size Created: 2017-06-01  Updated: 2021-10-26  Resolved: 2021-10-26

Status: Closed
Project: MariaDB Server
Component/s: Backup
Affects Version/s: 10.1.24
Fix Version/s: 10.2.7

Type: Bug Priority: Major
Reporter: Andrii Nikitin (Inactive) Assignee: Jan Lindström (Inactive)
Resolution: Won't Fix Votes: 1
Labels: None

Issue Links:
Blocks
is blocked by MDEV-12709 mariabackup mtr tests don't handle pr... Closed
Relates
relates to MDEV-12975 InnoDB redo log minimum size check us... Closed

 Description   

Currently mariabackup doesn't work properly with custom innodb_page_size

$ for k in 4k 8k 32k 64k; do ./mtr --mysqld=--innodb-buffer-pool-size=28M --mysqld=--innodb-page-size=$k --mem --parallel=auto --suite=mariabackup --force; done | grep -E '\[ fail \]|--mysqld'
 
Logging: 10.1/mysql-test/mysql-test-run.pl  --mysqld=--innodb-buffer-pool-size=28M --mysqld=--innodb-page-size=4k --mem --parallel=auto --suite=mariabackup --force
 
mariabackup.xb_fulltext_encrypted        w3 [ fail ]
 
mariabackup.xb_file_key_management       w7 [ fail ]
 
mariabackup.full_backup                  w5 [ fail ]
 
mariabackup.partition_data_external      w3 [ fail ]
 
mariabackup.incremental_backup           w8 [ fail ]
 
mariabackup.incremental_encrypted        w6 [ fail ]
 
mariabackup.xb_partition                 w4 [ fail ]
 
mariabackup.partial                      w2 [ fail ]
 
mariabackup.xb_compressed_encrypted      w1 [ fail ]
 
mariabackup.xbstream                     w5 [ fail ]
 
mysql-test-run: *** ERROR: there were failing test cases
 
Logging: 10.1/mysql-test/mysql-test-run.pl  --mysqld=--innodb-buffer-pool-size=28M --mysqld=--innodb-page-size=8k --mem --parallel=auto --suite=mariabackup --force
 
mariabackup.xb_partition                 w1 [ fail ]
 
mariabackup.partition_data_external      w1 [ fail ]
 
...

the suite could not be run for 32k and 64k page size, because that would require bigger --innodb-log-file-size , which is blocked by MDEV-12709 "mariabackup mtr tests don't handle properly option --mysqld='--innodb..."

typical errors:
InnoDB: Unable to open undo tablespace './undo30579'.
xtrabackup: innodb_init(): Error occured.
...
InnoDB: Error: Current page size 8192 != page size on page 16384
InnoDB: innodb-page-size mismatch in tablespace ./test/p#P#p5.ibd (table test/p#P#p5)
2017-06-01 08:54:43 7f4cdeba4bc0 InnoDB: Operating system error number 2 in a file operation.
...

 Comments   
Comment by Marko Mäkelä [ 2017-07-05 ]

For the MariaDB 10.2 series, this is addressed by MDEV-12548, which introduces Mariabackup for 10.2.

Comment by Marko Mäkelä [ 2017-09-12 ]

A starting point would be to backport the relevant test changes from 10.2 to 10.1, like this:

diff --git a/mysql-test/suite/mariabackup/full_backup.test b/mysql-test/suite/mariabackup/full_backup.test
 
index f987410cf76..d1d2ea21c08 100644
 
--- a/mysql-test/suite/mariabackup/full_backup.test
 
+++ b/mysql-test/suite/mariabackup/full_backup.test
 
@@ -1,3 +1,5 @@
 
+--source include/innodb_page_size.inc
 
+
 
 CREATE TABLE t(i INT) ENGINE INNODB;
 
 INSERT INTO t VALUES(1);
 
 echo # xtrabackup backup;
 
diff --git a/mysql-test/suite/mariabackup/incremental_backup.test b/mysql-test/suite/mariabackup/incremental_backup.test
 
index b92d7b323ac..af6260dd591 100644
 
--- a/mysql-test/suite/mariabackup/incremental_backup.test
 
+++ b/mysql-test/suite/mariabackup/incremental_backup.test
 
@@ -1,10 +1,10 @@
 
-call mtr.add_suppression("InnoDB: New log files created");
 
+--source include/innodb_page_size.inc
 
 
+call mtr.add_suppression("InnoDB: New log files created");
 
 
 let $basedir=$MYSQLTEST_VARDIR/tmp/backup;
 
 let $incremental_dir=$MYSQLTEST_VARDIR/tmp/backup_inc1;
 
 
-
 
 CREATE TABLE t(i INT) ENGINE INNODB;
 
 INSERT INTO t VALUES(1);
 
 
@@ -32,4 +32,3 @@ DROP TABLE t;
 
 # Cleanup
 
 rmdir $basedir;
 
 rmdir $incremental_dir;
 
-
 
diff --git a/mysql-test/suite/mariabackup/incremental_encrypted.test b/mysql-test/suite/mariabackup/incremental_encrypted.test
 
index 04f4ba5d18a..d8f5e182ca2 100644
 
--- a/mysql-test/suite/mariabackup/incremental_encrypted.test
 
+++ b/mysql-test/suite/mariabackup/incremental_encrypted.test
 
@@ -1,3 +1,5 @@
 
+--source include/innodb_page_size.inc
 
+
 
 if (!$EXAMPLE_KEY_MANAGEMENT_SO)
 
 {
 
   --skip needs example_key_management plugin
 
diff --git a/mysql-test/suite/mariabackup/partial.test b/mysql-test/suite/mariabackup/partial.test
 
index 3b1de8ebc25..61a6450bb2d 100644
 
--- a/mysql-test/suite/mariabackup/partial.test
 
+++ b/mysql-test/suite/mariabackup/partial.test
 
@@ -1,8 +1,13 @@
 
+--source include/innodb_page_size.inc
 
+
 
 # Export single table from backup
 
 # (xtrabackup with --prepare --export)
 
 
 CREATE TABLE t1(i INT) ENGINE INNODB;
 
 INSERT INTO t1 VALUES(1);
 
+CREATE TABLE t21(i INT) ENGINE INNODB;
 
+INSERT INTO t21 VALUES(1);
 
+
 
 CREATE TABLE t2(i int) ENGINE INNODB;
 
 
 echo # xtrabackup backup;
 
@@ -13,13 +18,23 @@ exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf  --backup "--tables=te
 
 --enable_result_log
 
 list_files $targetdir/test *.ibd;
 
 
+# Inject a junk .ibd file into backup dir to
 
+# see if prepare does not choke on it.
 
+write_file $targetdir/test/junk.ibd;
 
+EOF
 
+write_file $targetdir/test/junk.frm;
 
+EOF
 
+
 
+
 
 echo # xtrabackup prepare;
 
 --disable_result_log
 
-exec $XTRABACKUP --prepare --export --target-dir=$targetdir;
 
+exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --defaults-group-suffix=.1 --prepare --export --target-dir=$targetdir;
 
 --enable_result_log
 
 
-ALTER TABLE t1 DISCARD TABLESPACE;
 
+list_files $targetdir/test *.cfg;
 
+
 
 let $MYSQLD_DATADIR= `select @@datadir`;
 
+ALTER TABLE t1 DISCARD TABLESPACE;
 
 copy_file $targetdir/test/t1.ibd $MYSQLD_DATADIR/test/t1.ibd;
 
 copy_file $targetdir/test/t1.cfg $MYSQLD_DATADIR/test/t1.cfg;
 
 ALTER TABLE t1 IMPORT TABLESPACE;
 
@@ -27,5 +42,5 @@ ALTER TABLE t1 IMPORT TABLESPACE;
 
 SELECT * FROM t1;
 
 DROP TABLE t1;
 
 DROP TABLE t2;
 
+DROP TABLE t21;
 
 rmdir $targetdir;
 
-
 
diff --git a/mysql-test/suite/mariabackup/partial_exclude.test b/mysql-test/suite/mariabackup/partial_exclude.test
 
index 631f9d7ee71..67020f28bb3 100644
 
--- a/mysql-test/suite/mariabackup/partial_exclude.test
 
+++ b/mysql-test/suite/mariabackup/partial_exclude.test
 
@@ -1,3 +1,5 @@
 
+--source include/innodb_page_size.inc
 
+
 
 # Test --databases-exclude and --tables-exclude feature of xtrabackup 2.3.8
 
 
 CREATE TABLE t1(i INT) ENGINE INNODB;
 
@@ -27,4 +29,3 @@ DROP TABLE t1;
 
 DROP TABLE t2;
 
 DROP DATABASE db2;
 
 rmdir $targetdir;
 
-
 
diff --git a/mysql-test/suite/mariabackup/small_ibd.test b/mysql-test/suite/mariabackup/small_ibd.test
 
index 23d6b08c047..e8175fce7c9 100644
 
--- a/mysql-test/suite/mariabackup/small_ibd.test
 
+++ b/mysql-test/suite/mariabackup/small_ibd.test
 
@@ -1,3 +1,5 @@
 
+--source include/innodb_page_size.inc
 
+
 
 # Check if ibd smaller than page size are skipped
 
 # It is possible, due to race conditions that new file
 
 # is created by server while xtrabackup is running
 
@@ -15,4 +17,3 @@ exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --backup --target-dir=
 
 --enable_result_log
 
 remove_file  $_datadir/test/small.ibd;
 
 rmdir $targetdir;
 
-
 
diff --git a/mysql-test/suite/mariabackup/xb_aws_key_management.test b/mysql-test/suite/mariabackup/xb_aws_key_management.test
 
index 149e422c058..9f69cbec8b8 100644
 
--- a/mysql-test/suite/mariabackup/xb_aws_key_management.test
 
+++ b/mysql-test/suite/mariabackup/xb_aws_key_management.test
 
@@ -1,7 +1,9 @@
 
+--source include/innodb_page_size.inc
 
 --source include/have_debug.inc
 
+
 
 if (`SELECT COUNT(*)=0 FROM INFORMATION_SCHEMA.PLUGINS WHERE PLUGIN_NAME = 'aws_key_management' AND PLUGIN_STATUS='ACTIVE'`)
 
 {
 
-  --skip needs aws_key_management plugin plugin
 
+  --skip needs aws_key_management plugin
 
 }
 
 
 CREATE TABLE t(c VARCHAR(10)) ENGINE INNODB encrypted=yes;
 
diff --git a/mysql-test/suite/mariabackup/xb_compressed_encrypted.test b/mysql-test/suite/mariabackup/xb_compressed_encrypted.test
 
index 9a5be4b19d6..2cca740d7dd 100644
 
--- a/mysql-test/suite/mariabackup/xb_compressed_encrypted.test
 
+++ b/mysql-test/suite/mariabackup/xb_compressed_encrypted.test
 
@@ -1,25 +1,22 @@
 
-source include/have_file_key_management.inc;
 
+--source include/innodb_page_size_small.inc
 
+--source include/have_file_key_management.inc
 
 
-CREATE TABLE t1(c1 INT, b VARCHAR(2400), index(b(100),c1)) ENGINE=INNODB ROW_FORMAT=compressed ENCRYPTED=YES;
 
+CREATE TABLE t1(c1 INT, b VARCHAR(2400), index(b(100),c1))
 
+ENGINE=INNODB ROW_FORMAT=compressed ENCRYPTED=YES;
 
 
-DELIMITER //;
 
-CREATE PROCEDURE innodb_insert_proc (REPEAT_COUNT INT)
 
-BEGIN
 
-  DECLARE CURRENT_NUM INT;
 
-  SET CURRENT_NUM = 0;
 
-  WHILE CURRENT_NUM < REPEAT_COUNT DO
 
-    INSERT INTO t1 VALUES(CURRENT_NUM, concat(uuid(), CURRENT_NUM, repeat('ab', floor(rand()*100) ), uuid()));
 
-    SET CURRENT_NUM = CURRENT_NUM + 1;
 
-  END WHILE;
 
-END//
 
-DELIMITER ;//
 
+BEGIN;
 
+let $n= 5000;
 
+let $i= $n;
 
+let $u= `SELECT uuid()`;
 
+--disable_query_log
 
+while ($i) {
 
+  eval INSERT INTO t1
 
+  VALUES($n-$i, concat('$u', $n-$i, repeat('ab', floor(rand()*100)), '$u'));
 
+  dec $i;
 
+}
 
+--enable_query_log
 
 COMMIT;
 
 
-SET AUTOCOMMIT=0;
 
-CALL innodb_insert_proc(50000);
 
-COMMIT;
 
-
 
-
 
 echo # xtrabackup backup;
 
 --disable_result_log
 
 let $targetdir=$MYSQLTEST_VARDIR/tmp/backup;
 
@@ -31,5 +28,4 @@ exec $INNOBACKUPEX --apply-log $targetdir;
 
 --enable_result_log
 
 select sum(c1) from t1;
 
 DROP TABLE t1;
 
-drop procedure innodb_insert_proc;
 
 rmdir $targetdir;
 
diff --git a/mysql-test/suite/mariabackup/xb_fulltext_encrypted.test b/mysql-test/suite/mariabackup/xb_fulltext_encrypted.test
 
index a98d7802d76..ed22769b2fe 100644
 
--- a/mysql-test/suite/mariabackup/xb_fulltext_encrypted.test
 
+++ b/mysql-test/suite/mariabackup/xb_fulltext_encrypted.test
 
@@ -1,4 +1,5 @@
 
-source include/have_file_key_management.inc;
 
+--source include/innodb_page_size.inc
 
+--source include/have_file_key_management.inc
 
 
 CREATE TABLE film_text (
 
 film_id SMALLINT NOT NULL,
 
diff --git a/mysql-test/suite/mariabackup/xb_history.test b/mysql-test/suite/mariabackup/xb_history.test
 
index 28de50127c6..5eca32db2db 100644
 
--- a/mysql-test/suite/mariabackup/xb_history.test
 
+++ b/mysql-test/suite/mariabackup/xb_history.test
 
@@ -1,3 +1,5 @@
 
+--source include/innodb_page_size.inc
 
+
 
 let $targetdir=$MYSQLTEST_VARDIR/tmp/backup;
 
 --disable_result_log
 
 exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --history=foo --backup --target-dir=$targetdir;
 
diff --git a/mysql-test/suite/mariabackup/xb_page_compress.test b/mysql-test/suite/mariabackup/xb_page_compress.test
 
index 876aa1a2791..7180ac555bc 100644
 
--- a/mysql-test/suite/mariabackup/xb_page_compress.test
 
+++ b/mysql-test/suite/mariabackup/xb_page_compress.test
 
@@ -1,3 +1,5 @@
 
+--source include/innodb_page_size.inc
 
+
 
 CREATE TABLE t1(c1 INT, b CHAR(20)) ENGINE=INNODB PAGE_COMPRESSED=1;
 
 
 DELIMITER //;
 
@@ -41,4 +43,3 @@ SELECT COUNT(*) FROM t1;
 
 DROP PROCEDURE innodb_insert_proc;
 
 DROP TABLE t1;
 
 rmdir $targetdir;
 
-
 
diff --git a/mysql-test/suite/mariabackup/xb_partition.test b/mysql-test/suite/mariabackup/xb_partition.test
 
index f051a52edbc..93fa3b3bcbe 100644
 
--- a/mysql-test/suite/mariabackup/xb_partition.test
 
+++ b/mysql-test/suite/mariabackup/xb_partition.test
 
@@ -1,3 +1,4 @@
 
+--source include/innodb_page_size.inc
 
 --source include/have_partition.inc
 
 
 CREATE TABLE t1(a INT) ENGINE=InnoDB;
 
diff --git a/mysql-test/suite/mariabackup/xbstream.test b/mysql-test/suite/mariabackup/xbstream.test
 
index f2b4704a87e..9161d227b20 100644
 
--- a/mysql-test/suite/mariabackup/xbstream.test
 
+++ b/mysql-test/suite/mariabackup/xbstream.test
 
@@ -1,3 +1,5 @@
 
+--source include/innodb_page_size.inc
 
+
 
 CREATE TABLE t(i INT) ENGINE INNODB;
 
 INSERT INTO t VALUES(1);

Currently, numerous tests fail, many crashing in --prepare due to this code invoked from log_group_init():

	if (free >= smallest_capacity / 2) {
 
		ib_logf(IB_LOG_LEVEL_FATAL,
 
			"The combined size of ib_logfiles"
 
			" should be bigger than\n"
 
			"InnoDB: 200 kB * innodb_thread_concurrency.");
 
	}

Comment by David Hill (Inactive) [ 2020-12-09 ]

Customer reported this issue using the version 10.5. Its been open for awhile, is there a plan for fix in a roadmap?

Comment by Marko Mäkelä [ 2021-10-26 ]

In the MariaDB Server 10.2 and later release series, MDEV-12548 fixed this bug and introduced the mariabackup tool. Before the 10.2.7 release, there was no really working online backup solution for MariaDB Server 10.2.

This reported bug only affected the 10.1 series. That series reached its announced end of life on October 17, 2020. The last release (mariadb-10.1.48) was made a couple weeks after that date, and this bug was not fixed there.

Generated at Thu Feb 08 08:01:52 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.