Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
10.2.6
Description
Attachments
Issue Links
- causes
-
MDEV-12794 innodb_zip.recover failed in buildbot
-
- Closed
-
- is caused by
-
MDEV-12123 Page contains nonzero PAGE_MAX_TRX_ID
-
- Closed
-
Activity
Field | Original Value | New Value |
---|---|---|
Attachment | MDEV-12720.sh [ 43621 ] |
Assignee | Andrii Nikitin [ anikitin ] | Jan Lindström [ jplindst ] |
Fix Version/s | 10.2 [ 14601 ] |
Attachment | ib_logfile0 [ 43628 ] |
Summary | xtrabackup 2.4.7 buf_flush_page_cleaner_coordinator may crash with ROW_FORMAT=compressed in 10.2 | recovery fails with "Generic error" for ROW_FORMAT=compressed |
Labels | 10.2-ga |
Description |
Following Xtrabackup 2.4.7 crash is observed during --apply-log in current 10.2 tree for tables with ROW_FORMAT=compressed
{noformat} $ /usr/bin/innobackupex --defaults-file=/dev/shm/__var0/var1/my.cnf --no-version-check --apply-log /dev/shm/__var0/var1/backup 170507 09:27:01 innobackupex: Starting the apply-log operation ... /usr/bin/innobackupex version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799) ... InnoDB: Doing recovery: scanned up to log sequence number 1770278 (2%) InnoDB: Plugin initialization aborted with error Generic error 07:27:01 UTC - xtrabackup got signal 11 ; ... /usr/bin/innobackupex(my_print_stacktrace+0x3b)[0x55b65d5dc52b] /usr/bin/innobackupex(handle_fatal_signal+0x291)[0x55b65d450711] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11630)[0x7f7744ce0630] /usr/bin/innobackupex(_Z19rw_lock_x_lock_funcP9rw_lock_tmPKcm+0x6a)[0x55b65cfae03a] /usr/bin/innobackupex(_Z15trx_purge_statev+0x2a)[0x55b65cfb375a] /usr/bin/innobackupex(_Z26srv_get_active_thread_typev+0xdb)[0x55b65cf948cb] /usr/bin/innobackupex(buf_flush_page_cleaner_coordinator+0x1dfc)[0x55b65cdcf89c] /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ca)[0x7f7744cd66ca] /lib/x86_64-linux-gnu/libc.so.6(clone+0x5f)[0x7f7742d4c0af] {noformat} The crash doesn't happen in 10.1.23, 10.2.5, 5.7.18 or when "ROW_FORMAT=compressed" is removed from the test or when table's data is not generated. Steps to reproduce (with installed xtrabackup 2.4.7) are below; please note that the crash doesn't happen if any delay (>1s) happens between INSERT and backup command, meaning that manual testing most probably will not trigger the problem: 1. Generate data: {code:sql} create table test.a(i int primary key auto_increment, s varchar(255) ) ENGINE=InnoDB ROW_FORMAT=compressed; insert into test.a(i) select null; insert into test.a select null, uuid() from test.a a, test.a b, test.a c; insert into test.a select null, uuid() from test.a a, test.a b, test.a c; insert into test.a select null, uuid() from test.a a, test.a b, test.a c; {code} 2. run backup: {code:bash} mkdir -p ~/backup innobackupex --defaults-file=/path/my.cnf --no-timestamp ~/backup innobackupex --apply-log ~/backup {code} Example 100% repeatable output: {noformat} $ ./runsuite.sh m1 x9 _plugin/xtrabackup/t/b.sh Checking prerequisites... starting test _plugin/xtrabackup/t/b.sh /usr/bin/xtrabackup /usr/bin/innobackupex /usr/bin/mysql Product=10.2.6 Product=version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799) OS="Ubuntu 16.10" configure_rest_encryption.sh=NA MYSQL_INSTALL_DB set to /home/a/test1/mariadb-environs/_depot/m-branch/10.2/scripts/mysql_install_db.sh MYSQLD set to /home/a/test1/mariadb-environs/m1-10.2/build/sql//mysqld MYSQL set to /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql MYSQLADMIN set to /home/a/test1/mariadb-environs/m1-10.2/build/client//mysqladmin MYSQLDUMP set to /home/a/test1/mariadb-environs/m1-10.2/build/client//mysqldump attempting to clean previous mysqld instances ls: cannot access '/dev/shm/__var0/mysqld*.pid': No such file or directory STARTING TEST 2017-05-07 09:52:05: bash: Starting server with id=1... 2017-05-07 09:52:05: bash: Creating server root directory: /dev/shm/__var0/var1 2017-05-07 09:52:05: bash: Creating server temporary directory: /dev/shm/__var0/var1/tmp 2017-05-07 09:52:05: bash: Creating server data directory: /dev/shm/__var0/var1/data 2017-05-07 09:52:05: bash: Calling mysql_install_db /home/a/test1/mariadb-environs/_depot/m-branch/10.2/scripts/mysql_install_db.sh: line 375: @HOSTNAME@: command not found WARNING: The host '' could not be looked up with resolveip. This probably means that your libc libraries are not 100 % compatible with this binary MariaDB version. The MariaDB daemon, mysqld, should work normally with the exception that host name resolving will not work. This means that you should use IP addresses instead of hostnames when specifying MariaDB privileges ! Installing MariaDB/MySQL system tables in '/dev/shm/__var0/var1/data' ... OK Starting /home/a/test1/mariadb-environs/m1-10.2/build/sql//mysqld --defaults-file=/dev/shm/__var0/var1/my.cnf --innodb_file_per_table --innodb_file_format=Barracuda 2017-05-07 9:52:07 139834146273088 [Note] /home/a/test1/mariadb-environs/m1-10.2/build/sql//mysqld (mysqld 10.2.6-MariaDB-log) starting as process 4660 ... 2017-05-07 9:52:07 139834146273088 [Warning] Changed limits: max_open_files: 1024 max_connections: 151 table_cache: 431 2017-05-07 09:52:08: bash: Made 1 attempts to connect to server 2017-05-07 09:52:08: bash: Server with id=1 has been started on port 12127, socket /dev/shm/__var0/tmp/mysql.sock.qOMbdv 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e create table test.a(i int primary key auto_increment, s varchar(255) ) ENGINE=InnoDB ROW_FORMAT=compressed 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e insert into test.a(i) select null 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e insert into test.a select null, uuid() from test.a a, test.a b, test.a c 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e insert into test.a select null, uuid() from test.a a, test.a b, test.a c 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e insert into test.a select null, uuid() from test.a a, test.a b, test.a c 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e select count(*) from test.a count(*) 1010 2017-05-07 09:52:08: bash: ===> /usr/bin/innobackupex --defaults-file=/dev/shm/__var0/var1/my.cnf --no-version-check --no-timestamp /dev/shm/__var0/var1/backup 170507 09:52:08 innobackupex: Starting the backup operation IMPORTANT: Please check that the backup run completes successfully. At the end of a successful backup run innobackupex prints "completed OK!". 170507 09:52:08 Connecting to MySQL server host: localhost, user: root, password: not set, port: not set, socket: /dev/shm/__var0/tmp/mysql.sock.qOMbdv Using server version 10.2.6-MariaDB-log /usr/bin/innobackupex version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799) xtrabackup: uses posix_fadvise(). xtrabackup: cd to /dev/shm/__var0/var1/data xtrabackup: open files limit requested 0, set to 1024 xtrabackup: using the following InnoDB configuration: xtrabackup: innodb_data_home_dir = . xtrabackup: innodb_data_file_path = ibdata1:12M:autoextend xtrabackup: innodb_log_group_home_dir = ./ xtrabackup: innodb_log_files_in_group = 2 xtrabackup: innodb_log_file_size = 50331648 InnoDB: Number of pools: 1 170507 09:52:08 >> log scanned up to (1770243) xtrabackup: Generating a list of tablespaces InnoDB: Allocated tablespace ID 4 for test/a, old maximum was 0 170507 09:52:08 [01] Copying ./ibdata1 to /dev/shm/__var0/var1/backup/ibdata1 170507 09:52:08 [01] ...done [01] ./test/a.ibd is compressed with page size = 8192 bytes 170507 09:52:08 [01] Copying ./test/a.ibd to /dev/shm/__var0/var1/backup/test/a.ibd 170507 09:52:08 [01] ...done 170507 09:52:08 [01] Copying ./mysql/gtid_slave_pos.ibd to /dev/shm/__var0/var1/backup/mysql/gtid_slave_pos.ibd 170507 09:52:08 [01] ...done 170507 09:52:08 [01] Copying ./mysql/innodb_index_stats.ibd to /dev/shm/__var0/var1/backup/mysql/innodb_index_stats.ibd 170507 09:52:08 [01] ...done 170507 09:52:08 [01] Copying ./mysql/innodb_table_stats.ibd to /dev/shm/__var0/var1/backup/mysql/innodb_table_stats.ibd 170507 09:52:08 [01] ...done 170507 09:52:09 >> log scanned up to (1770243) 170507 09:52:09 Executing FLUSH NO_WRITE_TO_BINLOG TABLES... 170507 09:52:09 Executing FLUSH TABLES WITH READ LOCK... 170507 09:52:09 Starting to backup non-InnoDB tables and files 170507 09:52:09 [01] Copying ./performance_schema/db.opt to /dev/shm/__var0/var1/backup/performance_schema/db.opt 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./test/a.frm to /dev/shm/__var0/var1/backup/test/a.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/gtid_slave_pos.frm to /dev/shm/__var0/var1/backup/mysql/gtid_slave_pos.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/index_stats.MYD to /dev/shm/__var0/var1/backup/mysql/index_stats.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/index_stats.MYI to /dev/shm/__var0/var1/backup/mysql/index_stats.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/index_stats.frm to /dev/shm/__var0/var1/backup/mysql/index_stats.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/column_stats.MYD to /dev/shm/__var0/var1/backup/mysql/column_stats.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/column_stats.MYI to /dev/shm/__var0/var1/backup/mysql/column_stats.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/column_stats.frm to /dev/shm/__var0/var1/backup/mysql/column_stats.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/table_stats.MYD to /dev/shm/__var0/var1/backup/mysql/table_stats.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/table_stats.MYI to /dev/shm/__var0/var1/backup/mysql/table_stats.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/table_stats.frm to /dev/shm/__var0/var1/backup/mysql/table_stats.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proxies_priv.MYD to /dev/shm/__var0/var1/backup/mysql/proxies_priv.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proxies_priv.MYI to /dev/shm/__var0/var1/backup/mysql/proxies_priv.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proxies_priv.frm to /dev/shm/__var0/var1/backup/mysql/proxies_priv.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/innodb_index_stats.frm to /dev/shm/__var0/var1/backup/mysql/innodb_index_stats.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/innodb_table_stats.frm to /dev/shm/__var0/var1/backup/mysql/innodb_table_stats.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/event.MYD to /dev/shm/__var0/var1/backup/mysql/event.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/event.MYI to /dev/shm/__var0/var1/backup/mysql/event.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/event.frm to /dev/shm/__var0/var1/backup/mysql/event.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/slow_log.CSV to /dev/shm/__var0/var1/backup/mysql/slow_log.CSV 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/slow_log.CSM to /dev/shm/__var0/var1/backup/mysql/slow_log.CSM 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/slow_log.frm to /dev/shm/__var0/var1/backup/mysql/slow_log.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/general_log.CSV to /dev/shm/__var0/var1/backup/mysql/general_log.CSV 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/general_log.CSM to /dev/shm/__var0/var1/backup/mysql/general_log.CSM 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/general_log.frm to /dev/shm/__var0/var1/backup/mysql/general_log.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/procs_priv.MYD to /dev/shm/__var0/var1/backup/mysql/procs_priv.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/procs_priv.MYI to /dev/shm/__var0/var1/backup/mysql/procs_priv.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/procs_priv.frm to /dev/shm/__var0/var1/backup/mysql/procs_priv.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proc.MYD to /dev/shm/__var0/var1/backup/mysql/proc.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proc.MYI to /dev/shm/__var0/var1/backup/mysql/proc.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proc.frm to /dev/shm/__var0/var1/backup/mysql/proc.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_leap_second.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone_leap_second.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_leap_second.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone_leap_second.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_leap_second.frm to /dev/shm/__var0/var1/backup/mysql/time_zone_leap_second.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition_type.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone_transition_type.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition_type.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone_transition_type.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition_type.frm to /dev/shm/__var0/var1/backup/mysql/time_zone_transition_type.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone_transition.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone_transition.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition.frm to /dev/shm/__var0/var1/backup/mysql/time_zone_transition.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone.frm to /dev/shm/__var0/var1/backup/mysql/time_zone.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_name.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone_name.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_name.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone_name.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_name.frm to /dev/shm/__var0/var1/backup/mysql/time_zone_name.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_keyword.MYD to /dev/shm/__var0/var1/backup/mysql/help_keyword.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_keyword.MYI to /dev/shm/__var0/var1/backup/mysql/help_keyword.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_keyword.frm to /dev/shm/__var0/var1/backup/mysql/help_keyword.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_relation.MYD to /dev/shm/__var0/var1/backup/mysql/help_relation.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_relation.MYI to /dev/shm/__var0/var1/backup/mysql/help_relation.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_relation.frm to /dev/shm/__var0/var1/backup/mysql/help_relation.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_category.MYD to /dev/shm/__var0/var1/backup/mysql/help_category.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_category.MYI to /dev/shm/__var0/var1/backup/mysql/help_category.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_category.frm to /dev/shm/__var0/var1/backup/mysql/help_category.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_topic.MYD to /dev/shm/__var0/var1/backup/mysql/help_topic.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_topic.MYI to /dev/shm/__var0/var1/backup/mysql/help_topic.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_topic.frm to /dev/shm/__var0/var1/backup/mysql/help_topic.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/columns_priv.MYD to /dev/shm/__var0/var1/backup/mysql/columns_priv.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/columns_priv.MYI to /dev/shm/__var0/var1/backup/mysql/columns_priv.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/columns_priv.frm to /dev/shm/__var0/var1/backup/mysql/columns_priv.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/tables_priv.MYD to /dev/shm/__var0/var1/backup/mysql/tables_priv.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/tables_priv.MYI to /dev/shm/__var0/var1/backup/mysql/tables_priv.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/tables_priv.frm to /dev/shm/__var0/var1/backup/mysql/tables_priv.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/servers.MYD to /dev/shm/__var0/var1/backup/mysql/servers.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/servers.MYI to /dev/shm/__var0/var1/backup/mysql/servers.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/servers.frm to /dev/shm/__var0/var1/backup/mysql/servers.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/plugin.MYD to /dev/shm/__var0/var1/backup/mysql/plugin.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/plugin.MYI to /dev/shm/__var0/var1/backup/mysql/plugin.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/plugin.frm to /dev/shm/__var0/var1/backup/mysql/plugin.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/func.MYD to /dev/shm/__var0/var1/backup/mysql/func.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/func.MYI to /dev/shm/__var0/var1/backup/mysql/func.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/func.frm to /dev/shm/__var0/var1/backup/mysql/func.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/roles_mapping.MYD to /dev/shm/__var0/var1/backup/mysql/roles_mapping.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/roles_mapping.MYI to /dev/shm/__var0/var1/backup/mysql/roles_mapping.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/roles_mapping.frm to /dev/shm/__var0/var1/backup/mysql/roles_mapping.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/user.MYD to /dev/shm/__var0/var1/backup/mysql/user.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/user.MYI to /dev/shm/__var0/var1/backup/mysql/user.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/user.frm to /dev/shm/__var0/var1/backup/mysql/user.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/host.MYD to /dev/shm/__var0/var1/backup/mysql/host.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/host.MYI to /dev/shm/__var0/var1/backup/mysql/host.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/host.frm to /dev/shm/__var0/var1/backup/mysql/host.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/db.MYD to /dev/shm/__var0/var1/backup/mysql/db.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/db.MYI to /dev/shm/__var0/var1/backup/mysql/db.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/db.frm to /dev/shm/__var0/var1/backup/mysql/db.frm 170507 09:52:09 [01] ...done 170507 09:52:09 Finished backing up non-InnoDB tables and files 170507 09:52:09 [00] Writing xtrabackup_binlog_info 170507 09:52:09 [00] ...done 170507 09:52:09 Executing FLUSH NO_WRITE_TO_BINLOG ENGINE LOGS... xtrabackup: The latest check point (for incremental): '1770243' xtrabackup: Stopping log copying thread. .170507 09:52:09 >> log scanned up to (1770343) 170507 09:52:09 Executing UNLOCK TABLES 170507 09:52:09 All tables unlocked 170507 09:52:09 [00] Copying ib_buffer_pool to /dev/shm/__var0/var1/backup/ib_buffer_pool 170507 09:52:09 [00] ...done 170507 09:52:09 Backup created in directory '/dev/shm/__var0/var1/backup/' MySQL binlog position: filename 'mysql-bin.000002', position '44041', GTID of the last change '0-1-54' 170507 09:52:09 [00] Writing backup-my.cnf 170507 09:52:09 [00] ...done 170507 09:52:09 [00] Writing xtrabackup_info 170507 09:52:09 [00] ...done xtrabackup: Transaction log of lsn (1620117) to (1770343) was copied. 170507 09:52:09 completed OK! 2017-05-07 09:52:09: bash: Killing server with id=1... inc/common.sh: line 419: 4660 Killed ${MYSQLD} ${MYSQLD_ARGS} $* 2017-05-07 09:52:09: bash: Applying log 2017-05-07 09:52:09: bash: ===> /usr/bin/innobackupex --defaults-file=/dev/shm/__var0/var1/my.cnf --no-version-check --apply-log /dev/shm/__var0/var1/backup 170507 09:52:09 innobackupex: Starting the apply-log operation IMPORTANT: Please check that the apply-log run completes successfully. At the end of a successful apply-log run innobackupex prints "completed OK!". /usr/bin/innobackupex version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799) xtrabackup: cd to /dev/shm/__var0/var1/backup/ xtrabackup: This target seems to be not prepared yet. InnoDB: Number of pools: 1 xtrabackup: xtrabackup_logfile detected: size=8388608, start_lsn=(1620117) xtrabackup: using the following InnoDB configuration for recovery: xtrabackup: innodb_data_home_dir = . xtrabackup: innodb_data_file_path = ibdata1:12M:autoextend xtrabackup: innodb_log_group_home_dir = . xtrabackup: innodb_log_files_in_group = 1 xtrabackup: innodb_log_file_size = 8388608 xtrabackup: using the following InnoDB configuration for recovery: xtrabackup: innodb_data_home_dir = . xtrabackup: innodb_data_file_path = ibdata1:12M:autoextend xtrabackup: innodb_log_group_home_dir = . xtrabackup: innodb_log_files_in_group = 1 xtrabackup: innodb_log_file_size = 8388608 xtrabackup: Starting InnoDB instance for recovery. xtrabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter) InnoDB: PUNCH HOLE support available InnoDB: Mutexes and rw_locks use GCC atomic builtins InnoDB: Uses event mutexes InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier InnoDB: Compressed tables use zlib 1.2.8 InnoDB: Number of pools: 1 InnoDB: Using CPU crc32 instructions InnoDB: Initializing buffer pool, total size = 100M, instances = 1, chunk size = 100M InnoDB: Completed initialization of buffer pool InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). InnoDB: Highest supported file format is Barracuda. InnoDB: Log scan progressed past the checkpoint lsn 1620117 InnoDB: Doing recovery: scanned up to log sequence number 1770343 (2%) InnoDB: Plugin initialization aborted with error Generic error 07:52:10 UTC - xtrabackup got signal 11 ; This could be because you hit a bug or data is corrupted. This error can also be caused by malfunctioning hardware. Attempting to collect some information that could help diagnose the problem. As this is a crash and something is definitely wrong, the information collection process might fail. Thread pointer: 0x0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0 thread_stack 0x10000 /usr/bin/innobackupex(my_print_stacktrace+0x3b)[0x55cc1d72552b] /usr/bin/innobackupex(handle_fatal_signal+0x291)[0x55cc1d599711] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11630)[0x7f5d86edc630] /usr/bin/innobackupex(_Z19rw_lock_x_lock_funcP9rw_lock_tmPKcm+0x6a)[0x55cc1d0f703a] /usr/bin/innobackupex(_Z15trx_purge_statev+0x2a)[0x55cc1d0fc75a] /usr/bin/innobackupex(_Z26srv_get_active_thread_typev+0xdb)[0x55cc1d0dd8cb] /usr/bin/innobackupex(buf_flush_page_cleaner_coordinator+0x1dfc)[0x55cc1cf1889c] /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ca)[0x7f5d86ed26ca] /lib/x86_64-linux-gnu/libc.so.6(clone+0x5f)[0x7f5d84f480af] Please report a bug at https://bugs.launchpad.net/percona-xtrabackup 2017-05-07 09:52:10: bash: ===> innobackupex failed with exit code 2 {noformat} |
EDIT: please see test case at
Following Xtrabackup 2.4.7 crash is observed during --apply-log in current 10.2 tree for tables with ROW_FORMAT=compressed {noformat} $ /usr/bin/innobackupex --defaults-file=/dev/shm/__var0/var1/my.cnf --no-version-check --apply-log /dev/shm/__var0/var1/backup 170507 09:27:01 innobackupex: Starting the apply-log operation ... /usr/bin/innobackupex version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799) ... InnoDB: Doing recovery: scanned up to log sequence number 1770278 (2%) InnoDB: Plugin initialization aborted with error Generic error 07:27:01 UTC - xtrabackup got signal 11 ; ... /usr/bin/innobackupex(my_print_stacktrace+0x3b)[0x55b65d5dc52b] /usr/bin/innobackupex(handle_fatal_signal+0x291)[0x55b65d450711] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11630)[0x7f7744ce0630] /usr/bin/innobackupex(_Z19rw_lock_x_lock_funcP9rw_lock_tmPKcm+0x6a)[0x55b65cfae03a] /usr/bin/innobackupex(_Z15trx_purge_statev+0x2a)[0x55b65cfb375a] /usr/bin/innobackupex(_Z26srv_get_active_thread_typev+0xdb)[0x55b65cf948cb] /usr/bin/innobackupex(buf_flush_page_cleaner_coordinator+0x1dfc)[0x55b65cdcf89c] /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ca)[0x7f7744cd66ca] /lib/x86_64-linux-gnu/libc.so.6(clone+0x5f)[0x7f7742d4c0af] {noformat} The crash doesn't happen in 10.1.23, 10.2.5, 5.7.18 or when "ROW_FORMAT=compressed" is removed from the test or when table's data is not generated. Steps to reproduce (with installed xtrabackup 2.4.7) are below; please note that the crash doesn't happen if any delay (>1s) happens between INSERT and backup command, meaning that manual testing most probably will not trigger the problem: 1. Generate data: {code:sql} create table test.a(i int primary key auto_increment, s varchar(255) ) ENGINE=InnoDB ROW_FORMAT=compressed; insert into test.a(i) select null; insert into test.a select null, uuid() from test.a a, test.a b, test.a c; insert into test.a select null, uuid() from test.a a, test.a b, test.a c; insert into test.a select null, uuid() from test.a a, test.a b, test.a c; {code} 2. run backup: {code:bash} mkdir -p ~/backup innobackupex --defaults-file=/path/my.cnf --no-timestamp ~/backup innobackupex --apply-log ~/backup {code} Example 100% repeatable output: {noformat} $ ./runsuite.sh m1 x9 _plugin/xtrabackup/t/b.sh Checking prerequisites... starting test _plugin/xtrabackup/t/b.sh /usr/bin/xtrabackup /usr/bin/innobackupex /usr/bin/mysql Product=10.2.6 Product=version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799) OS="Ubuntu 16.10" configure_rest_encryption.sh=NA MYSQL_INSTALL_DB set to /home/a/test1/mariadb-environs/_depot/m-branch/10.2/scripts/mysql_install_db.sh MYSQLD set to /home/a/test1/mariadb-environs/m1-10.2/build/sql//mysqld MYSQL set to /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql MYSQLADMIN set to /home/a/test1/mariadb-environs/m1-10.2/build/client//mysqladmin MYSQLDUMP set to /home/a/test1/mariadb-environs/m1-10.2/build/client//mysqldump attempting to clean previous mysqld instances ls: cannot access '/dev/shm/__var0/mysqld*.pid': No such file or directory STARTING TEST 2017-05-07 09:52:05: bash: Starting server with id=1... 2017-05-07 09:52:05: bash: Creating server root directory: /dev/shm/__var0/var1 2017-05-07 09:52:05: bash: Creating server temporary directory: /dev/shm/__var0/var1/tmp 2017-05-07 09:52:05: bash: Creating server data directory: /dev/shm/__var0/var1/data 2017-05-07 09:52:05: bash: Calling mysql_install_db /home/a/test1/mariadb-environs/_depot/m-branch/10.2/scripts/mysql_install_db.sh: line 375: @HOSTNAME@: command not found WARNING: The host '' could not be looked up with resolveip. This probably means that your libc libraries are not 100 % compatible with this binary MariaDB version. The MariaDB daemon, mysqld, should work normally with the exception that host name resolving will not work. This means that you should use IP addresses instead of hostnames when specifying MariaDB privileges ! Installing MariaDB/MySQL system tables in '/dev/shm/__var0/var1/data' ... OK Starting /home/a/test1/mariadb-environs/m1-10.2/build/sql//mysqld --defaults-file=/dev/shm/__var0/var1/my.cnf --innodb_file_per_table --innodb_file_format=Barracuda 2017-05-07 9:52:07 139834146273088 [Note] /home/a/test1/mariadb-environs/m1-10.2/build/sql//mysqld (mysqld 10.2.6-MariaDB-log) starting as process 4660 ... 2017-05-07 9:52:07 139834146273088 [Warning] Changed limits: max_open_files: 1024 max_connections: 151 table_cache: 431 2017-05-07 09:52:08: bash: Made 1 attempts to connect to server 2017-05-07 09:52:08: bash: Server with id=1 has been started on port 12127, socket /dev/shm/__var0/tmp/mysql.sock.qOMbdv 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e create table test.a(i int primary key auto_increment, s varchar(255) ) ENGINE=InnoDB ROW_FORMAT=compressed 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e insert into test.a(i) select null 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e insert into test.a select null, uuid() from test.a a, test.a b, test.a c 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e insert into test.a select null, uuid() from test.a a, test.a b, test.a c 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e insert into test.a select null, uuid() from test.a a, test.a b, test.a c 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e select count(*) from test.a count(*) 1010 2017-05-07 09:52:08: bash: ===> /usr/bin/innobackupex --defaults-file=/dev/shm/__var0/var1/my.cnf --no-version-check --no-timestamp /dev/shm/__var0/var1/backup 170507 09:52:08 innobackupex: Starting the backup operation IMPORTANT: Please check that the backup run completes successfully. At the end of a successful backup run innobackupex prints "completed OK!". 170507 09:52:08 Connecting to MySQL server host: localhost, user: root, password: not set, port: not set, socket: /dev/shm/__var0/tmp/mysql.sock.qOMbdv Using server version 10.2.6-MariaDB-log /usr/bin/innobackupex version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799) xtrabackup: uses posix_fadvise(). xtrabackup: cd to /dev/shm/__var0/var1/data xtrabackup: open files limit requested 0, set to 1024 xtrabackup: using the following InnoDB configuration: xtrabackup: innodb_data_home_dir = . xtrabackup: innodb_data_file_path = ibdata1:12M:autoextend xtrabackup: innodb_log_group_home_dir = ./ xtrabackup: innodb_log_files_in_group = 2 xtrabackup: innodb_log_file_size = 50331648 InnoDB: Number of pools: 1 170507 09:52:08 >> log scanned up to (1770243) xtrabackup: Generating a list of tablespaces InnoDB: Allocated tablespace ID 4 for test/a, old maximum was 0 170507 09:52:08 [01] Copying ./ibdata1 to /dev/shm/__var0/var1/backup/ibdata1 170507 09:52:08 [01] ...done [01] ./test/a.ibd is compressed with page size = 8192 bytes 170507 09:52:08 [01] Copying ./test/a.ibd to /dev/shm/__var0/var1/backup/test/a.ibd 170507 09:52:08 [01] ...done 170507 09:52:08 [01] Copying ./mysql/gtid_slave_pos.ibd to /dev/shm/__var0/var1/backup/mysql/gtid_slave_pos.ibd 170507 09:52:08 [01] ...done 170507 09:52:08 [01] Copying ./mysql/innodb_index_stats.ibd to /dev/shm/__var0/var1/backup/mysql/innodb_index_stats.ibd 170507 09:52:08 [01] ...done 170507 09:52:08 [01] Copying ./mysql/innodb_table_stats.ibd to /dev/shm/__var0/var1/backup/mysql/innodb_table_stats.ibd 170507 09:52:08 [01] ...done 170507 09:52:09 >> log scanned up to (1770243) 170507 09:52:09 Executing FLUSH NO_WRITE_TO_BINLOG TABLES... 170507 09:52:09 Executing FLUSH TABLES WITH READ LOCK... 170507 09:52:09 Starting to backup non-InnoDB tables and files 170507 09:52:09 [01] Copying ./performance_schema/db.opt to /dev/shm/__var0/var1/backup/performance_schema/db.opt 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./test/a.frm to /dev/shm/__var0/var1/backup/test/a.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/gtid_slave_pos.frm to /dev/shm/__var0/var1/backup/mysql/gtid_slave_pos.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/index_stats.MYD to /dev/shm/__var0/var1/backup/mysql/index_stats.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/index_stats.MYI to /dev/shm/__var0/var1/backup/mysql/index_stats.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/index_stats.frm to /dev/shm/__var0/var1/backup/mysql/index_stats.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/column_stats.MYD to /dev/shm/__var0/var1/backup/mysql/column_stats.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/column_stats.MYI to /dev/shm/__var0/var1/backup/mysql/column_stats.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/column_stats.frm to /dev/shm/__var0/var1/backup/mysql/column_stats.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/table_stats.MYD to /dev/shm/__var0/var1/backup/mysql/table_stats.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/table_stats.MYI to /dev/shm/__var0/var1/backup/mysql/table_stats.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/table_stats.frm to /dev/shm/__var0/var1/backup/mysql/table_stats.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proxies_priv.MYD to /dev/shm/__var0/var1/backup/mysql/proxies_priv.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proxies_priv.MYI to /dev/shm/__var0/var1/backup/mysql/proxies_priv.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proxies_priv.frm to /dev/shm/__var0/var1/backup/mysql/proxies_priv.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/innodb_index_stats.frm to /dev/shm/__var0/var1/backup/mysql/innodb_index_stats.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/innodb_table_stats.frm to /dev/shm/__var0/var1/backup/mysql/innodb_table_stats.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/event.MYD to /dev/shm/__var0/var1/backup/mysql/event.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/event.MYI to /dev/shm/__var0/var1/backup/mysql/event.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/event.frm to /dev/shm/__var0/var1/backup/mysql/event.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/slow_log.CSV to /dev/shm/__var0/var1/backup/mysql/slow_log.CSV 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/slow_log.CSM to /dev/shm/__var0/var1/backup/mysql/slow_log.CSM 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/slow_log.frm to /dev/shm/__var0/var1/backup/mysql/slow_log.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/general_log.CSV to /dev/shm/__var0/var1/backup/mysql/general_log.CSV 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/general_log.CSM to /dev/shm/__var0/var1/backup/mysql/general_log.CSM 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/general_log.frm to /dev/shm/__var0/var1/backup/mysql/general_log.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/procs_priv.MYD to /dev/shm/__var0/var1/backup/mysql/procs_priv.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/procs_priv.MYI to /dev/shm/__var0/var1/backup/mysql/procs_priv.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/procs_priv.frm to /dev/shm/__var0/var1/backup/mysql/procs_priv.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proc.MYD to /dev/shm/__var0/var1/backup/mysql/proc.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proc.MYI to /dev/shm/__var0/var1/backup/mysql/proc.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proc.frm to /dev/shm/__var0/var1/backup/mysql/proc.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_leap_second.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone_leap_second.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_leap_second.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone_leap_second.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_leap_second.frm to /dev/shm/__var0/var1/backup/mysql/time_zone_leap_second.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition_type.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone_transition_type.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition_type.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone_transition_type.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition_type.frm to /dev/shm/__var0/var1/backup/mysql/time_zone_transition_type.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone_transition.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone_transition.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition.frm to /dev/shm/__var0/var1/backup/mysql/time_zone_transition.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone.frm to /dev/shm/__var0/var1/backup/mysql/time_zone.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_name.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone_name.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_name.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone_name.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_name.frm to /dev/shm/__var0/var1/backup/mysql/time_zone_name.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_keyword.MYD to /dev/shm/__var0/var1/backup/mysql/help_keyword.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_keyword.MYI to /dev/shm/__var0/var1/backup/mysql/help_keyword.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_keyword.frm to /dev/shm/__var0/var1/backup/mysql/help_keyword.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_relation.MYD to /dev/shm/__var0/var1/backup/mysql/help_relation.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_relation.MYI to /dev/shm/__var0/var1/backup/mysql/help_relation.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_relation.frm to /dev/shm/__var0/var1/backup/mysql/help_relation.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_category.MYD to /dev/shm/__var0/var1/backup/mysql/help_category.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_category.MYI to /dev/shm/__var0/var1/backup/mysql/help_category.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_category.frm to /dev/shm/__var0/var1/backup/mysql/help_category.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_topic.MYD to /dev/shm/__var0/var1/backup/mysql/help_topic.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_topic.MYI to /dev/shm/__var0/var1/backup/mysql/help_topic.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_topic.frm to /dev/shm/__var0/var1/backup/mysql/help_topic.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/columns_priv.MYD to /dev/shm/__var0/var1/backup/mysql/columns_priv.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/columns_priv.MYI to /dev/shm/__var0/var1/backup/mysql/columns_priv.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/columns_priv.frm to /dev/shm/__var0/var1/backup/mysql/columns_priv.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/tables_priv.MYD to /dev/shm/__var0/var1/backup/mysql/tables_priv.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/tables_priv.MYI to /dev/shm/__var0/var1/backup/mysql/tables_priv.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/tables_priv.frm to /dev/shm/__var0/var1/backup/mysql/tables_priv.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/servers.MYD to /dev/shm/__var0/var1/backup/mysql/servers.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/servers.MYI to /dev/shm/__var0/var1/backup/mysql/servers.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/servers.frm to /dev/shm/__var0/var1/backup/mysql/servers.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/plugin.MYD to /dev/shm/__var0/var1/backup/mysql/plugin.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/plugin.MYI to /dev/shm/__var0/var1/backup/mysql/plugin.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/plugin.frm to /dev/shm/__var0/var1/backup/mysql/plugin.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/func.MYD to /dev/shm/__var0/var1/backup/mysql/func.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/func.MYI to /dev/shm/__var0/var1/backup/mysql/func.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/func.frm to /dev/shm/__var0/var1/backup/mysql/func.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/roles_mapping.MYD to /dev/shm/__var0/var1/backup/mysql/roles_mapping.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/roles_mapping.MYI to /dev/shm/__var0/var1/backup/mysql/roles_mapping.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/roles_mapping.frm to /dev/shm/__var0/var1/backup/mysql/roles_mapping.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/user.MYD to /dev/shm/__var0/var1/backup/mysql/user.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/user.MYI to /dev/shm/__var0/var1/backup/mysql/user.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/user.frm to /dev/shm/__var0/var1/backup/mysql/user.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/host.MYD to /dev/shm/__var0/var1/backup/mysql/host.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/host.MYI to /dev/shm/__var0/var1/backup/mysql/host.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/host.frm to /dev/shm/__var0/var1/backup/mysql/host.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/db.MYD to /dev/shm/__var0/var1/backup/mysql/db.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/db.MYI to /dev/shm/__var0/var1/backup/mysql/db.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/db.frm to /dev/shm/__var0/var1/backup/mysql/db.frm 170507 09:52:09 [01] ...done 170507 09:52:09 Finished backing up non-InnoDB tables and files 170507 09:52:09 [00] Writing xtrabackup_binlog_info 170507 09:52:09 [00] ...done 170507 09:52:09 Executing FLUSH NO_WRITE_TO_BINLOG ENGINE LOGS... xtrabackup: The latest check point (for incremental): '1770243' xtrabackup: Stopping log copying thread. .170507 09:52:09 >> log scanned up to (1770343) 170507 09:52:09 Executing UNLOCK TABLES 170507 09:52:09 All tables unlocked 170507 09:52:09 [00] Copying ib_buffer_pool to /dev/shm/__var0/var1/backup/ib_buffer_pool 170507 09:52:09 [00] ...done 170507 09:52:09 Backup created in directory '/dev/shm/__var0/var1/backup/' MySQL binlog position: filename 'mysql-bin.000002', position '44041', GTID of the last change '0-1-54' 170507 09:52:09 [00] Writing backup-my.cnf 170507 09:52:09 [00] ...done 170507 09:52:09 [00] Writing xtrabackup_info 170507 09:52:09 [00] ...done xtrabackup: Transaction log of lsn (1620117) to (1770343) was copied. 170507 09:52:09 completed OK! 2017-05-07 09:52:09: bash: Killing server with id=1... inc/common.sh: line 419: 4660 Killed ${MYSQLD} ${MYSQLD_ARGS} $* 2017-05-07 09:52:09: bash: Applying log 2017-05-07 09:52:09: bash: ===> /usr/bin/innobackupex --defaults-file=/dev/shm/__var0/var1/my.cnf --no-version-check --apply-log /dev/shm/__var0/var1/backup 170507 09:52:09 innobackupex: Starting the apply-log operation IMPORTANT: Please check that the apply-log run completes successfully. At the end of a successful apply-log run innobackupex prints "completed OK!". /usr/bin/innobackupex version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799) xtrabackup: cd to /dev/shm/__var0/var1/backup/ xtrabackup: This target seems to be not prepared yet. InnoDB: Number of pools: 1 xtrabackup: xtrabackup_logfile detected: size=8388608, start_lsn=(1620117) xtrabackup: using the following InnoDB configuration for recovery: xtrabackup: innodb_data_home_dir = . xtrabackup: innodb_data_file_path = ibdata1:12M:autoextend xtrabackup: innodb_log_group_home_dir = . xtrabackup: innodb_log_files_in_group = 1 xtrabackup: innodb_log_file_size = 8388608 xtrabackup: using the following InnoDB configuration for recovery: xtrabackup: innodb_data_home_dir = . xtrabackup: innodb_data_file_path = ibdata1:12M:autoextend xtrabackup: innodb_log_group_home_dir = . xtrabackup: innodb_log_files_in_group = 1 xtrabackup: innodb_log_file_size = 8388608 xtrabackup: Starting InnoDB instance for recovery. xtrabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter) InnoDB: PUNCH HOLE support available InnoDB: Mutexes and rw_locks use GCC atomic builtins InnoDB: Uses event mutexes InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier InnoDB: Compressed tables use zlib 1.2.8 InnoDB: Number of pools: 1 InnoDB: Using CPU crc32 instructions InnoDB: Initializing buffer pool, total size = 100M, instances = 1, chunk size = 100M InnoDB: Completed initialization of buffer pool InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). InnoDB: Highest supported file format is Barracuda. InnoDB: Log scan progressed past the checkpoint lsn 1620117 InnoDB: Doing recovery: scanned up to log sequence number 1770343 (2%) InnoDB: Plugin initialization aborted with error Generic error 07:52:10 UTC - xtrabackup got signal 11 ; This could be because you hit a bug or data is corrupted. This error can also be caused by malfunctioning hardware. Attempting to collect some information that could help diagnose the problem. As this is a crash and something is definitely wrong, the information collection process might fail. Thread pointer: 0x0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0 thread_stack 0x10000 /usr/bin/innobackupex(my_print_stacktrace+0x3b)[0x55cc1d72552b] /usr/bin/innobackupex(handle_fatal_signal+0x291)[0x55cc1d599711] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11630)[0x7f5d86edc630] /usr/bin/innobackupex(_Z19rw_lock_x_lock_funcP9rw_lock_tmPKcm+0x6a)[0x55cc1d0f703a] /usr/bin/innobackupex(_Z15trx_purge_statev+0x2a)[0x55cc1d0fc75a] /usr/bin/innobackupex(_Z26srv_get_active_thread_typev+0xdb)[0x55cc1d0dd8cb] /usr/bin/innobackupex(buf_flush_page_cleaner_coordinator+0x1dfc)[0x55cc1cf1889c] /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ca)[0x7f5d86ed26ca] /lib/x86_64-linux-gnu/libc.so.6(clone+0x5f)[0x7f5d84f480af] Please report a bug at https://bugs.launchpad.net/percona-xtrabackup 2017-05-07 09:52:10: bash: ===> innobackupex failed with exit code 2 {noformat} |
Description |
EDIT: please see test case at
Following Xtrabackup 2.4.7 crash is observed during --apply-log in current 10.2 tree for tables with ROW_FORMAT=compressed {noformat} $ /usr/bin/innobackupex --defaults-file=/dev/shm/__var0/var1/my.cnf --no-version-check --apply-log /dev/shm/__var0/var1/backup 170507 09:27:01 innobackupex: Starting the apply-log operation ... /usr/bin/innobackupex version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799) ... InnoDB: Doing recovery: scanned up to log sequence number 1770278 (2%) InnoDB: Plugin initialization aborted with error Generic error 07:27:01 UTC - xtrabackup got signal 11 ; ... /usr/bin/innobackupex(my_print_stacktrace+0x3b)[0x55b65d5dc52b] /usr/bin/innobackupex(handle_fatal_signal+0x291)[0x55b65d450711] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11630)[0x7f7744ce0630] /usr/bin/innobackupex(_Z19rw_lock_x_lock_funcP9rw_lock_tmPKcm+0x6a)[0x55b65cfae03a] /usr/bin/innobackupex(_Z15trx_purge_statev+0x2a)[0x55b65cfb375a] /usr/bin/innobackupex(_Z26srv_get_active_thread_typev+0xdb)[0x55b65cf948cb] /usr/bin/innobackupex(buf_flush_page_cleaner_coordinator+0x1dfc)[0x55b65cdcf89c] /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ca)[0x7f7744cd66ca] /lib/x86_64-linux-gnu/libc.so.6(clone+0x5f)[0x7f7742d4c0af] {noformat} The crash doesn't happen in 10.1.23, 10.2.5, 5.7.18 or when "ROW_FORMAT=compressed" is removed from the test or when table's data is not generated. Steps to reproduce (with installed xtrabackup 2.4.7) are below; please note that the crash doesn't happen if any delay (>1s) happens between INSERT and backup command, meaning that manual testing most probably will not trigger the problem: 1. Generate data: {code:sql} create table test.a(i int primary key auto_increment, s varchar(255) ) ENGINE=InnoDB ROW_FORMAT=compressed; insert into test.a(i) select null; insert into test.a select null, uuid() from test.a a, test.a b, test.a c; insert into test.a select null, uuid() from test.a a, test.a b, test.a c; insert into test.a select null, uuid() from test.a a, test.a b, test.a c; {code} 2. run backup: {code:bash} mkdir -p ~/backup innobackupex --defaults-file=/path/my.cnf --no-timestamp ~/backup innobackupex --apply-log ~/backup {code} Example 100% repeatable output: {noformat} $ ./runsuite.sh m1 x9 _plugin/xtrabackup/t/b.sh Checking prerequisites... starting test _plugin/xtrabackup/t/b.sh /usr/bin/xtrabackup /usr/bin/innobackupex /usr/bin/mysql Product=10.2.6 Product=version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799) OS="Ubuntu 16.10" configure_rest_encryption.sh=NA MYSQL_INSTALL_DB set to /home/a/test1/mariadb-environs/_depot/m-branch/10.2/scripts/mysql_install_db.sh MYSQLD set to /home/a/test1/mariadb-environs/m1-10.2/build/sql//mysqld MYSQL set to /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql MYSQLADMIN set to /home/a/test1/mariadb-environs/m1-10.2/build/client//mysqladmin MYSQLDUMP set to /home/a/test1/mariadb-environs/m1-10.2/build/client//mysqldump attempting to clean previous mysqld instances ls: cannot access '/dev/shm/__var0/mysqld*.pid': No such file or directory STARTING TEST 2017-05-07 09:52:05: bash: Starting server with id=1... 2017-05-07 09:52:05: bash: Creating server root directory: /dev/shm/__var0/var1 2017-05-07 09:52:05: bash: Creating server temporary directory: /dev/shm/__var0/var1/tmp 2017-05-07 09:52:05: bash: Creating server data directory: /dev/shm/__var0/var1/data 2017-05-07 09:52:05: bash: Calling mysql_install_db /home/a/test1/mariadb-environs/_depot/m-branch/10.2/scripts/mysql_install_db.sh: line 375: @HOSTNAME@: command not found WARNING: The host '' could not be looked up with resolveip. This probably means that your libc libraries are not 100 % compatible with this binary MariaDB version. The MariaDB daemon, mysqld, should work normally with the exception that host name resolving will not work. This means that you should use IP addresses instead of hostnames when specifying MariaDB privileges ! Installing MariaDB/MySQL system tables in '/dev/shm/__var0/var1/data' ... OK Starting /home/a/test1/mariadb-environs/m1-10.2/build/sql//mysqld --defaults-file=/dev/shm/__var0/var1/my.cnf --innodb_file_per_table --innodb_file_format=Barracuda 2017-05-07 9:52:07 139834146273088 [Note] /home/a/test1/mariadb-environs/m1-10.2/build/sql//mysqld (mysqld 10.2.6-MariaDB-log) starting as process 4660 ... 2017-05-07 9:52:07 139834146273088 [Warning] Changed limits: max_open_files: 1024 max_connections: 151 table_cache: 431 2017-05-07 09:52:08: bash: Made 1 attempts to connect to server 2017-05-07 09:52:08: bash: Server with id=1 has been started on port 12127, socket /dev/shm/__var0/tmp/mysql.sock.qOMbdv 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e create table test.a(i int primary key auto_increment, s varchar(255) ) ENGINE=InnoDB ROW_FORMAT=compressed 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e insert into test.a(i) select null 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e insert into test.a select null, uuid() from test.a a, test.a b, test.a c 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e insert into test.a select null, uuid() from test.a a, test.a b, test.a c 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e insert into test.a select null, uuid() from test.a a, test.a b, test.a c 2017-05-07 09:52:08: bash: ===> /home/a/test1/mariadb-environs/m1-10.2/build/client//mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e select count(*) from test.a count(*) 1010 2017-05-07 09:52:08: bash: ===> /usr/bin/innobackupex --defaults-file=/dev/shm/__var0/var1/my.cnf --no-version-check --no-timestamp /dev/shm/__var0/var1/backup 170507 09:52:08 innobackupex: Starting the backup operation IMPORTANT: Please check that the backup run completes successfully. At the end of a successful backup run innobackupex prints "completed OK!". 170507 09:52:08 Connecting to MySQL server host: localhost, user: root, password: not set, port: not set, socket: /dev/shm/__var0/tmp/mysql.sock.qOMbdv Using server version 10.2.6-MariaDB-log /usr/bin/innobackupex version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799) xtrabackup: uses posix_fadvise(). xtrabackup: cd to /dev/shm/__var0/var1/data xtrabackup: open files limit requested 0, set to 1024 xtrabackup: using the following InnoDB configuration: xtrabackup: innodb_data_home_dir = . xtrabackup: innodb_data_file_path = ibdata1:12M:autoextend xtrabackup: innodb_log_group_home_dir = ./ xtrabackup: innodb_log_files_in_group = 2 xtrabackup: innodb_log_file_size = 50331648 InnoDB: Number of pools: 1 170507 09:52:08 >> log scanned up to (1770243) xtrabackup: Generating a list of tablespaces InnoDB: Allocated tablespace ID 4 for test/a, old maximum was 0 170507 09:52:08 [01] Copying ./ibdata1 to /dev/shm/__var0/var1/backup/ibdata1 170507 09:52:08 [01] ...done [01] ./test/a.ibd is compressed with page size = 8192 bytes 170507 09:52:08 [01] Copying ./test/a.ibd to /dev/shm/__var0/var1/backup/test/a.ibd 170507 09:52:08 [01] ...done 170507 09:52:08 [01] Copying ./mysql/gtid_slave_pos.ibd to /dev/shm/__var0/var1/backup/mysql/gtid_slave_pos.ibd 170507 09:52:08 [01] ...done 170507 09:52:08 [01] Copying ./mysql/innodb_index_stats.ibd to /dev/shm/__var0/var1/backup/mysql/innodb_index_stats.ibd 170507 09:52:08 [01] ...done 170507 09:52:08 [01] Copying ./mysql/innodb_table_stats.ibd to /dev/shm/__var0/var1/backup/mysql/innodb_table_stats.ibd 170507 09:52:08 [01] ...done 170507 09:52:09 >> log scanned up to (1770243) 170507 09:52:09 Executing FLUSH NO_WRITE_TO_BINLOG TABLES... 170507 09:52:09 Executing FLUSH TABLES WITH READ LOCK... 170507 09:52:09 Starting to backup non-InnoDB tables and files 170507 09:52:09 [01] Copying ./performance_schema/db.opt to /dev/shm/__var0/var1/backup/performance_schema/db.opt 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./test/a.frm to /dev/shm/__var0/var1/backup/test/a.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/gtid_slave_pos.frm to /dev/shm/__var0/var1/backup/mysql/gtid_slave_pos.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/index_stats.MYD to /dev/shm/__var0/var1/backup/mysql/index_stats.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/index_stats.MYI to /dev/shm/__var0/var1/backup/mysql/index_stats.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/index_stats.frm to /dev/shm/__var0/var1/backup/mysql/index_stats.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/column_stats.MYD to /dev/shm/__var0/var1/backup/mysql/column_stats.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/column_stats.MYI to /dev/shm/__var0/var1/backup/mysql/column_stats.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/column_stats.frm to /dev/shm/__var0/var1/backup/mysql/column_stats.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/table_stats.MYD to /dev/shm/__var0/var1/backup/mysql/table_stats.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/table_stats.MYI to /dev/shm/__var0/var1/backup/mysql/table_stats.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/table_stats.frm to /dev/shm/__var0/var1/backup/mysql/table_stats.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proxies_priv.MYD to /dev/shm/__var0/var1/backup/mysql/proxies_priv.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proxies_priv.MYI to /dev/shm/__var0/var1/backup/mysql/proxies_priv.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proxies_priv.frm to /dev/shm/__var0/var1/backup/mysql/proxies_priv.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/innodb_index_stats.frm to /dev/shm/__var0/var1/backup/mysql/innodb_index_stats.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/innodb_table_stats.frm to /dev/shm/__var0/var1/backup/mysql/innodb_table_stats.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/event.MYD to /dev/shm/__var0/var1/backup/mysql/event.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/event.MYI to /dev/shm/__var0/var1/backup/mysql/event.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/event.frm to /dev/shm/__var0/var1/backup/mysql/event.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/slow_log.CSV to /dev/shm/__var0/var1/backup/mysql/slow_log.CSV 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/slow_log.CSM to /dev/shm/__var0/var1/backup/mysql/slow_log.CSM 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/slow_log.frm to /dev/shm/__var0/var1/backup/mysql/slow_log.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/general_log.CSV to /dev/shm/__var0/var1/backup/mysql/general_log.CSV 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/general_log.CSM to /dev/shm/__var0/var1/backup/mysql/general_log.CSM 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/general_log.frm to /dev/shm/__var0/var1/backup/mysql/general_log.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/procs_priv.MYD to /dev/shm/__var0/var1/backup/mysql/procs_priv.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/procs_priv.MYI to /dev/shm/__var0/var1/backup/mysql/procs_priv.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/procs_priv.frm to /dev/shm/__var0/var1/backup/mysql/procs_priv.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proc.MYD to /dev/shm/__var0/var1/backup/mysql/proc.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proc.MYI to /dev/shm/__var0/var1/backup/mysql/proc.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/proc.frm to /dev/shm/__var0/var1/backup/mysql/proc.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_leap_second.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone_leap_second.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_leap_second.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone_leap_second.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_leap_second.frm to /dev/shm/__var0/var1/backup/mysql/time_zone_leap_second.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition_type.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone_transition_type.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition_type.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone_transition_type.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition_type.frm to /dev/shm/__var0/var1/backup/mysql/time_zone_transition_type.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone_transition.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone_transition.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_transition.frm to /dev/shm/__var0/var1/backup/mysql/time_zone_transition.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone.frm to /dev/shm/__var0/var1/backup/mysql/time_zone.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_name.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone_name.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_name.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone_name.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/time_zone_name.frm to /dev/shm/__var0/var1/backup/mysql/time_zone_name.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_keyword.MYD to /dev/shm/__var0/var1/backup/mysql/help_keyword.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_keyword.MYI to /dev/shm/__var0/var1/backup/mysql/help_keyword.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_keyword.frm to /dev/shm/__var0/var1/backup/mysql/help_keyword.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_relation.MYD to /dev/shm/__var0/var1/backup/mysql/help_relation.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_relation.MYI to /dev/shm/__var0/var1/backup/mysql/help_relation.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_relation.frm to /dev/shm/__var0/var1/backup/mysql/help_relation.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_category.MYD to /dev/shm/__var0/var1/backup/mysql/help_category.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_category.MYI to /dev/shm/__var0/var1/backup/mysql/help_category.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_category.frm to /dev/shm/__var0/var1/backup/mysql/help_category.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_topic.MYD to /dev/shm/__var0/var1/backup/mysql/help_topic.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_topic.MYI to /dev/shm/__var0/var1/backup/mysql/help_topic.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/help_topic.frm to /dev/shm/__var0/var1/backup/mysql/help_topic.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/columns_priv.MYD to /dev/shm/__var0/var1/backup/mysql/columns_priv.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/columns_priv.MYI to /dev/shm/__var0/var1/backup/mysql/columns_priv.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/columns_priv.frm to /dev/shm/__var0/var1/backup/mysql/columns_priv.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/tables_priv.MYD to /dev/shm/__var0/var1/backup/mysql/tables_priv.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/tables_priv.MYI to /dev/shm/__var0/var1/backup/mysql/tables_priv.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/tables_priv.frm to /dev/shm/__var0/var1/backup/mysql/tables_priv.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/servers.MYD to /dev/shm/__var0/var1/backup/mysql/servers.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/servers.MYI to /dev/shm/__var0/var1/backup/mysql/servers.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/servers.frm to /dev/shm/__var0/var1/backup/mysql/servers.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/plugin.MYD to /dev/shm/__var0/var1/backup/mysql/plugin.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/plugin.MYI to /dev/shm/__var0/var1/backup/mysql/plugin.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/plugin.frm to /dev/shm/__var0/var1/backup/mysql/plugin.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/func.MYD to /dev/shm/__var0/var1/backup/mysql/func.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/func.MYI to /dev/shm/__var0/var1/backup/mysql/func.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/func.frm to /dev/shm/__var0/var1/backup/mysql/func.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/roles_mapping.MYD to /dev/shm/__var0/var1/backup/mysql/roles_mapping.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/roles_mapping.MYI to /dev/shm/__var0/var1/backup/mysql/roles_mapping.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/roles_mapping.frm to /dev/shm/__var0/var1/backup/mysql/roles_mapping.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/user.MYD to /dev/shm/__var0/var1/backup/mysql/user.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/user.MYI to /dev/shm/__var0/var1/backup/mysql/user.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/user.frm to /dev/shm/__var0/var1/backup/mysql/user.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/host.MYD to /dev/shm/__var0/var1/backup/mysql/host.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/host.MYI to /dev/shm/__var0/var1/backup/mysql/host.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/host.frm to /dev/shm/__var0/var1/backup/mysql/host.frm 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/db.MYD to /dev/shm/__var0/var1/backup/mysql/db.MYD 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/db.MYI to /dev/shm/__var0/var1/backup/mysql/db.MYI 170507 09:52:09 [01] ...done 170507 09:52:09 [01] Copying ./mysql/db.frm to /dev/shm/__var0/var1/backup/mysql/db.frm 170507 09:52:09 [01] ...done 170507 09:52:09 Finished backing up non-InnoDB tables and files 170507 09:52:09 [00] Writing xtrabackup_binlog_info 170507 09:52:09 [00] ...done 170507 09:52:09 Executing FLUSH NO_WRITE_TO_BINLOG ENGINE LOGS... xtrabackup: The latest check point (for incremental): '1770243' xtrabackup: Stopping log copying thread. .170507 09:52:09 >> log scanned up to (1770343) 170507 09:52:09 Executing UNLOCK TABLES 170507 09:52:09 All tables unlocked 170507 09:52:09 [00] Copying ib_buffer_pool to /dev/shm/__var0/var1/backup/ib_buffer_pool 170507 09:52:09 [00] ...done 170507 09:52:09 Backup created in directory '/dev/shm/__var0/var1/backup/' MySQL binlog position: filename 'mysql-bin.000002', position '44041', GTID of the last change '0-1-54' 170507 09:52:09 [00] Writing backup-my.cnf 170507 09:52:09 [00] ...done 170507 09:52:09 [00] Writing xtrabackup_info 170507 09:52:09 [00] ...done xtrabackup: Transaction log of lsn (1620117) to (1770343) was copied. 170507 09:52:09 completed OK! 2017-05-07 09:52:09: bash: Killing server with id=1... inc/common.sh: line 419: 4660 Killed ${MYSQLD} ${MYSQLD_ARGS} $* 2017-05-07 09:52:09: bash: Applying log 2017-05-07 09:52:09: bash: ===> /usr/bin/innobackupex --defaults-file=/dev/shm/__var0/var1/my.cnf --no-version-check --apply-log /dev/shm/__var0/var1/backup 170507 09:52:09 innobackupex: Starting the apply-log operation IMPORTANT: Please check that the apply-log run completes successfully. At the end of a successful apply-log run innobackupex prints "completed OK!". /usr/bin/innobackupex version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799) xtrabackup: cd to /dev/shm/__var0/var1/backup/ xtrabackup: This target seems to be not prepared yet. InnoDB: Number of pools: 1 xtrabackup: xtrabackup_logfile detected: size=8388608, start_lsn=(1620117) xtrabackup: using the following InnoDB configuration for recovery: xtrabackup: innodb_data_home_dir = . xtrabackup: innodb_data_file_path = ibdata1:12M:autoextend xtrabackup: innodb_log_group_home_dir = . xtrabackup: innodb_log_files_in_group = 1 xtrabackup: innodb_log_file_size = 8388608 xtrabackup: using the following InnoDB configuration for recovery: xtrabackup: innodb_data_home_dir = . xtrabackup: innodb_data_file_path = ibdata1:12M:autoextend xtrabackup: innodb_log_group_home_dir = . xtrabackup: innodb_log_files_in_group = 1 xtrabackup: innodb_log_file_size = 8388608 xtrabackup: Starting InnoDB instance for recovery. xtrabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter) InnoDB: PUNCH HOLE support available InnoDB: Mutexes and rw_locks use GCC atomic builtins InnoDB: Uses event mutexes InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier InnoDB: Compressed tables use zlib 1.2.8 InnoDB: Number of pools: 1 InnoDB: Using CPU crc32 instructions InnoDB: Initializing buffer pool, total size = 100M, instances = 1, chunk size = 100M InnoDB: Completed initialization of buffer pool InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). InnoDB: Highest supported file format is Barracuda. InnoDB: Log scan progressed past the checkpoint lsn 1620117 InnoDB: Doing recovery: scanned up to log sequence number 1770343 (2%) InnoDB: Plugin initialization aborted with error Generic error 07:52:10 UTC - xtrabackup got signal 11 ; This could be because you hit a bug or data is corrupted. This error can also be caused by malfunctioning hardware. Attempting to collect some information that could help diagnose the problem. As this is a crash and something is definitely wrong, the information collection process might fail. Thread pointer: 0x0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0 thread_stack 0x10000 /usr/bin/innobackupex(my_print_stacktrace+0x3b)[0x55cc1d72552b] /usr/bin/innobackupex(handle_fatal_signal+0x291)[0x55cc1d599711] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11630)[0x7f5d86edc630] /usr/bin/innobackupex(_Z19rw_lock_x_lock_funcP9rw_lock_tmPKcm+0x6a)[0x55cc1d0f703a] /usr/bin/innobackupex(_Z15trx_purge_statev+0x2a)[0x55cc1d0fc75a] /usr/bin/innobackupex(_Z26srv_get_active_thread_typev+0xdb)[0x55cc1d0dd8cb] /usr/bin/innobackupex(buf_flush_page_cleaner_coordinator+0x1dfc)[0x55cc1cf1889c] /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ca)[0x7f5d86ed26ca] /lib/x86_64-linux-gnu/libc.so.6(clone+0x5f)[0x7f5d84f480af] Please report a bug at https://bugs.launchpad.net/percona-xtrabackup 2017-05-07 09:52:10: bash: ===> innobackupex failed with exit code 2 {noformat} |
EDIT: please see test case at https://jira.mariadb.org/browse/MDEV-12720?focusedCommentId=94886&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-94886 |
Assignee | Jan Lindström [ jplindst ] | Marko Mäkelä [ marko ] |
Link |
This issue is caused by |
Status | Open [ 1 ] | Confirmed [ 10101 ] |
Status | Confirmed [ 10101 ] | In Progress [ 3 ] |
issue.field.resolutiondate | 2017-05-09 08:45:07.0 | 2017-05-09 08:45:07.745 |
Fix Version/s | 10.2.6 [ 22527 ] | |
Fix Version/s | 10.2 [ 14601 ] | |
Resolution | Fixed [ 1 ] | |
Status | In Progress [ 3 ] | Closed [ 6 ] |
Affects Version/s | 10.2.6 [ 22527 ] | |
Affects Version/s | 10.2 [ 14601 ] |
Link |
This issue causes |
Workflow | MariaDB v3 [ 80643 ] | MariaDB v4 [ 152105 ] |
Optional steps to reproduce the same with environs framework:
Setup environment:
# getting required plugin
# getting and building 10.2 tree
# installing xtrabackup 2.4.7
# better to install because .tar distribution typically have dependency problems
Run the test with command below, observe Xtrabackup crash:
./runsuite.sh m1 x1 _plugin/xtrabackup/t/
MDEV-12720.shObserve that no crash happens when line "# sleep 2" is uncommented in _plugin/xtrabackup/t/
MDEV-12720.sh (increase sleep amount otherwise).Optional - install tar 10.1.23 , 10.2.5 and 5.7.18 and observe no crash with those (undo changes to
MDEV-12720.sh if any):# get required oracle-mysql plugin