[#MDEV-12720] recovery fails with "Generic error" for ROW

Comment by Andrii Nikitin (Inactive) [ 2017-05-07 ]

Optional steps to reproduce the same with environs framework:

Setup environment:

git clone https://github.com/AndriiNikitin/mariadb-environs

cd mariadb-environs

# getting required plugin

git clone https://github.com/AndriiNikitin/mariadb-environs-xtrabackup _plugin/xtrabackup

# getting and building 10.2 tree

./replant.sh m1-10.2

m1-10.2/checkout.sh

m1-10.2/make.sh

m1-10.2/build.sh

# installing xtrabackup 2.4.7

# better to install because .tar distribution typically have dependency problems

./replant.sh x1-system

sudo x1-system/install.sh 2.4.7

Run the test with command below, observe Xtrabackup crash:
./runsuite.sh m1 x1 _plugin/xtrabackup/t/~~MDEV-12720~~.sh

Observe that no crash happens when line "# sleep 2" is uncommented in _plugin/xtrabackup/t/~~MDEV-12720~~.sh (increase sleep amount otherwise).

Optional - install tar 10.1.23 , 10.2.5 and 5.7.18 and observe no crash with those (undo changes to ~~MDEV-12720~~.sh if any):

./replant.sh m2-10.1.23

m2*/download.sh

./runsuite.sh m2 x1 _plugin/xtrabackup/t/MDEV-12720.sh

./replant.sh m3-10.2.5

m3*/download.sh

./runsuite.sh m3 x1 _plugin/xtrabackup/t/MDEV-12720.sh

# get required oracle-mysql plugin

git clone https://github.com/AndriiNikitin/mariadb-environs-oracle-mysql _plugin/oracle-mysql

./replant.sh o1-5.7.18

o1*/download.sh

./runsuite.sh o1 x1 _plugin/xtrabackup/t/MDEV-12720.sh

Comment by Andrii Nikitin (Inactive) [ 2017-05-08 ]

Alternatively - test with the latest 10.2 build instead of source tree:

git clone https://github.com/AndriiNikitin/mariadb-environs-hasky _plugin/hasky

./replant.sh m5-10.2~latest

m5*/download.sh

./runsuite.sh m5 x1 _plugin/xtrabackup/t/MDEV-12720.sh

a@UBINTI:~/test1/mariadb-environs$ git clone https://github.com/AndriiNikitin/mariadb-environs-hasky _plugin/hasky

Cloning into '_plugin/hasky'...

remote: Counting objects: 30, done.

remote: Compressing objects: 100% (22/22), done.

remote: Total 30 (delta 9), reused 16 (delta 4), pack-reused 0

Unpacking objects: 100% (30/30), done.

Checking connectivity... done.

a@UBINTI:~/test1/mariadb-environs$ ./replant.sh m5-10.2~latest

a@UBINTI:~/test1/mariadb-environs$ m5-10.2~latest/download.sh

downloading http://hasky.askmonty.org/archive/10.2/build-14294/kvm-bintar-quantal-amd64/mariadb-*-linux-x86_64.tar.gz

a@UBINTI:~/test1/mariadb-environs$ ./runsuite.sh m5 x1 _plugin/xtrabackup/t/MDEV-12720.sh

Checking prerequisites...

starting test _plugin/xtrabackup/t/MDEV-12720.sh

/usr/bin/xtrabackup

/usr/bin/innobackupex

/usr/bin/mysql

Product=10.2.6

Product=version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799)

OS="Ubuntu 16.10"

configure_rest_encryption.sh=NA

MYSQL_INSTALL_DB set to /home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/scripts/mysql_install_db

MYSQLD set to /home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/bin/mysqld

MYSQL set to /home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/bin/mysql

MYSQLADMIN set to /home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/bin/mysqladmin

MYSQLDUMP set to /home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/bin/mysqldump

attempting to clean previous mysqld instances

ls: cannot access '/dev/shm/__var0/mysqld*.pid': No such file or directory

STARTING TEST

2017-05-08 07:52:06: bash: Starting server with id=1...

2017-05-08 07:52:06: bash: Creating server root directory: /dev/shm/__var0/var1

2017-05-08 07:52:06: bash: Creating server temporary directory: /dev/shm/__var0/var1/tmp

2017-05-08 07:52:06: bash: Creating server data directory: /dev/shm/__var0/var1/data

2017-05-08 07:52:06: bash: Calling mysql_install_db

Installing MariaDB/MySQL system tables in '/dev/shm/__var0/var1/data' ...

OK

To start mysqld at boot time you have to copy

support-files/mysql.server to the right place for your system

PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !

To do so, start the server, then issue the following commands:

'/home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/bin/mysqladmin' -u root password 'new-password'

'/home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/bin/mysqladmin' -u root -h UBINTI password 'new-password'

Alternatively you can run:

'/home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/bin/mysql_secure_installation'

which will also give you the option of removing the test

databases and anonymous user created by default.  This is

strongly recommended for production servers.

See the MariaDB Knowledgebase at http://mariadb.com/kb or the

MySQL manual for more instructions.

You can start the MariaDB daemon with:

cd '/home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294' ; /home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/bin/mysqld_safe --datadir='/dev/shm/__var0/var1/data'

You can test the MariaDB daemon with mysql-test-run.pl

cd '/home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/mysql-test' ; perl mysql-test-run.pl

Please report any problems at http://mariadb.org/jira

The latest information about MariaDB is available at http://mariadb.org/.

You can find additional information about the MySQL part at:

http://dev.mysql.com

Consider joining MariaDB's strong and vibrant community:

https://mariadb.org/get-involved/

Starting /home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/bin/mysqld --defaults-file=/dev/shm/__var0/var1/my.cnf

2017-05-08  7:52:08 140034108753408 [Note] /home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/bin/mysqld (mysqld 10.2.6-MariaDB-log) starting as process 6501 ...

2017-05-08  7:52:08 140034108753408 [Warning] Changed limits: max_open_files: 1024  max_connections: 151  table_cache: 431

2017-05-08 07:52:09: bash: Made 1 attempts to connect to server

2017-05-08 07:52:09: bash: Server with id=1 has been started on port 16502, socket /dev/shm/__var0/tmp/mysql.sock.DqFFUd

2017-05-08 07:52:09: bash: ===> /home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/bin/mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e create table test.a(i int primary key auto_increment, s varchar(255) ) ENGINE=InnoDB ROW_FORMAT=compressed

2017-05-08 07:52:09: bash: ===> /home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/bin/mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e insert into test.a(i) select null

2017-05-08 07:52:09: bash: ===> /home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/bin/mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e insert into test.a select null, uuid() from test.a a, test.a b, test.a c

2017-05-08 07:52:09: bash: ===> /home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/bin/mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e insert into test.a select null, uuid() from test.a a, test.a b, test.a c

2017-05-08 07:52:09: bash: ===> /home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/bin/mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e insert into test.a select null, uuid() from test.a a, test.a b, test.a c

2017-05-08 07:52:09: bash: ===> /home/a/test1/mariadb-environs/m5-10.2~latest/../_depot/m-tar/10.2-14294/bin/mysql --defaults-file=/dev/shm/__var0/var1/my.cnf -e select count(*) from test.a

count(*)

2017-05-08 07:52:09: bash: ===> /usr/bin/innobackupex --defaults-file=/dev/shm/__var0/var1/my.cnf --no-version-check --no-timestamp /dev/shm/__var0/var1/backup

170508 07:52:09 innobackupex: Starting the backup operation

IMPORTANT: Please check that the backup run completes successfully.

           At the end of a successful backup run innobackupex

           prints "completed OK!".

170508 07:52:09 Connecting to MySQL server host: localhost, user: root, password: not set, port: not set, socket: /dev/shm/__var0/tmp/mysql.sock.DqFFUd

Using server version 10.2.6-MariaDB-log

/usr/bin/innobackupex version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799)

xtrabackup: uses posix_fadvise().

xtrabackup: cd to /dev/shm/__var0/var1/data

xtrabackup: open files limit requested 0, set to 1024

xtrabackup: using the following InnoDB configuration:

xtrabackup:   innodb_data_home_dir = .

xtrabackup:   innodb_data_file_path = ibdata1:12M:autoextend

xtrabackup:   innodb_log_group_home_dir = ./

xtrabackup:   innodb_log_files_in_group = 2

xtrabackup:   innodb_log_file_size = 50331648

InnoDB: Number of pools: 1

170508 07:52:09 >> log scanned up to (1770195)

xtrabackup: Generating a list of tablespaces

InnoDB: Allocated tablespace ID 4 for test/a, old maximum was 0

170508 07:52:10 [01] Copying ./ibdata1 to /dev/shm/__var0/var1/backup/ibdata1

170508 07:52:10 [01]        ...done

[01] ./test/a.ibd is compressed with page size = 8192 bytes

170508 07:52:10 [01] Copying ./test/a.ibd to /dev/shm/__var0/var1/backup/test/a.ibd

170508 07:52:10 [01]        ...done

170508 07:52:10 [01] Copying ./mysql/gtid_slave_pos.ibd to /dev/shm/__var0/var1/backup/mysql/gtid_slave_pos.ibd

170508 07:52:10 [01]        ...done

170508 07:52:10 [01] Copying ./mysql/innodb_index_stats.ibd to /dev/shm/__var0/var1/backup/mysql/innodb_index_stats.ibd

170508 07:52:10 [01]        ...done

170508 07:52:10 [01] Copying ./mysql/innodb_table_stats.ibd to /dev/shm/__var0/var1/backup/mysql/innodb_table_stats.ibd

170508 07:52:10 [01]        ...done

170508 07:52:10 >> log scanned up to (1770295)

170508 07:52:11 Executing FLUSH NO_WRITE_TO_BINLOG TABLES...

170508 07:52:11 Executing FLUSH TABLES WITH READ LOCK...

170508 07:52:11 Starting to backup non-InnoDB tables and files

170508 07:52:11 [01] Copying ./performance_schema/db.opt to /dev/shm/__var0/var1/backup/performance_schema/db.opt

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./test/a.frm to /dev/shm/__var0/var1/backup/test/a.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/gtid_slave_pos.frm to /dev/shm/__var0/var1/backup/mysql/gtid_slave_pos.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/index_stats.MYD to /dev/shm/__var0/var1/backup/mysql/index_stats.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/index_stats.MYI to /dev/shm/__var0/var1/backup/mysql/index_stats.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/index_stats.frm to /dev/shm/__var0/var1/backup/mysql/index_stats.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/column_stats.MYD to /dev/shm/__var0/var1/backup/mysql/column_stats.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/column_stats.MYI to /dev/shm/__var0/var1/backup/mysql/column_stats.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/column_stats.frm to /dev/shm/__var0/var1/backup/mysql/column_stats.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/table_stats.MYD to /dev/shm/__var0/var1/backup/mysql/table_stats.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/table_stats.MYI to /dev/shm/__var0/var1/backup/mysql/table_stats.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/table_stats.frm to /dev/shm/__var0/var1/backup/mysql/table_stats.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/proxies_priv.MYD to /dev/shm/__var0/var1/backup/mysql/proxies_priv.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/proxies_priv.MYI to /dev/shm/__var0/var1/backup/mysql/proxies_priv.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/proxies_priv.frm to /dev/shm/__var0/var1/backup/mysql/proxies_priv.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/innodb_index_stats.frm to /dev/shm/__var0/var1/backup/mysql/innodb_index_stats.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/innodb_table_stats.frm to /dev/shm/__var0/var1/backup/mysql/innodb_table_stats.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/event.MYD to /dev/shm/__var0/var1/backup/mysql/event.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/event.MYI to /dev/shm/__var0/var1/backup/mysql/event.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/event.frm to /dev/shm/__var0/var1/backup/mysql/event.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/slow_log.CSV to /dev/shm/__var0/var1/backup/mysql/slow_log.CSV

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/slow_log.CSM to /dev/shm/__var0/var1/backup/mysql/slow_log.CSM

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/slow_log.frm to /dev/shm/__var0/var1/backup/mysql/slow_log.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/general_log.CSV to /dev/shm/__var0/var1/backup/mysql/general_log.CSV

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/general_log.CSM to /dev/shm/__var0/var1/backup/mysql/general_log.CSM

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/general_log.frm to /dev/shm/__var0/var1/backup/mysql/general_log.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/procs_priv.MYD to /dev/shm/__var0/var1/backup/mysql/procs_priv.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/procs_priv.MYI to /dev/shm/__var0/var1/backup/mysql/procs_priv.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/procs_priv.frm to /dev/shm/__var0/var1/backup/mysql/procs_priv.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/proc.MYD to /dev/shm/__var0/var1/backup/mysql/proc.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/proc.MYI to /dev/shm/__var0/var1/backup/mysql/proc.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/proc.frm to /dev/shm/__var0/var1/backup/mysql/proc.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/time_zone_leap_second.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone_leap_second.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/time_zone_leap_second.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone_leap_second.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/time_zone_leap_second.frm to /dev/shm/__var0/var1/backup/mysql/time_zone_leap_second.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/time_zone_transition_type.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone_transition_type.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/time_zone_transition_type.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone_transition_type.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/time_zone_transition_type.frm to /dev/shm/__var0/var1/backup/mysql/time_zone_transition_type.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/time_zone_transition.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone_transition.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/time_zone_transition.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone_transition.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/time_zone_transition.frm to /dev/shm/__var0/var1/backup/mysql/time_zone_transition.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/time_zone.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/time_zone.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/time_zone.frm to /dev/shm/__var0/var1/backup/mysql/time_zone.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/time_zone_name.MYD to /dev/shm/__var0/var1/backup/mysql/time_zone_name.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/time_zone_name.MYI to /dev/shm/__var0/var1/backup/mysql/time_zone_name.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/time_zone_name.frm to /dev/shm/__var0/var1/backup/mysql/time_zone_name.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/help_keyword.MYD to /dev/shm/__var0/var1/backup/mysql/help_keyword.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/help_keyword.MYI to /dev/shm/__var0/var1/backup/mysql/help_keyword.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/help_keyword.frm to /dev/shm/__var0/var1/backup/mysql/help_keyword.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/help_relation.MYD to /dev/shm/__var0/var1/backup/mysql/help_relation.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/help_relation.MYI to /dev/shm/__var0/var1/backup/mysql/help_relation.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/help_relation.frm to /dev/shm/__var0/var1/backup/mysql/help_relation.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/help_category.MYD to /dev/shm/__var0/var1/backup/mysql/help_category.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/help_category.MYI to /dev/shm/__var0/var1/backup/mysql/help_category.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/help_category.frm to /dev/shm/__var0/var1/backup/mysql/help_category.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/help_topic.MYD to /dev/shm/__var0/var1/backup/mysql/help_topic.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/help_topic.MYI to /dev/shm/__var0/var1/backup/mysql/help_topic.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/help_topic.frm to /dev/shm/__var0/var1/backup/mysql/help_topic.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/columns_priv.MYD to /dev/shm/__var0/var1/backup/mysql/columns_priv.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/columns_priv.MYI to /dev/shm/__var0/var1/backup/mysql/columns_priv.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/columns_priv.frm to /dev/shm/__var0/var1/backup/mysql/columns_priv.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/tables_priv.MYD to /dev/shm/__var0/var1/backup/mysql/tables_priv.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/tables_priv.MYI to /dev/shm/__var0/var1/backup/mysql/tables_priv.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/tables_priv.frm to /dev/shm/__var0/var1/backup/mysql/tables_priv.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/servers.MYD to /dev/shm/__var0/var1/backup/mysql/servers.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/servers.MYI to /dev/shm/__var0/var1/backup/mysql/servers.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/servers.frm to /dev/shm/__var0/var1/backup/mysql/servers.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/plugin.MYD to /dev/shm/__var0/var1/backup/mysql/plugin.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/plugin.MYI to /dev/shm/__var0/var1/backup/mysql/plugin.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/plugin.frm to /dev/shm/__var0/var1/backup/mysql/plugin.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/func.MYD to /dev/shm/__var0/var1/backup/mysql/func.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/func.MYI to /dev/shm/__var0/var1/backup/mysql/func.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/func.frm to /dev/shm/__var0/var1/backup/mysql/func.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/roles_mapping.MYD to /dev/shm/__var0/var1/backup/mysql/roles_mapping.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/roles_mapping.MYI to /dev/shm/__var0/var1/backup/mysql/roles_mapping.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/roles_mapping.frm to /dev/shm/__var0/var1/backup/mysql/roles_mapping.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/user.MYD to /dev/shm/__var0/var1/backup/mysql/user.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/user.MYI to /dev/shm/__var0/var1/backup/mysql/user.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/user.frm to /dev/shm/__var0/var1/backup/mysql/user.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/host.MYD to /dev/shm/__var0/var1/backup/mysql/host.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/host.MYI to /dev/shm/__var0/var1/backup/mysql/host.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/host.frm to /dev/shm/__var0/var1/backup/mysql/host.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/db.MYD to /dev/shm/__var0/var1/backup/mysql/db.MYD

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/db.MYI to /dev/shm/__var0/var1/backup/mysql/db.MYI

170508 07:52:11 [01]        ...done

170508 07:52:11 [01] Copying ./mysql/db.frm to /dev/shm/__var0/var1/backup/mysql/db.frm

170508 07:52:11 [01]        ...done

170508 07:52:11 Finished backing up non-InnoDB tables and files

170508 07:52:11 [00] Writing xtrabackup_binlog_info

170508 07:52:11 [00]        ...done

170508 07:52:11 Executing FLUSH NO_WRITE_TO_BINLOG ENGINE LOGS...

xtrabackup: The latest check point (for incremental): '1770195'

xtrabackup: Stopping log copying thread.

.170508 07:52:11 >> log scanned up to (1770295)

170508 07:52:11 Executing UNLOCK TABLES

170508 07:52:11 All tables unlocked

170508 07:52:11 [00] Copying ib_buffer_pool to /dev/shm/__var0/var1/backup/ib_buffer_pool

170508 07:52:11 [00]        ...done

170508 07:52:11 Backup created in directory '/dev/shm/__var0/var1/backup/'

MySQL binlog position: filename 'mysql-bin.000002', position '44041', GTID of the last change '0-1-54'

170508 07:52:11 [00] Writing backup-my.cnf

170508 07:52:11 [00]        ...done

170508 07:52:11 [00] Writing xtrabackup_info

170508 07:52:11 [00]        ...done

xtrabackup: Transaction log of lsn (1620117) to (1770295) was copied.

170508 07:52:11 completed OK!

2017-05-08 07:52:11: bash: Killing server with id=1...

inc/common.sh: line 419:  6501 Killed                  ${MYSQLD} ${MYSQLD_ARGS} $*

2017-05-08 07:52:11: bash: Applying log

2017-05-08 07:52:11: bash: ===> /usr/bin/innobackupex --defaults-file=/dev/shm/__var0/var1/my.cnf --no-version-check --apply-log /dev/shm/__var0/var1/backup

170508 07:52:11 innobackupex: Starting the apply-log operation

IMPORTANT: Please check that the apply-log run completes successfully.

           At the end of a successful apply-log run innobackupex

           prints "completed OK!".

/usr/bin/innobackupex version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799)

xtrabackup: cd to /dev/shm/__var0/var1/backup/

xtrabackup: This target seems to be not prepared yet.

InnoDB: Number of pools: 1

xtrabackup: xtrabackup_logfile detected: size=8388608, start_lsn=(1620117)

xtrabackup: using the following InnoDB configuration for recovery:

xtrabackup:   innodb_data_home_dir = .

xtrabackup:   innodb_data_file_path = ibdata1:12M:autoextend

xtrabackup:   innodb_log_group_home_dir = .

xtrabackup:   innodb_log_files_in_group = 1

xtrabackup:   innodb_log_file_size = 8388608

xtrabackup: using the following InnoDB configuration for recovery:

xtrabackup:   innodb_data_home_dir = .

xtrabackup:   innodb_data_file_path = ibdata1:12M:autoextend

xtrabackup:   innodb_log_group_home_dir = .

xtrabackup:   innodb_log_files_in_group = 1

xtrabackup:   innodb_log_file_size = 8388608

xtrabackup: Starting InnoDB instance for recovery.

xtrabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter)

InnoDB: PUNCH HOLE support available

InnoDB: Mutexes and rw_locks use GCC atomic builtins

InnoDB: Uses event mutexes

InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier

InnoDB: Compressed tables use zlib 1.2.8

InnoDB: Number of pools: 1

InnoDB: Using CPU crc32 instructions

InnoDB: Initializing buffer pool, total size = 100M, instances = 1, chunk size = 100M

InnoDB: Completed initialization of buffer pool

InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().

InnoDB: Highest supported file format is Barracuda.

InnoDB: Log scan progressed past the checkpoint lsn 1620117

InnoDB: Doing recovery: scanned up to log sequence number 1770295 (2%)

InnoDB: Plugin initialization aborted with error Generic error

05:52:11 UTC - xtrabackup got signal 11 ;

This could be because you hit a bug or data is corrupted.

This error can also be caused by malfunctioning hardware.

Attempting to collect some information that could help diagnose the problem.

As this is a crash and something is definitely wrong, the information

collection process might fail.

Thread pointer: 0x0

Attempting backtrace. You can use the following information to find out

where mysqld died. If you see no messages after this, something went

terribly wrong...

stack_bottom = 0 thread_stack 0x10000

/usr/bin/innobackupex(my_print_stacktrace+0x3b)[0x55e91d7ee52b]

/usr/bin/innobackupex(handle_fatal_signal+0x291)[0x55e91d662711]

/lib/x86_64-linux-gnu/libpthread.so.0(+0x11630)[0x7f5d3964d630]

/usr/bin/innobackupex(_Z19rw_lock_x_lock_funcP9rw_lock_tmPKcm+0x6a)[0x55e91d1c003a]

/usr/bin/innobackupex(_Z15trx_purge_statev+0x2a)[0x55e91d1c575a]

/usr/bin/innobackupex(_Z26srv_get_active_thread_typev+0xdb)[0x55e91d1a68cb]

/usr/bin/innobackupex(buf_flush_page_cleaner_coordinator+0x1dfc)[0x55e91cfe189c]

/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ca)[0x7f5d396436ca]

/lib/x86_64-linux-gnu/libc.so.6(clone+0x5f)[0x7f5d376b90af]

Please report a bug at https://bugs.launchpad.net/percona-xtrabackup

2017-05-08 07:52:11: bash: ===> innobackupex failed with exit code 2

[MDEV-12720] (5 sec) (res 1) FAIL

Comment by Andrii Nikitin (Inactive) [ 2017-05-08 ]

gdb with "thr a a bt full"

GNU gdb (Ubuntu 7.11.90.20161005-0ubuntu2) 7.11.90.20161005-git

Copyright (C) 2016 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.  Type "show copying"

and "show warranty" for details.

This GDB was configured as "x86_64-linux-gnu".

Type "show configuration" for configuration details.

For bug reporting instructions, please see:

<http://www.gnu.org/software/gdb/bugs/>.

Find the GDB manual and other documentation resources online at:

<http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".

Type "apropos word" to search for commands related to "word"...

Reading symbols from innobackupex...Reading symbols from /usr/lib/debug//usr/bin/xtrabackup...done.

done.

(gdb) run

Starting program: /usr/bin/innobackupex --apply-log /dev/shm/__var0/var1/backup

[Thread debugging using libthread_db enabled]

Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

170508 12:18:13 innobackupex: Starting the apply-log operation

IMPORTANT: Please check that the apply-log run completes successfully.

           At the end of a successful apply-log run innobackupex

           prints "completed OK!".

/usr/bin/innobackupex version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 6f7a799)

xtrabackup: cd to /dev/shm/__var0/var1/backup/

xtrabackup: This target seems to be not prepared yet.

InnoDB: Number of pools: 1

xtrabackup: xtrabackup_logfile detected: size=8388608, start_lsn=(1620117)

xtrabackup: using the following InnoDB configuration for recovery:

xtrabackup:   innodb_data_home_dir = .

xtrabackup:   innodb_data_file_path = ibdata1:12M:autoextend

xtrabackup:   innodb_log_group_home_dir = .

xtrabackup:   innodb_log_files_in_group = 1

xtrabackup:   innodb_log_file_size = 8388608

xtrabackup: using the following InnoDB configuration for recovery:

xtrabackup:   innodb_data_home_dir = .

xtrabackup:   innodb_data_file_path = ibdata1:12M:autoextend

xtrabackup:   innodb_log_group_home_dir = .

xtrabackup:   innodb_log_files_in_group = 1

xtrabackup:   innodb_log_file_size = 8388608

xtrabackup: Starting InnoDB instance for recovery.

xtrabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter)

InnoDB: PUNCH HOLE support available

InnoDB: Mutexes and rw_locks use GCC atomic builtins

InnoDB: Uses event mutexes

InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier

InnoDB: Compressed tables use zlib 1.2.8

InnoDB: Number of pools: 1

InnoDB: Using CPU crc32 instructions

InnoDB: Initializing buffer pool, total size = 100M, instances = 1, chunk size = 100M

InnoDB: Completed initialization of buffer pool

[New Thread 0x7fffef0b0700 (LWP 20873)]

[New Thread 0x7fffe68af700 (LWP 20874)]

[New Thread 0x7fffee8af700 (LWP 20875)]

[New Thread 0x7fffee0ae700 (LWP 20876)]

[New Thread 0x7fffed8ad700 (LWP 20877)]

[New Thread 0x7fffed0ac700 (LWP 20878)]

[New Thread 0x7fffec8ab700 (LWP 20879)]

[New Thread 0x7fffe7fff700 (LWP 20880)]

[New Thread 0x7fffe77fe700 (LWP 20881)]

[New Thread 0x7fffe60ae700 (LWP 20882)]

[New Thread 0x7fffe58ad700 (LWP 20883)]

InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().

InnoDB: Highest supported file format is Barracuda.

InnoDB: Log scan progressed past the checkpoint lsn 1620117

InnoDB: Doing recovery: scanned up to log sequence number 1770296 (2%)

InnoDB: Plugin initialization aborted with error Generic error

Thread 12 "innobackupex" received signal SIGSEGV, Segmentation fault.

[Switching to Thread 0x7fffe58ad700 (LWP 20883)]

rw_lock_lock_word_decr (threshold=268435456, amount=536870912, lock=0x10)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/include/sync0rw.ic:276

276	/mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/include/sync0rw.ic: No such file or directory.

(gdb) thr a a bt full

Thread 12 (Thread 0x7fffe58ad700 (LWP 20883)):

#0  rw_lock_lock_word_decr (threshold=268435456, amount=536870912, lock=0x10)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/include/sync0rw.ic:276

        local_lock_word = <optimized out>

#1  rw_lock_x_lock_low (line=1902,

    file_name=0x55555651f178 "/mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/trx/trx0purge.cc", pass=0, lock=0x10)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/sync/sync0rw.cc:554

No locals.

#2  rw_lock_x_lock_func (lock=0x10, pass=pass@entry=0,

    file_name=file_name@entry=0x55555651f178 "/mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/trx/trx0purge.cc", line=line@entry=1902)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/sync/sync0rw.cc:729

        i = 0

        spin_count = <optimized out>

        count_os_wait = <optimized out>

        cell = 0x555558135ff0

#3  0x0000555555d4375a in trx_purge_state ()

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/trx/trx0purge.cc:1902

        state = <optimized out>

#4  0x0000555555d248cb in srv_get_active_thread_type ()

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/srv/srv0srv.cc:1806

        ret = SRV_NONE

#5  0x0000555555b5f89c in buf_flush_page_cleaner_coordinator (arg=<optimized out>)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/buf/buf0flu.cc:3412

        next_loop_time = 1494238694058

        n_flushed = 0

        last_activity = 0

        last_pages = 0

        ret_sleep = <optimized out>

        n_evicted = 0

        n_flushed_last = 0

        warn_interval = 1

        warn_count = 0

        sig_count = 2

        success = <optimized out>

#6  0x00007ffff7bc16ca in start_thread (arg=0x7fffe58ad700) at pthread_create.c:333

        __res = <optimized out>

        pd = 0x7fffe58ad700

        now = <optimized out>

        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737044469504, 8604873729023514632, 0, 140737488282671, 140737044470208, 140737044469504,

                -8604852742180464632, -8604891898041886712}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0,

              canceltype = 0}}}

        not_first_call = <optimized out>

        pagesize_m1 = <optimized out>

        sp = <optimized out>

        freesize = <optimized out>

        __PRETTY_FUNCTION__ = "start_thread"

#7  0x00007ffff5c370af in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105

No locals.

Thread 11 (Thread 0x7fffe60ae700 (LWP 20882)):

#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185

No locals.

#1  0x0000555555c7acd6 in os_event::wait (this=0x555556ed7d70)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:165

        ret = <optimized out>

#2  os_event::wait_low (this=this@entry=0x555556ed7d70, reset_sig_count=1, reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:335

No locals.

#3  0x0000555555c7b13a in os_event_wait_low (event=event@entry=0x555556ed7d70, reset_sig_count=reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:534

No locals.

#4  0x0000555555c8b484 in os_aio_simulated_handler (type=0x7fffe60ade80, m2=0x7fffe60ade78, m1=0x7fffe60ade70, global_segment=9)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:7968

        n_slots = <optimized out>

        n_reserved = <optimized out>

        slot = <optimized out>

        array = 0x555556ed67e0

        segment = <optimized out>

        event = 0x555556ed7d70

        handler = {m_oldest = 0, m_n_elems = 0, m_lowest_offset = 18446744073709551615, m_array = 0x555556ed67e0, m_n_slots = 256, m_segment = 3,

          m_slots = std::vector of length 64, capacity 64 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, m_ptr = 0x0, m_buf = 0x0}

#5  os_aio_handler (segment=segment@entry=9, m1=m1@entry=0x7fffe60ade70, m2=m2@entry=0x7fffe60ade78, request=request@entry=0x7fffe60ade80)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:6268

        err = 4294966784

#6  0x0000555555ba5ce1 in fil_aio_wait (segment=segment@entry=9)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/fil/fil0fil.cc:6094

        node = 0x0

        type = {m_block_size = 512, m_type = 1, m_compression = {m_type = Compression::NONE}, m_encryption = {m_type = Encryption::NONE, m_key = 0x0, m_klen = 0,

            m_iv = 0x0, static master_key_id = 0, static uuid = '\000' <repeats 36 times>}}

        message = 0x0

        err = <optimized out>

#7  0x0000555555d2b7d8 in io_handler_thread (arg=<optimized out>)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/srv/srv0start.cc:309

        segment = 9

#8  0x00007ffff7bc16ca in start_thread (arg=0x7fffe60ae700) at pthread_create.c:333

        __res = <optimized out>

        pd = 0x7fffe60ae700

        now = <optimized out>

        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737052862208, 8604873729023514632, 0, 140737488282671, 140737052862912, 140737052862208,

                -8604853842228963320, -8604891898041886712}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0,

              canceltype = 0}}}

        not_first_call = <optimized out>

        pagesize_m1 = <optimized out>

        sp = <optimized out>

        freesize = <optimized out>

        __PRETTY_FUNCTION__ = "start_thread"

#9  0x00007ffff5c370af in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105

No locals.

Thread 10 (Thread 0x7fffe77fe700 (LWP 20881)):

#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185

No locals.

#1  0x0000555555c7acd6 in os_event::wait (this=0x555556ed7cf0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:165

        ret = <optimized out>

#2  os_event::wait_low (this=this@entry=0x555556ed7cf0, reset_sig_count=1, reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:335

No locals.

#3  0x0000555555c7b13a in os_event_wait_low (event=event@entry=0x555556ed7cf0, reset_sig_count=reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:534

No locals.

#4  0x0000555555c8b484 in os_aio_simulated_handler (type=0x7fffe77fde80, m2=0x7fffe77fde78, m1=0x7fffe77fde70, global_segment=8)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:7968

        n_slots = <optimized out>

        n_reserved = <optimized out>

        slot = <optimized out>

        array = 0x555556ed67e0

        segment = <optimized out>

        event = 0x555556ed7cf0

        handler = {m_oldest = 0, m_n_elems = 0, m_lowest_offset = 18446744073709551615, m_array = 0x555556ed67e0, m_n_slots = 256, m_segment = 2,

          m_slots = std::vector of length 64, capacity 64 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, m_ptr = 0x0, m_buf = 0x0}

#5  os_aio_handler (segment=segment@entry=8, m1=m1@entry=0x7fffe77fde70, m2=m2@entry=0x7fffe77fde78, request=request@entry=0x7fffe77fde80)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:6268

        err = 4294966784

#6  0x0000555555ba5ce1 in fil_aio_wait (segment=segment@entry=8)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/fil/fil0fil.cc:6094

        node = 0x0

        type = {m_block_size = 512, m_type = 1, m_compression = {m_type = Compression::NONE}, m_encryption = {m_type = Encryption::NONE, m_key = 0x0, m_klen = 0,

            m_iv = 0x0, static master_key_id = 0, static uuid = '\000' <repeats 36 times>}}

        message = 0x0

        err = <optimized out>

#7  0x0000555555d2b7d8 in io_handler_thread (arg=<optimized out>)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/srv/srv0start.cc:309

        segment = 8

#8  0x00007ffff7bc16ca in start_thread (arg=0x7fffe77fe700) at pthread_create.c:333

        __res = <optimized out>

        pd = 0x7fffe77fe700

        now = <optimized out>

        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737077307136, 8604873729023514632, 0, 140737488282671, 140737077307840, 140737077307136,

                -8604855036229871608, -8604891898041886712}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0,

              canceltype = 0}}}

        not_first_call = <optimized out>

        pagesize_m1 = <optimized out>

        sp = <optimized out>

        freesize = <optimized out>

        __PRETTY_FUNCTION__ = "start_thread"

#9  0x00007ffff5c370af in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105

No locals.

Thread 9 (Thread 0x7fffe7fff700 (LWP 20880)):

#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185

No locals.

#1  0x0000555555c7acd6 in os_event::wait (this=0x555556ed7c70)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:165

        ret = <optimized out>

#2  os_event::wait_low (this=this@entry=0x555556ed7c70, reset_sig_count=1, reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:335

No locals.

#3  0x0000555555c7b13a in os_event_wait_low (event=event@entry=0x555556ed7c70, reset_sig_count=reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:534

No locals.

#4  0x0000555555c8b484 in os_aio_simulated_handler (type=0x7fffe7ffee80, m2=0x7fffe7ffee78, m1=0x7fffe7ffee70, global_segment=7)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:7968

        n_slots = <optimized out>

        n_reserved = <optimized out>

        slot = <optimized out>

        array = 0x555556ed67e0

        segment = <optimized out>

        event = 0x555556ed7c70

        handler = {m_oldest = 0, m_n_elems = 0, m_lowest_offset = 18446744073709551615, m_array = 0x555556ed67e0, m_n_slots = 256, m_segment = 1,

          m_slots = std::vector of length 64, capacity 64 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, m_ptr = 0x0, m_buf = 0x0}

#5  os_aio_handler (segment=segment@entry=7, m1=m1@entry=0x7fffe7ffee70, m2=m2@entry=0x7fffe7ffee78, request=request@entry=0x7fffe7ffee80)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:6268

        err = 4294966784

#6  0x0000555555ba5ce1 in fil_aio_wait (segment=segment@entry=7)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/fil/fil0fil.cc:6094

        node = 0x0

        type = {m_block_size = 512, m_type = 1, m_compression = {m_type = Compression::NONE}, m_encryption = {m_type = Encryption::NONE, m_key = 0x0, m_klen = 0,

            m_iv = 0x0, static master_key_id = 0, static uuid = '\000' <repeats 36 times>}}

        message = 0x0

        err = <optimized out>

#7  0x0000555555d2b7d8 in io_handler_thread (arg=<optimized out>)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/srv/srv0start.cc:309

        segment = 7

#8  0x00007ffff7bc16ca in start_thread (arg=0x7fffe7fff700) at pthread_create.c:333

        __res = <optimized out>

        pd = 0x7fffe7fff700

        now = <optimized out>

        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737085699840, 8604873729023514632, 0, 140737488282671, 140737085700544, 140737085699840,

                -8604856136278370296, -8604891898041886712}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0,

              canceltype = 0}}}

        not_first_call = <optimized out>

        pagesize_m1 = <optimized out>

        sp = <optimized out>

        freesize = <optimized out>

        __PRETTY_FUNCTION__ = "start_thread"

#9  0x00007ffff5c370af in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105

No locals.

Thread 8 (Thread 0x7fffec8ab700 (LWP 20879)):

#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185

No locals.

#1  0x0000555555c7acd6 in os_event::wait (this=0x555556ed7bf0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:165

        ret = <optimized out>

#2  os_event::wait_low (this=this@entry=0x555556ed7bf0, reset_sig_count=1, reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:335

No locals.

#3  0x0000555555c7b13a in os_event_wait_low (event=event@entry=0x555556ed7bf0, reset_sig_count=reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:534

No locals.

#4  0x0000555555c8b484 in os_aio_simulated_handler (type=0x7fffec8aae80, m2=0x7fffec8aae78, m1=0x7fffec8aae70, global_segment=6)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:7968

        n_slots = <optimized out>

        n_reserved = <optimized out>

        slot = <optimized out>

        array = 0x555556ed67e0

        segment = <optimized out>

        event = 0x555556ed7bf0

        handler = {m_oldest = 0, m_n_elems = 0, m_lowest_offset = 18446744073709551615, m_array = 0x555556ed67e0, m_n_slots = 256, m_segment = 0,

          m_slots = std::vector of length 64, capacity 64 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, m_ptr = 0x0, m_buf = 0x0}

#5  os_aio_handler (segment=segment@entry=6, m1=m1@entry=0x7fffec8aae70, m2=m2@entry=0x7fffec8aae78, request=request@entry=0x7fffec8aae80)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:6268

        err = 4294966784

#6  0x0000555555ba5ce1 in fil_aio_wait (segment=segment@entry=6)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/fil/fil0fil.cc:6094

        node = 0x0

        type = {m_block_size = 512, m_type = 1, m_compression = {m_type = Compression::NONE}, m_encryption = {m_type = Encryption::NONE, m_key = 0x0, m_klen = 0,

            m_iv = 0x0, static master_key_id = 0, static uuid = '\000' <repeats 36 times>}}

        message = 0x0

        err = <optimized out>

#7  0x0000555555d2b7d8 in io_handler_thread (arg=<optimized out>)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/srv/srv0start.cc:309

        segment = 6

#8  0x00007ffff7bc16ca in start_thread (arg=0x7fffec8ab700) at pthread_create.c:333

        __res = <optimized out>

        pd = 0x7fffec8ab700

        now = <optimized out>

        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737161901824, 8604873729023514632, 0, 140737488282671, 140737161902528, 140737161901824,

                -8604832949897422840, -8604891898041886712}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0,

              canceltype = 0}}}

        not_first_call = <optimized out>

        pagesize_m1 = <optimized out>

        sp = <optimized out>

        freesize = <optimized out>

        __PRETTY_FUNCTION__ = "start_thread"

#9  0x00007ffff5c370af in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105

No locals.

Thread 7 (Thread 0x7fffed0ac700 (LWP 20878)):

#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185

No locals.

#1  0x0000555555c7acd6 in os_event::wait (this=0x555556ed7b70)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:165

        ret = <optimized out>

#2  os_event::wait_low (this=this@entry=0x555556ed7b70, reset_sig_count=1, reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:335

No locals.

#3  0x0000555555c7b13a in os_event_wait_low (event=event@entry=0x555556ed7b70, reset_sig_count=reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:534

No locals.

#4  0x0000555555c8b484 in os_aio_simulated_handler (type=0x7fffed0abe80, m2=0x7fffed0abe78, m1=0x7fffed0abe70, global_segment=5)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:7968

        n_slots = <optimized out>

        n_reserved = <optimized out>

        slot = <optimized out>

        array = 0x555556ed41d0

        segment = <optimized out>

        event = 0x555556ed7b70

        handler = {m_oldest = 0, m_n_elems = 0, m_lowest_offset = 18446744073709551615, m_array = 0x555556ed41d0, m_n_slots = 256, m_segment = 3,

          m_slots = std::vector of length 64, capacity 64 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, m_ptr = 0x0, m_buf = 0x0}

#5  os_aio_handler (segment=segment@entry=5, m1=m1@entry=0x7fffed0abe70, m2=m2@entry=0x7fffed0abe78, request=request@entry=0x7fffed0abe80)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:6268

        err = 4294966784

#6  0x0000555555ba5ce1 in fil_aio_wait (segment=segment@entry=5)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/fil/fil0fil.cc:6094

        node = 0x0

        type = {m_block_size = 512, m_type = 1, m_compression = {m_type = Compression::NONE}, m_encryption = {m_type = Encryption::NONE, m_key = 0x0, m_klen = 0,

            m_iv = 0x0, static master_key_id = 0, static uuid = '\000' <repeats 36 times>}}

        message = 0x0

        err = <optimized out>

#7  0x0000555555d2b7d8 in io_handler_thread (arg=<optimized out>)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/srv/srv0start.cc:309

        segment = 5

#8  0x00007ffff7bc16ca in start_thread (arg=0x7fffed0ac700) at pthread_create.c:333

        __res = <optimized out>

        pd = 0x7fffed0ac700

        now = <optimized out>

        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737170294528, 8604873729023514632, 0, 140737488282671, 140737170295232, 140737170294528,

                -8604834049945921528, -8604891898041886712}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0,

              canceltype = 0}}}

        not_first_call = <optimized out>

        pagesize_m1 = <optimized out>

        sp = <optimized out>

        freesize = <optimized out>

        __PRETTY_FUNCTION__ = "start_thread"

#9  0x00007ffff5c370af in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105

No locals.

Thread 6 (Thread 0x7fffed8ad700 (LWP 20877)):

#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185

No locals.

#1  0x0000555555c7acd6 in os_event::wait (this=0x555556ed7af0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:165

        ret = <optimized out>

#2  os_event::wait_low (this=this@entry=0x555556ed7af0, reset_sig_count=1, reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:335

No locals.

#3  0x0000555555c7b13a in os_event_wait_low (event=event@entry=0x555556ed7af0, reset_sig_count=reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:534

No locals.

#4  0x0000555555c8b484 in os_aio_simulated_handler (type=0x7fffed8ace80, m2=0x7fffed8ace78, m1=0x7fffed8ace70, global_segment=4)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:7968

        n_slots = <optimized out>

        n_reserved = <optimized out>

        slot = <optimized out>

        array = 0x555556ed41d0

        segment = <optimized out>

        event = 0x555556ed7af0

        handler = {m_oldest = 0, m_n_elems = 0, m_lowest_offset = 18446744073709551615, m_array = 0x555556ed41d0, m_n_slots = 256, m_segment = 2,

          m_slots = std::vector of length 64, capacity 64 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, m_ptr = 0x0, m_buf = 0x0}

#5  os_aio_handler (segment=segment@entry=4, m1=m1@entry=0x7fffed8ace70, m2=m2@entry=0x7fffed8ace78, request=request@entry=0x7fffed8ace80)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:6268

        err = 4294966784

#6  0x0000555555ba5ce1 in fil_aio_wait (segment=segment@entry=4)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/fil/fil0fil.cc:6094

        node = 0x0

        type = {m_block_size = 512, m_type = 1, m_compression = {m_type = Compression::NONE}, m_encryption = {m_type = Encryption::NONE, m_key = 0x0, m_klen = 0,

            m_iv = 0x0, static master_key_id = 0, static uuid = '\000' <repeats 36 times>}}

        message = 0x0

        err = <optimized out>

#7  0x0000555555d2b7d8 in io_handler_thread (arg=<optimized out>)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/srv/srv0start.cc:309

        segment = 4

#8  0x00007ffff7bc16ca in start_thread (arg=0x7fffed8ad700) at pthread_create.c:333

        __res = <optimized out>

        pd = 0x7fffed8ad700

        now = <optimized out>

        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737178687232, 8604873729023514632, 0, 140737488282671, 140737178687936, 140737178687232,

                -8604835149994420216, -8604891898041886712}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0,

              canceltype = 0}}}

        not_first_call = <optimized out>

        pagesize_m1 = <optimized out>

        sp = <optimized out>

        freesize = <optimized out>

        __PRETTY_FUNCTION__ = "start_thread"

#9  0x00007ffff5c370af in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105

No locals.

Thread 5 (Thread 0x7fffee0ae700 (LWP 20876)):

#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185

No locals.

#1  0x0000555555c7acd6 in os_event::wait (this=0x555556ed7a70)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:165

        ret = <optimized out>

#2  os_event::wait_low (this=this@entry=0x555556ed7a70, reset_sig_count=1, reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:335

No locals.

#3  0x0000555555c7b13a in os_event_wait_low (event=event@entry=0x555556ed7a70, reset_sig_count=reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:534

No locals.

#4  0x0000555555c8b484 in os_aio_simulated_handler (type=0x7fffee0ade80, m2=0x7fffee0ade78, m1=0x7fffee0ade70, global_segment=3)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:7968

        n_slots = <optimized out>

        n_reserved = <optimized out>

        slot = <optimized out>

        array = 0x555556ed41d0

        segment = <optimized out>

        event = 0x555556ed7a70

        handler = {m_oldest = 0, m_n_elems = 0, m_lowest_offset = 18446744073709551615, m_array = 0x555556ed41d0, m_n_slots = 256, m_segment = 1,

          m_slots = std::vector of length 64, capacity 64 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, m_ptr = 0x0, m_buf = 0x0}

#5  os_aio_handler (segment=segment@entry=3, m1=m1@entry=0x7fffee0ade70, m2=m2@entry=0x7fffee0ade78, request=request@entry=0x7fffee0ade80)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:6268

        err = 4294966784

#6  0x0000555555ba5ce1 in fil_aio_wait (segment=segment@entry=3)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/fil/fil0fil.cc:6094

        node = 0x0

        type = {m_block_size = 512, m_type = 1, m_compression = {m_type = Compression::NONE}, m_encryption = {m_type = Encryption::NONE, m_key = 0x0, m_klen = 0,

            m_iv = 0x0, static master_key_id = 0, static uuid = '\000' <repeats 36 times>}}

        message = 0x0

        err = <optimized out>

#7  0x0000555555d2b7d8 in io_handler_thread (arg=<optimized out>)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/srv/srv0start.cc:309

        segment = 3

#8  0x00007ffff7bc16ca in start_thread (arg=0x7fffee0ae700) at pthread_create.c:333

        __res = <optimized out>

        pd = 0x7fffee0ae700

        now = <optimized out>

        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737187079936, 8604873729023514632, 0, 140737488282671, 140737187080640, 140737187079936,

                -8604836250042918904, -8604891898041886712}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0,

              canceltype = 0}}}

        not_first_call = <optimized out>

        pagesize_m1 = <optimized out>

        sp = <optimized out>

        freesize = <optimized out>

        __PRETTY_FUNCTION__ = "start_thread"

#9  0x00007ffff5c370af in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105

No locals.

Thread 4 (Thread 0x7fffee8af700 (LWP 20875)):

#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185

No locals.

#1  0x0000555555c7acd6 in os_event::wait (this=0x555556ed79f0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:165

        ret = <optimized out>

#2  os_event::wait_low (this=this@entry=0x555556ed79f0, reset_sig_count=1, reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:335

No locals.

#3  0x0000555555c7b13a in os_event_wait_low (event=event@entry=0x555556ed79f0, reset_sig_count=reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:534

No locals.

#4  0x0000555555c8b484 in os_aio_simulated_handler (type=0x7fffee8aee80, m2=0x7fffee8aee78, m1=0x7fffee8aee70, global_segment=2)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:7968

        n_slots = <optimized out>

        n_reserved = <optimized out>

        slot = <optimized out>

        array = 0x555556ed41d0

        segment = <optimized out>

        event = 0x555556ed79f0

        handler = {m_oldest = 0, m_n_elems = 0, m_lowest_offset = 18446744073709551615, m_array = 0x555556ed41d0, m_n_slots = 256, m_segment = 0,

          m_slots = std::vector of length 64, capacity 64 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, m_ptr = 0x0, m_buf = 0x0}

#5  os_aio_handler (segment=segment@entry=2, m1=m1@entry=0x7fffee8aee70, m2=m2@entry=0x7fffee8aee78, request=request@entry=0x7fffee8aee80)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:6268

        err = 4294966784

#6  0x0000555555ba5ce1 in fil_aio_wait (segment=segment@entry=2)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/fil/fil0fil.cc:6094

        node = 0x0

        type = {m_block_size = 512, m_type = 1, m_compression = {m_type = Compression::NONE}, m_encryption = {m_type = Encryption::NONE, m_key = 0x0, m_klen = 0,

            m_iv = 0x0, static master_key_id = 0, static uuid = '\000' <repeats 36 times>}}

        message = 0x0

        err = <optimized out>

#7  0x0000555555d2b7d8 in io_handler_thread (arg=<optimized out>)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/srv/srv0start.cc:309

        segment = 2

#8  0x00007ffff7bc16ca in start_thread (arg=0x7fffee8af700) at pthread_create.c:333

        __res = <optimized out>

        pd = 0x7fffee8af700

        now = <optimized out>

        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737195472640, 8604873729023514632, 0, 140737488282671, 140737195473344, 140737195472640,

                -8604837350091417592, -8604891898041886712}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0,

              canceltype = 0}}}

        not_first_call = <optimized out>

        pagesize_m1 = <optimized out>

        sp = <optimized out>

        freesize = <optimized out>

        __PRETTY_FUNCTION__ = "start_thread"

#9  0x00007ffff5c370af in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105

No locals.

Thread 3 (Thread 0x7fffe68af700 (LWP 20874)):

#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185

No locals.

#1  0x0000555555c7acd6 in os_event::wait (this=0x555556ed7970)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:165

        ret = <optimized out>

#2  os_event::wait_low (this=this@entry=0x555556ed7970, reset_sig_count=1, reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:335

No locals.

#3  0x0000555555c7b13a in os_event_wait_low (event=event@entry=0x555556ed7970, reset_sig_count=reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:534

No locals.

#4  0x0000555555c8b484 in os_aio_simulated_handler (type=0x7fffe68aee80, m2=0x7fffe68aee78, m1=0x7fffe68aee70, global_segment=1)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:7968

        n_slots = <optimized out>

        n_reserved = <optimized out>

        slot = <optimized out>

        array = 0x555556ed65e0

        segment = <optimized out>

        event = 0x555556ed7970

        handler = {m_oldest = 0, m_n_elems = 0, m_lowest_offset = 18446744073709551615, m_array = 0x555556ed65e0, m_n_slots = 256, m_segment = 0,

          m_slots = std::vector of length 64, capacity 64 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, m_ptr = 0x0, m_buf = 0x0}

#5  os_aio_handler (segment=segment@entry=1, m1=m1@entry=0x7fffe68aee70, m2=m2@entry=0x7fffe68aee78, request=request@entry=0x7fffe68aee80)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:6268

        err = 4294966784

#6  0x0000555555ba5ce1 in fil_aio_wait (segment=segment@entry=1)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/fil/fil0fil.cc:6094

        node = 0x0

        type = {m_block_size = 512, m_type = 1, m_compression = {m_type = Compression::NONE}, m_encryption = {m_type = Encryption::NONE, m_key = 0x0, m_klen = 0,

            m_iv = 0x0, static master_key_id = 0, static uuid = '\000' <repeats 36 times>}}

        message = 0x0

        err = <optimized out>

#7  0x0000555555d2b7d8 in io_handler_thread (arg=<optimized out>)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/srv/srv0start.cc:309

        segment = 1

#8  0x00007ffff7bc16ca in start_thread (arg=0x7fffe68af700) at pthread_create.c:333

        __res = <optimized out>

        pd = 0x7fffe68af700

        now = <optimized out>

        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737061254912, 8604873729023514632, 0, 140737488282671, 140737061255616, 140737061254912,

                -8604854942277462008, -8604891898041886712}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0,

              canceltype = 0}}}

        not_first_call = <optimized out>

        pagesize_m1 = <optimized out>

        sp = <optimized out>

        freesize = <optimized out>

        __PRETTY_FUNCTION__ = "start_thread"

#9  0x00007ffff5c370af in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105

No locals.

Thread 2 (Thread 0x7fffef0b0700 (LWP 20873)):

#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185

No locals.

#1  0x0000555555c7acd6 in os_event::wait (this=0x555556ed78f0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:165

        ret = <optimized out>

#2  os_event::wait_low (this=this@entry=0x555556ed78f0, reset_sig_count=1, reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:335

No locals.

#3  0x0000555555c7b13a in os_event_wait_low (event=event@entry=0x555556ed78f0, reset_sig_count=reset_sig_count@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0event.cc:534

No locals.

#4  0x0000555555c8b484 in os_aio_simulated_handler (type=0x7fffef0afe80, m2=0x7fffef0afe78, m1=0x7fffef0afe70, global_segment=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:7968

        n_slots = <optimized out>

        n_reserved = <optimized out>

        slot = <optimized out>

        array = 0x555556ed43d0

        segment = <optimized out>

        event = 0x555556ed78f0

        handler = {m_oldest = 0, m_n_elems = 0, m_lowest_offset = 18446744073709551615, m_array = 0x555556ed43d0, m_n_slots = 256, m_segment = 0,

          m_slots = std::vector of length 64, capacity 64 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, m_ptr = 0x0, m_buf = 0x0}

#5  os_aio_handler (segment=segment@entry=0, m1=m1@entry=0x7fffef0afe70, m2=m2@entry=0x7fffef0afe78, request=request@entry=0x7fffef0afe80)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0file.cc:6268

        err = 4294966784

#6  0x0000555555ba5ce1 in fil_aio_wait (segment=segment@entry=0)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/fil/fil0fil.cc:6094

        node = 0x0

        type = {m_block_size = 512, m_type = 1, m_compression = {m_type = Compression::NONE}, m_encryption = {m_type = Encryption::NONE, m_key = 0x0, m_klen = 0,

            m_iv = 0x0, static master_key_id = 0, static uuid = '\000' <repeats 36 times>}}

        message = 0x0

        err = <optimized out>

#7  0x0000555555d2b7d8 in io_handler_thread (arg=<optimized out>)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/srv/srv0start.cc:309

        segment = 0

#8  0x00007ffff7bc16ca in start_thread (arg=0x7fffef0b0700) at pthread_create.c:333

        __res = <optimized out>

        pd = 0x7fffef0b0700

        now = <optimized out>

        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737203865344, 8604873729023514632, 0, 140737488282671, 140737203866048, 140737203865344,

                -8604838450139916280, -8604891898041886712}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0,

              canceltype = 0}}}

        not_first_call = <optimized out>

        pagesize_m1 = <optimized out>

        sp = <optimized out>

        freesize = <optimized out>

        __PRETTY_FUNCTION__ = "start_thread"

#9  0x00007ffff5c370af in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105

No locals.

Thread 1 (Thread 0x7ffff7fd46c0 (LWP 20869)):

#0  0x00007ffff7bcaced in nanosleep () at ../sysdeps/unix/syscall-template.S:84

No locals.

#1  0x0000555555c8cc8f in os_thread_sleep (tm=tm@entry=100000)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/os/os0thread.cc:254

        t = {tv_sec = 0, tv_nsec = 100000000}

#2  0x0000555555d2c063 in srv_shutdown_all_bg_threads ()

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/srv/srv0start.cc:1272

        active = true

#3  0x0000555555d2c60e in srv_init_abort_low (create_new_db=<optimized out>, err=err@entry=DB_ERROR)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/srv/srv0start.cc:1332

No locals.

#4  0x0000555555d33366 in innobase_start_or_create_for_mysql ()

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/srv/srv0start.cc:2414

        create_new_db = false

        flushed_lsn = 1620098

        sum_of_data_file_sizes = <optimized out>

        tablespace_size_in_header = <optimized out>

        err = <optimized out>

        srv_n_log_files_found = <optimized out>

        mtr = {m_impl = {m_memo = {m_heap = 0x0, m_list = {count = 1, start = 0x7fffffff2530, end = 0x7fffffff2530, node = &dyn_buf_t<512ul>::block_t::m_node},

              m_size = 0, m_first_block = {m_data = '\000' <repeats 499 times>, m_node = {prev = 0x0, next = 0x0}, m_used = 0}}, m_log = {m_heap = 0x0, m_list = {

                count = 1, start = 0x7fffffff2770, end = 0x7fffffff2770, node = &dyn_buf_t<512ul>::block_t::m_node}, m_size = 0, m_first_block = {

                m_data = '\000' <repeats 499 times>, m_node = {prev = 0x0, next = 0x0}, m_used = 0}}, m_made_dirty = false, m_inside_ibuf = false,

            m_modifications = false, m_n_log_recs = 0, m_log_mode = 0, m_user_space = 0x0, m_undo_space = 0x0, m_sys_space = 0x0, m_state = MTR_STATE_INIT,

            m_flush_observer = 0x0, m_mtr = 0x0}, m_commit_lsn = 0, m_sync = false}

        purge_queue = <optimized out>

        logfilename = "./ib_logfile0\000\000\000\320y\377\377\377\177", '\000' <repeats 58 times>, "\377\377\377\377\004", '\000' <repeats 27 times>, " \025\357\365\377\177", '\000' <repeats 26 times>, "xtrabackup:   in\377\377\377\377_log_file_size = 838\300\310\356\365\377\177\000\000 \025\357\365\377\177\000\000"...

        logfile0 = 0x0

        dirnamelen = <optimized out>

        i = <optimized out>

        size = <optimized out>

        unit = <optimized out>

        chunk_size = <optimized out>

        chunk_unit = <optimized out>

        sum_of_new_sizes = 0

#5  0x0000555555ac2ae9 in innodb_init ()

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/xtrabackup/src/xtrabackup.cc:1961

        err = <optimized out>

#6  0x0000555555accb62 in xtrabackup_prepare_func ()

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/xtrabackup/src/xtrabackup.cc:7441

        err = <optimized out>

        it = <optimized out>

        node = <optimized out>

        space = <optimized out>

        metadata_path = "./xtrabackup_checkpoints", '\000' <repeats 487 times>

        write_request = {m_block_size = 512, m_type = 2, m_compression = {m_type = Compression::NONE}, m_encryption = {m_type = Encryption::NONE, m_key = 0x0,

            m_klen = 0, m_iv = 0x0, static master_key_id = 0, static uuid = '\000' <repeats 36 times>}}

#7  0x0000555555aad2e4 in main (argc=<optimized out>, argv=<optimized out>)

    at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/xtrabackup/src/xtrabackup.cc:8321

        client_defaults = 0x555556eaff90

        server_defaults = 0x555556eaf288

        cwd = "/home/a/test1/mariadb-environs/_plugin/xtrabackup/\000\000\000\000\000\000\070\000\264\365\377\177\000\000\260\310\377\377\377\177\000\000\220-\263\365\377\177\000\000\324\307\377\377\377\177\000\000\240\310\377\377\377\177\000\000\370a\375\367\377\177\000\000\000\000\000\000\377\177\000\000\020\000\000\000\000\000\000\000\060\060\375\367\377\177\000\000\001", '\000' <repeats 15 times>, "\001\000\000\000\000\000\000\000@X\375\367\377\177\000\000@\311\377\377\377\177\000\000h7\375\367\377\177\000\000\001", '\000' <repeats 15 times>, "h\311\377\377\377\177\000\000"...

Comment by Marko Mäkelä [ 2017-05-08 ]

It seems to me that the crash occurs because MySQL 5.7 (which Percona Xtrabackup is based on) is not doing a proper shutdown after a failed startup.
Because of the failed startup, purge_sys must have been left at NULL, and then shutdown would hit SIGSEGV in the function trx_purge_state().

The real issue seems to be the following:

InnoDB: Doing recovery: scanned up to log sequence number 1770296 (2%)

InnoDB: Plugin initialization aborted with error Generic error

Why was the startup aborted? I cannot think of any redo log format changes between MySQL 5.7.9 and MariaDB 10.2.6, other than ~~MDEV-11782~~ which changed the encrypted redo log format (which is not compatible with MySQL 5.7 to begin with). At this stage, I do not think that other file format changes (such as the transaction system state format change in ~~MDEV-12289~~) would matter.

Comment by Andrii Nikitin (Inactive) [ 2017-05-08 ]

Apparently it doesn't like format of part of the log , attaching the log in a moment

InnoDB: Log scan progressed past the checkpoint lsn 1620117

Thread 1 "innobackupex" hit Hardware watchpoint 2: recv_sys->found_corrupt_log

Old value = false

New value = true

page_zip_parse_write_header (ptr=0x555557cb9bd1 "3\004\003", end_ptr=0x555557cbd407 "", page=0x0, page_zip=0x0) at /mnt/workspace/percona-xtrabackup-2.4-debian-binary/label_exp/ubuntu-yakkety-64bit/percona-xtrabackup-24-2.4.7/storage/innobase/page/page0zip.cc:4549

4549			return(NULL);

Comment by Andrii Nikitin (Inactive) [ 2017-05-08 ]

Output of 10.2 debug server started on backup folder.

recv_group_scan_log_recs: ib_log: scan 1664698: multi-log rec MLOG_ZIP_WRITE_HEADER len 13 page 4:4

recv_group_scan_log_recs: ib_log: scan 1664698: multi-log rec MLOG_ZIP_WRITE_HEADER len 9 page 4:4

recv_group_scan_log_recs: ib_log: scan 1664698: multi-log rec MLOG_ZIP_WRITE_HEADER len 9 page 4:4

recv_group_scan_log_recs: ib_log: scan 1664698: multi-log rec MLOG_ZIP_PAGE_COMPRESS len 5611 page 4:4

2017-05-08 14:21:23 139851747288896 [ERROR] InnoDB: Plugin initialization aborted at srv0start.cc[2201] with error Generic error

This indicates that most probably regular crash recovery may fail on 10.2 (without backup). Marko promised to look into log file.

Comment by Andrii Nikitin (Inactive) [ 2017-05-08 ]

InnoDB fails to start

2017-05-08 15:46:33 140093471946240 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1634146

2017-05-08 15:46:33 140093471946240 [ERROR] InnoDB: Plugin initialization aborted with error Generic error

with following mtr test:

--source include/have_innodb.inc

--source include/not_embedded.inc

create table test.a(i int primary key auto_increment, s varchar(255) ) ENGINE=InnoDB ROW_FORMAT=compressed;

insert into a(i) select null;

insert into a select null, uuid() from test.a a, test.a b, test.a c;

insert into a select null, uuid() from test.a a, test.a b, test.a c;

insert into a select null, uuid() from test.a a, test.a b, test.a c;

--source include/kill_mysqld.inc

--source include/start_mysqld.inc

select count(*) from a;

Comment by Marko Mäkelä [ 2017-05-09 ]

My patch for ~~MDEV-12123~~ is misusing the page_zip_write_header() API, passing length=0 which corrupts the redo log.

[MDEV-12720] recovery fails with "Generic error" for ROW_FORMAT=compressed Created: 2017-05-07 Updated: 2017-05-18 Resolved: 2017-05-09
Status:	Closed
Project:	MariaDB Server
Component/s:	Storage Engine - InnoDB
Affects Version/s:	10.2.6
Fix Version/s:	10.2.6

[MDEV-12720] recovery fails with "Generic error" for ROW_FORMAT=compressed Created: 2017-05-07 Updated: 2017-05-18 Resolved: 2017-05-09