[MDEV-11976] incremental backup may crash when KEY_BLOCK_SIZE=1 and innodb-track-changed-pages Created: 2017-02-02  Updated: 2017-10-28  Resolved: 2017-10-28

Status: Closed
Project: MariaDB Server
Component/s: Backup
Affects Version/s: 10.1
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Andrii Nikitin (Inactive) Assignee: Vladislav Vaintroub
Resolution: Cannot Reproduce Votes: 0
Labels: None

Attachments: Text File case1.log     Text File case2.log     Text File case5.log     File case5.sh     File incremental_compressed_bitmap_1kb.opt     File incremental_compressed_bitmap_1kb.test    

 Description   

Attached mtr test :
1. crashes reliably in virtual Ubuntu 16.04 on Windows 10 host (with -DWITH_LIBARCHIVE=1)
2. sometimes crashes in virtual Centos 7 on Ubuntu 16.04 host (with -DWITH_LIBARCHIVE=1)
3. Doesn't crash in the same virtual Centos 7 on Ubuntu 16.04 host when binaries built without libarchive
4. doesn't crash in native Ubuntu 16.04 with and without DWITH_LIBARCHIVE=1
5. 'Similar' shell script on the same virtual Ubuntu 16.04 on Windows 10 host (with -DWITH_LIBARCHIVE=1) shows little different crash
6. Never crashed for me with KEY_BLOCK_SIZE=2
7. Other small changes to test case in 1. or 5. (like using different database name for test table or inserting data different way or reducing number of rows or do no restart or disable innodb-track-changed-pages) usually makes the test pass without crash.

Stack trace of mtr test for virtual ubuntu 16.04 (case 1):
(Note that stack traces are missing and it continues printing messages after the crash and test case aborts on timeout)

[01] ./incremental_sample/test.ibd is compressed with page size = 1024 bytes
170202 13:29:15 [01] Copying ./incremental_sample/test.ibd to /farm/m0-bb-10.1-wlad-xtrabackup/build/mysql-test/var/tmp/backup_inc1/incremental_sample/test.ibd.
delta
170202 13:29:15 [01]        ...done
170202 13:29:15 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
...
Thread pointer: 0x0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
170202 13:29:15 >> log scanned up to (2532997)
170202 13:29:16 >> log scanned up to (2532997)
...
170202 13:29:41 >> log scanned up to (2532997)
170202 13:29:42 >> log scanned up to (2532997)

Stack trace for virtual Centos 7 (case 2)

[01] ./incremental_sample/test.ibd is compressed with page size = 1024 bytes
170202 13:53:02 [01] Copying ./incremental_sample/test.ibd to /farm/m0-bb-10.1-wlad-xtrabackup/build/mysql-test/var/tmp/backup_inc1/incremental_sample/test.ibd.delta
170202 13:53:02 [ERROR] mysqld got signal 11 ;
...
Thread pointer: 0x0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x48400
/farm/m0-bb-10.1-wlad-xtrabackup/build/extra/xtrabackup/xtrabackup(my_print_stacktrace+0x29)[0x55d2a8e81a69]
/farm/m0-bb-10.1-wlad-xtrabackup/build/extra/xtrabackup/xtrabackup(handle_fatal_signal+0x305)[0x55d2a8a42d35]
/lib64/libpthread.so.0(+0xf370)[0x7fd8f078e370]
/lib64/libc.so.6(+0x1493cb)[0x7fd8ef6953cb]
mysys/stacktrace.c:268(my_print_stacktrace)[0x55d2a882f889]
xtrabackup/xtrabackup.cc:2424(xtrabackup_copy_datafile)[0x55d2a8824799]
xtrabackup/xtrabackup.cc:2948(data_copy_thread_func)[0x55d2a88249f2]
/lib64/libpthread.so.0(+0x7dc5)[0x7fd8f0786dc5]
/lib64/libc.so.6(clone+0x6d)[0x7fd8ef64373d]

Stack trace for bash script (Ubuntu 16.04 - case 5)

[01] ./incremental_sample/test.ibd is compressed with page size = 1024 bytes
170202 14:20:13 [01] Copying ./incremental_sample/test.ibd to /farm/x1-2.4.4/var/var1/data/delta/incremental_sample/test.ibd.delta
170202 14:20:13 [ERROR] mysqld got signal 11 ;
...
stack_bottom = 0x0 thread_stack 0x48400
xtrabackup(my_print_stacktrace+0x29)[0x5641fd75f519]
xtrabackup(handle_fatal_signal+0x2f5)[0x5641fd349395]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f62f9232390]
xtrabackup(ds_write+0x4)[0x5641fd12eda4]
xtrabackup(+0x39634c)[0x5641fd13234c]
mysys/stacktrace.c:268(my_print_stacktrace)[0x5641fd127ae3]
xtrabackup.cc:2948(data_copy_thread_func(void*))[0x5641fd127cb2]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7f62f92286ba]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f62f779782d]

To repeat: copy attached mtr test into xtrabackup suite and run with --repeat=100
PS: I did run case 1. with percona-xtrabackup binary and it didn't crash



 Comments   
Comment by Andrii Nikitin (Inactive) [ 2017-10-28 ]

The issue is not reproducible anymore in current 10.1 , so closing it for now with "Can't reproduce"

Generated at Thu Feb 08 07:54:07 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.