[MDEV-10217] innodb.innodb_bug59641 fails sporadically in buildbot: InnoDB: Failing assertion: current_rec != insert_rec in file page0cur.c line 1052 Created: 2016-06-12  Updated: 2020-08-31  Resolved: 2020-08-31

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB, Tests
Affects Version/s: 5.5
Fix Version/s: N/A

Type: Bug Priority: Minor
Reporter: Elena Stepanova Assignee: Marko Mäkelä
Resolution: Duplicate Votes: 0
Labels: need_feedback

Issue Links:
Blocks
blocks MDEV-7069 Fix buildbot failures in main server ... Stalled
Relates
relates to MDEV-11799 InnoDB can abort if the doublewrite b... Closed
relates to MDEV-12699 Improve crash recovery of corrupted d... Closed
relates to MDEV-12905 InnoDB occasionally skips the doublew... Closed
relates to MDEV-21347 innodb_log_optimize_ddl=OFF is not cr... Closed

 Description   

http://buildbot.askmonty.org/buildbot/builders/p8-rhel71-bintar-debug/builds/1000/steps/test/logs/stdio

InnoDB: Progress in percents: 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 160612 17:16:22  InnoDB: Assertion failure in thread 70366885572992 in file page0cur.c line 1052
InnoDB: Failing assertion: current_rec != insert_rec

160612 17:16:22 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see http://kb.askmonty.org/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 5.5.50-MariaDB-debug
key_buffer_size=1048576
read_buffer_size=131072
max_used_connections=0
max_threads=153
thread_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 62567 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x48000
mysys/stacktrace.c:246(my_print_stacktrace)[0x10beab8c]
sql/signal_handler.cc:153(handle_fatal_signal)[0x1049ece0]
linux-vdso64.so.1(__kernel_sigtramp_rt64+0x0)[0x3fff9aac0478]
/opt/at8.0/lib64/power8/libc.so.6(gsignal+0x48)[0x3fff9a53ee38]
/opt/at8.0/lib64/power8/libc.so.6(abort+0x2ac)[0x3fff9a5419dc]
page/page0cur.c:1052(page_cur_insert_rec_low)[0x10b26904]
include/page0cur.ic:296(page_cur_rec_insert)[0x10b2491c]
page/page0cur.c:923(page_cur_parse_insert_rec)[0x10b2649c]
log/log0recv.c:1120(recv_parse_or_apply_log_rec_body)[0x10af591c]
log/log0recv.c:1690(recv_recover_page_func)[0x10af78bc]
buf/buf0buf.c:4045(buf_page_io_complete)[0x10a1a720]
fil/fil0fil.c:5734(fil_aio_wait)[0x10a906b8]
srv/srv0start.c:497(io_handler_thread)[0x1097b920]
/opt/at8.0/lib64/power8/libpthread.so.0(+0x7cac)[0x3fff9aa87cac]
/opt/at8.0/lib64/power8/libc.so.6(clone+0x98)[0x3fff9a62a7c0]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Writing a core file
----------SERVER LOG END-------------
mysqltest failed but provided no output
The result from queries just before the failure was:
< snip >
call mtr.add_suppression("Found 3 prepared XA transactions");
flush tables;
CREATE TABLE t(a INT PRIMARY KEY, b INT)ENGINE=InnoDB;
INSERT INTO t VALUES(2,2),(4,4),(8,8),(16,16),(32,32);
COMMIT;
XA START '123';
INSERT INTO t VALUES(1,1);
XA END '123';
XA PREPARE '123';
XA START '456';
INSERT INTO t VALUES(3,47),(5,67);
UPDATE t SET b=2*b WHERE a BETWEEN 5 AND 8;
XA END '456';
XA PREPARE '456';
XA START '789';
UPDATE t SET b=4*a WHERE a=32;
XA END '789';
XA PREPARE '789';

#0  0x00003fff9aa90ed0 in __pthread_kill (threadid=<optimized out>, signo=<optimized out>) at ../sysdeps/unix/sysv/linux/pthread_kill.c:60
60	../sysdeps/unix/sysv/linux/pthread_kill.c: No such file or directory.
#0  0x00003fff9aa90ed0 in __pthread_kill (threadid=<optimized out>, signo=<optimized out>) at ../sysdeps/unix/sysv/linux/pthread_kill.c:60
#1  0x0000000010bead2c in my_write_core (sig=6) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysys/stacktrace.c:457
#2  0x000000001049f17c in handle_fatal_signal (sig=6) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/signal_handler.cc:262
#3  <signal handler called>
#4  0x00003fff9a53ee38 in __GI_raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:55
#5  0x00003fff9a54193c in __GI_abort () at abort.c:89
#6  0x0000000010b26904 in page_cur_insert_rec_low (current_rec=0x3fff92f12e02 "", index=0x3fff80003848, rec=0x3fff9137d806 "", offsets=0x3fff9137dc18, mtr=0x3fff9137e0e8) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/page/page0cur.c:1052
#7  0x0000000010b2491c in page_cur_rec_insert (cursor=0x3fff9137dc00, rec=0x3fff9137d806 "", index=0x3fff80003848, offsets=0x3fff9137dc18, mtr=0x3fff9137e0e8) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/include/page0cur.ic:296
#8  0x0000000010b2649c in page_cur_parse_insert_rec (is_short=0, ptr=0x1000ea10653 "\261", end_ptr=0x1000ea10654 "", block=0x3fff92b9d640, index=0x3fff80003848, mtr=0x3fff9137e0e8) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/page/page0cur.c:923
#9  0x0000000010af591c in recv_parse_or_apply_log_rec_body (type=38 '&', ptr=0x1000ea10650 ".\002\002\261", end_ptr=0x1000ea10654 "", block=0x3fff92b9d640, mtr=0x3fff9137e0e8, space_id=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/log/log0recv.c:1120
#10 0x0000000010af78bc in recv_recover_page_func (just_read_in=1, block=0x3fff92b9d640) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/log/log0recv.c:1685
#11 0x0000000010a1a720 in buf_page_io_complete (bpage=0x3fff92b9d640) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/buf/buf0buf.c:4045
#12 0x0000000010a906b8 in fil_aio_wait (segment=3) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/fil/fil0fil.c:5734
#13 0x000000001097b920 in io_handler_thread (arg=0x11957c38 <n+24>) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/srv/srv0start.c:497
#14 0x00003fff9aa87cac in start_thread (arg=0x3fff9137f180) at pthread_create.c:312
#15 0x00003fff9a62a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 8 (LWP 17425):
#0  0x00003fff9aa8fca0 in __pthread_cond_timedwait (cond=0x11953b00 <COND_checkpoint>, mutex=0x11953a80 <LOCK_checkpoint+40>, abstime=0x3fff9a4fe610) at pthread_cond_timedwait.c:199
#1  0x0000000010bf7070 in safe_cond_timedwait (cond=0x11953b00 <COND_checkpoint>, mp=0x11953a58 <LOCK_checkpoint>, abstime=0x3fff9a4fe610, file=0x10d5d090 "/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/maria/ma_servicethread.c", line=116) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysys/thr_mutex.c:547
#2  0x000000001077a960 in inline_mysql_cond_timedwait (that=0x11953b00 <COND_checkpoint>, mutex=0x11953a58 <LOCK_checkpoint>, abstime=0x3fff9a4fe610, src_file=0x10d5d090 "/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/include/mysql/psi/mysql_thread.h:1017
#3  0x000000001077b07c in my_service_thread_sleep (control=0x10ff56b8 <checkpoint_control>, sleep_time=29000000000) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/maria/ma_servicethread.c:115
#4  0x0000000010769d34 in ma_checkpoint_background (arg=0x1e) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/maria/ma_checkpoint.c:704
#5  0x00000000108e0860 in pfs_spawn_thread (arg=0x1000de38630) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/perfschema/pfs.cc:1015
#6  0x00003fff9aa87cac in start_thread (arg=0x3fff9a4ff180) at pthread_create.c:312
#7  0x00003fff9a62a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 7 (LWP 17423):
#0  0x00003fff9a61d314 in select () at ../sysdeps/unix/syscall-template.S:81
#1  0x0000000010b1b070 in os_thread_sleep (tm=500000) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/os/os0thread.c:261
#2  0x0000000010af81f0 in recv_apply_hashed_log_recs (allow_ibuf=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/log/log0recv.c:1919
#3  0x0000000010af9cf8 in recv_scan_log_recs (available_memory=4177920, store_to_hash=1, buf=0x1000e904200 "", len=65536, start_lsn=8501248, contiguous_lsn=0x3ffff43f8e58, group_scanned_lsn=0x3ffff43f8e50) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/log/log0recv.c:2863
#4  0x0000000010af9e50 in recv_group_scan_log_recs (group=0x1000ea0ddb8, contiguous_lsn=0x3ffff43f8e58, group_scanned_lsn=0x3ffff43f8e50) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/log/log0recv.c:2906
#5  0x0000000010afa4f4 in recv_recovery_from_checkpoint_start_func (min_flushed_lsn=0, max_flushed_lsn=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/log/log0recv.c:3193
#6  0x000000001097ef64 in innobase_start_or_create_for_mysql () at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/srv/srv0start.c:1968
#7  0x00000000108eeb1c in innobase_init (p=0x1000deca4f0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/handler/ha_innodb.cc:3246
#8  0x00000000104a18c4 in ha_initialize_handlerton (plugin=0x1000de17078) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/handler.cc:470
#9  0x00000000102638fc in plugin_initialize (tmp_root=0x3ffff43f9e10, plugin=0x1000de17078, argc=0x11137e70 <remaining_argc>, argv=0x1000dd98c20, options_only=false) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_plugin.cc:1372
#10 0x000000001026449c in plugin_init (argc=0x11137e70 <remaining_argc>, argv=0x1000dd98c20, flags=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_plugin.cc:1597
#11 0x000000001015a96c in init_server_components () at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld.cc:4349
#12 0x000000001015c184 in mysqld_main (argc=90, argv=0x1000dd98c20) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld.cc:4946
#13 0x0000000010150678 in main (argc=12, argv=0x3ffff43fa6b8) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/main.cc:25
 
Thread 6 (LWP 17428):
#0  0x0000000010a21f88 in buf_flush_insert_sorted_into_flush_list (buf_pool=0x1000e879a18, block=0x3fff92ba3b80, lsn=7720640) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/buf/buf0flu.c:429
#1  0x0000000010af2a9c in buf_flush_recv_note_modification (block=0x3fff92ba3b80, start_lsn=7720640, end_lsn=7764075) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/include/buf0flu.ic:127
#2  0x0000000010af7a88 in recv_recover_page_func (just_read_in=1, block=0x3fff92ba3b80) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/log/log0recv.c:1730
#3  0x0000000010a1a720 in buf_page_io_complete (bpage=0x3fff92ba3b80) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/buf/buf0buf.c:4045
#4  0x0000000010a906b8 in fil_aio_wait (segment=2) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/fil/fil0fil.c:5734
#5  0x000000001097b920 in io_handler_thread (arg=0x11957c30 <n+16>) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/srv/srv0start.c:497
#6  0x00003fff9aa87cac in start_thread (arg=0x3fff91b7f180) at pthread_create.c:312
#7  0x00003fff9a62a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 5 (LWP 17427):
#0  0x00003fff9aa30738 in __io_getevents_0_4 () from /lib64/libaio.so.1
#1  0x00003fff9aa307a0 in io_getevents () from /lib64/libaio.so.1
#2  0x0000000010b16244 in os_aio_linux_collect (array=0x1000e80df20, segment=0, seg_size=256) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/os/os0file.c:4703
#3  0x0000000010b167b8 in os_aio_linux_handle (global_seg=1, message1=0x3fff9237e718, message2=0x3fff9237e720, type=0x3fff9237e728, space_id=0x3fff9237e730) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/os/os0file.c:4849
#4  0x0000000010a904d4 in fil_aio_wait (segment=1) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/fil/fil0fil.c:5697
#5  0x000000001097b920 in io_handler_thread (arg=0x11957c28 <n+8>) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/srv/srv0start.c:497
#6  0x00003fff9aa87cac in start_thread (arg=0x3fff9237f180) at pthread_create.c:312
#7  0x00003fff9a62a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 4 (LWP 17431):
#0  0x00003fff9aa30738 in __io_getevents_0_4 () from /lib64/libaio.so.1
#1  0x00003fff9aa307a0 in io_getevents () from /lib64/libaio.so.1
#2  0x0000000010b16244 in os_aio_linux_collect (array=0x1000e83ac20, segment=1, seg_size=256) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/os/os0file.c:4703
#3  0x0000000010b167b8 in os_aio_linux_handle (global_seg=5, message1=0x3fff9037e718, message2=0x3fff9037e720, type=0x3fff9037e728, space_id=0x3fff9037e730) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/os/os0file.c:4849
#4  0x0000000010a904d4 in fil_aio_wait (segment=5) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/fil/fil0fil.c:5697
#5  0x000000001097b920 in io_handler_thread (arg=0x11957c48 <n+40>) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/srv/srv0start.c:497
#6  0x00003fff9aa87cac in start_thread (arg=0x3fff9037f180) at pthread_create.c:312
#7  0x00003fff9a62a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 3 (LWP 17430):
#0  0x00003fff9aa30738 in __io_getevents_0_4 () from /lib64/libaio.so.1
#1  0x00003fff9aa307a0 in io_getevents () from /lib64/libaio.so.1
#2  0x0000000010b16244 in os_aio_linux_collect (array=0x1000e83ac20, segment=0, seg_size=256) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/os/os0file.c:4703
#3  0x0000000010b167b8 in os_aio_linux_handle (global_seg=4, message1=0x3fff90b7e718, message2=0x3fff90b7e720, type=0x3fff90b7e728, space_id=0x3fff90b7e730) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/os/os0file.c:4849
#4  0x0000000010a904d4 in fil_aio_wait (segment=4) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/fil/fil0fil.c:5697
#5  0x000000001097b920 in io_handler_thread (arg=0x11957c40 <n+32>) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/srv/srv0start.c:497
#6  0x00003fff9aa87cac in start_thread (arg=0x3fff90b7f180) at pthread_create.c:312
#7  0x00003fff9a62a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 2 (LWP 17426):
#0  0x00003fff9aa30738 in __io_getevents_0_4 () from /lib64/libaio.so.1
#1  0x00003fff9aa307a0 in io_getevents () from /lib64/libaio.so.1
#2  0x0000000010b16244 in os_aio_linux_collect (array=0x1000e7fe8a0, segment=0, seg_size=256) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/os/os0file.c:4703
#3  0x0000000010b167b8 in os_aio_linux_handle (global_seg=0, message1=0x3fff92b7e718, message2=0x3fff92b7e720, type=0x3fff92b7e728, space_id=0x3fff92b7e730) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/os/os0file.c:4849
#4  0x0000000010a904d4 in fil_aio_wait (segment=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/fil/fil0fil.c:5697
#5  0x000000001097b920 in io_handler_thread (arg=0x11957c20 <n>) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/srv/srv0start.c:497
#6  0x00003fff9aa87cac in start_thread (arg=0x3fff92b7f180) at pthread_create.c:312
#7  0x00003fff9a62a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 1 (LWP 17429):
#0  0x00003fff9aa90ed0 in __pthread_kill (threadid=<optimized out>, signo=<optimized out>) at ../sysdeps/unix/sysv/linux/pthread_kill.c:60
#1  0x0000000010bead2c in my_write_core (sig=6) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysys/stacktrace.c:457
#2  0x000000001049f17c in handle_fatal_signal (sig=6) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/signal_handler.cc:262
#3  <signal handler called>
#4  0x00003fff9a53ee38 in __GI_raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:55
#5  0x00003fff9a54193c in __GI_abort () at abort.c:89
#6  0x0000000010b26904 in page_cur_insert_rec_low (current_rec=0x3fff92f12e02 "", index=0x3fff80003848, rec=0x3fff9137d806 "", offsets=0x3fff9137dc18, mtr=0x3fff9137e0e8) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/page/page0cur.c:1052
#7  0x0000000010b2491c in page_cur_rec_insert (cursor=0x3fff9137dc00, rec=0x3fff9137d806 "", index=0x3fff80003848, offsets=0x3fff9137dc18, mtr=0x3fff9137e0e8) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/include/page0cur.ic:296
#8  0x0000000010b2649c in page_cur_parse_insert_rec (is_short=0, ptr=0x1000ea10653 "\261", end_ptr=0x1000ea10654 "", block=0x3fff92b9d640, index=0x3fff80003848, mtr=0x3fff9137e0e8) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/page/page0cur.c:923
#9  0x0000000010af591c in recv_parse_or_apply_log_rec_body (type=38 '&', ptr=0x1000ea10650 ".\002\002\261", end_ptr=0x1000ea10654 "", block=0x3fff92b9d640, mtr=0x3fff9137e0e8, space_id=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/log/log0recv.c:1120
#10 0x0000000010af78bc in recv_recover_page_func (just_read_in=1, block=0x3fff92b9d640) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/log/log0recv.c:1685
#11 0x0000000010a1a720 in buf_page_io_complete (bpage=0x3fff92b9d640) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/buf/buf0buf.c:4045
#12 0x0000000010a906b8 in fil_aio_wait (segment=3) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/fil/fil0fil.c:5734
#13 0x000000001097b920 in io_handler_thread (arg=0x11957c38 <n+24>) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/xtradb/srv/srv0start.c:497
#14 0x00003fff9aa87cac in start_thread (arg=0x3fff9137f180) at pthread_create.c:312
#15 0x00003fff9a62a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

The failure happens extremely rarely, cross-reference only returns 5 occasions over several years. All occasions on 5.5.



 Comments   
Comment by Marko Mäkelä [ 2017-10-11 ]

This looks like that redo log could be applied to a corrupted page.

In the server error log there is no preceding message about page corruption. But it is possible that the page was all-zero, which would pass the checksum. Or that the page checksum field happened to contain the magic value 0xdeadbeef, making InnoDB blind for the page corruption.
The bug could be caused by errors in the InnoDB doublewrite buffer handling. Both MDEV-12905 and MDEV-11799 are still open.

In MariaDB 5.5, the setting --debug=d,ib_log has no effect (and such setting was not enabled during the test), so we cannot know if this issue would be worked around by MDEV-12699 when it is implemented. If the ib_logfile* were available, their contents could be pretty-printed using 10.0 and --debug=d,ib_log.

Comment by Marko Mäkelä [ 2020-07-31 ]

elenst, I believe that MDEV-11799 may have fixed this. See MDEV-23231 for a scenario on 10.4 that was addressed by MDEV-11799.

There are also other rather recent fixes that make the crash recovery more robust, such as MDEV-12699. Have you seen this type of failures recently?

Comment by Elena Stepanova [ 2020-07-31 ]

I didn't get it in my tests, and the last recorded failure in buildbot is over a year ago on 5.5, so I suppose it can be closed.

Generated at Thu Feb 08 07:40:32 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.