[MDEV-20213] binlog_encryption.binlog_incident failed in buildbot, server crashed in Check::validate Created: 2019-07-30  Updated: 2020-06-27  Resolved: 2020-06-27

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB, Tests
Affects Version/s: 10.3, 10.4
Fix Version/s: 10.3.24, 10.4.14, 10.5.5

Type: Bug Priority: Critical
Reporter: Elena Stepanova Assignee: Eugene Kosov (Inactive)
Resolution: Fixed Votes: 0
Labels: affects-tests

Issue Links:
Duplicate
duplicates MDEV-19298 Assertion `space->id != 0xFFFFFFFEU |... Closed
is duplicated by MDEV-20159 Assertion `space->id != TRX_SYS_SPACE... Closed
Relates
relates to MDEV-19298 Assertion `space->id != 0xFFFFFFFEU |... Closed

 Description   

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/18767

10.4 de9e3930940c44458d254987066a81f1

binlog_encryption.binlog_incident 'mix'  w3 [ fail ]
        Test ended at 2019-07-26 12:19:52
 
CURRENT_TEST: binlog_encryption.binlog_incident
 
 
Failed to start mysqld.1
mysqltest failed but provided no output
 
 
 - found 'core' (0/0)
 
Trying 'dbx' to get a backtrace
 
Trying 'gdb' to get a backtrace from coredump /mnt/buildbot/build/mariadb-10.4.7/mysql-test/var/3/log/binlog_encryption.binlog_incident-mix/mysqld.1/data/core
Core generated by '/mnt/buildbot/build/mariadb-10.4.7/sql/mysqld'
Output from gdb follows. The first stack trace is from the failing thread.
The following stack traces are from all threads (so the failing one is
duplicated).
--------------------------
[New LWP 4437]
[New LWP 4431]
[New LWP 4438]
[New LWP 4436]
[New LWP 4435]
[New LWP 4434]
[New LWP 4433]
[New LWP 4432]
[New LWP 4430]
[New LWP 4427]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `/mnt/buildbot/build/mariadb-10.4.7/sql/mysqld --defaults-group-suffix=.1 --defa'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x810b189a in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.7/mysys/my_read.c:63
63	      int got_errno= my_errno= errno;
[Current thread is 1 (Thread 0xad5f9b40 (LWP 4437))]
#0  0x810b189a in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.7/mysys/my_read.c:63
#1  0x8091ef8f in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/signal_handler.cc:66
#2  0x8091f783 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb778fc31 in __kernel_vsyscall ()
#5  0xb706de89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb706f3e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb7066d07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb7066d8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x80f6fa62 in Check::validate (space=0x84447738) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4567
#10 0x80f6db68 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4598
#11 0x80f5fcf4 in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:216
#12 0x80f6d233 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4353
#13 0x80e17243 in io_handler_thread (arg=0x823ef6f4 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/srv/srv0start.cc:324
#14 0xb73ed2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb712916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 10 (Thread 0xb6eaf7c0 (LWP 4427)):
#0  0xb778fc31 in __kernel_vsyscall ()
#1  0xb73f2a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80d31854 in os_event::wait (this=0x843433a8) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0event.cc:158
#3  0x80d312b5 in os_event::wait_low (this=0x843433a8, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0event.cc:325
#4  0x80d315fb in os_event_wait_low (event=0x843433a8, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0event.cc:502
#5  0x80e1facf in sync_array_wait_event (arr=0x842d3510, cell=@0xbf9577e0: 0x842d35a0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/sync/sync0arr.cc:471
#6  0x80c799ad in TTASEventMutex<GenericPolicy>::enter (this=0x819e6870 <fil_system+16>, max_spins=60, max_delay=4, filename=0x814f08d8 "/home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc", line=4586) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/include/ib0mutex.h:478
#7  0x80c77caa in PolicyMutex<TTASEventMutex<GenericPolicy> >::enter (this=0x819e6870 <fil_system+16>, n_spins=30, n_delay=4, name=0x814f08d8 "/home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc", line=4586) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/include/ib0mutex.h:595
#8  0x80f6db17 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4586
#9  0x80f8ee60 in SysTablespace::open_or_create (this=0x823fc980 <srv_sys_space>, is_temp=false, create_new_db=false, sum_new_sizes=0xbf957968, flush_lsn=0xbf9579b8) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fsp/fsp0sysspace.cc:924
#10 0x80e1b59f in srv_start (create_new_db=false) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/srv/srv0start.cc:1583
#11 0x80c51cca in innodb_init (p=0x842bcb38) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/handler/ha_innodb.cc:4266
#12 0x80923110 in ha_initialize_handlerton (plugin=0x84260cb0) at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/handler.cc:557
#13 0x8060fb86 in plugin_initialize (tmp_root=0xbf95cb88, plugin=0x84260cb0, argc=0x81b8e0b4 <remaining_argc>, argv=0x840fa128, options_only=false) at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/sql_plugin.cc:1438
#14 0x80610724 in plugin_init (argc=0x81b8e0b4 <remaining_argc>, argv=0x840fa128, flags=0) at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/sql_plugin.cc:1720
#15 0x804cdcd5 in init_server_components () at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/mysqld.cc:5203
#16 0x804cf1b5 in mysqld_main (argc=137, argv=0x840fa128) at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/mysqld.cc:5726
#17 0x804c34fb in main (argc=9, argv=0xbf95d874) at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/main.cc:25
 
Thread 9 (Thread 0xb2c3cb40 (LWP 4430)):
#0  0xb778fc31 in __kernel_vsyscall ()
#1  0xb73f2e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x810bac4c in safe_cond_timedwait (cond=0x82404100 <COND_timer>, mp=0x82404080 <LOCK_timer>, abstime=0xb2c3c250, file=0x81543f10 "/home/buildbot/buildbot/build/mariadb-10.4.7/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.7/mysys/thr_mutex.c:546
#3  0x810bbb59 in inline_mysql_cond_timedwait (that=0x82404100 <COND_timer>, mutex=0x82404080 <LOCK_timer>, abstime=0xb2c3c250, src_file=0x81543f60 "/home/buildbot/buildbot/build/mariadb-10.4.7/mysys/thr_timer.c", src_line=292) at /home/buildbot/buildbot/build/mariadb-10.4.7/include/mysql/psi/mysql_thread.h:1204
#4  0x810bc6c9 in timer_handler (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.7/mysys/thr_timer.c:292
#5  0x80b5e064 in pfs_spawn_thread (arg=0x8423e328) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/perfschema/pfs.cc:1862
#6  0xb73ed2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb712916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 8 (Thread 0xafdfeb40 (LWP 4432)):
#0  0xb775b570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb775b5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d23a8f in LinuxAIOHandler::collect (this=0xafdfe0e0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1809
#3  0x80d24089 in LinuxAIOHandler::poll (this=0xafdfe0e0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1968
#4  0x80d241ed in os_aio_linux_handler (global_segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:2022
#5  0x80d290e5 in os_aio_handler (segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:5561
#6  0x80f6d284 in fil_aio_wait (segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4355
#7  0x80e17243 in io_handler_thread (arg=0x823ef6e0 <n>) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/srv/srv0start.cc:324
#8  0xb73ed2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb712916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 7 (Thread 0xaf5fdb40 (LWP 4433)):
#0  0xb775b570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb775b5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d23a8f in LinuxAIOHandler::collect (this=0xaf5fd0e0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1809
#3  0x80d24089 in LinuxAIOHandler::poll (this=0xaf5fd0e0, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1968
#4  0x80d241ed in os_aio_linux_handler (global_segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:2022
#5  0x80d290e5 in os_aio_handler (segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:5561
#6  0x80f6d284 in fil_aio_wait (segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4355
#7  0x80e17243 in io_handler_thread (arg=0x823ef6e4 <n+4>) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/srv/srv0start.cc:324
#8  0xb73ed2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb712916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 6 (Thread 0xaedfcb40 (LWP 4434)):
#0  0xb775b570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb775b5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d23a8f in LinuxAIOHandler::collect (this=0xaedfc0e0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1809
#3  0x80d24089 in LinuxAIOHandler::poll (this=0xaedfc0e0, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1968
#4  0x80d241ed in os_aio_linux_handler (global_segment=2, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:2022
#5  0x80d290e5 in os_aio_handler (segment=2, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:5561
#6  0x80f6d284 in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4355
#7  0x80e17243 in io_handler_thread (arg=0x823ef6e8 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/srv/srv0start.cc:324
#8  0xb73ed2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb712916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 5 (Thread 0xae5fbb40 (LWP 4435)):
#0  0xb775b570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb775b5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d23a8f in LinuxAIOHandler::collect (this=0xae5fb0e0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1809
#3  0x80d24089 in LinuxAIOHandler::poll (this=0xae5fb0e0, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1968
#4  0x80d241ed in os_aio_linux_handler (global_segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:2022
#5  0x80d290e5 in os_aio_handler (segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:5561
#6  0x80f6d284 in fil_aio_wait (segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4355
#7  0x80e17243 in io_handler_thread (arg=0x823ef6ec <n+12>) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/srv/srv0start.cc:324
#8  0xb73ed2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb712916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 4 (Thread 0xaddfab40 (LWP 4436)):
#0  0xb775b570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb775b5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d23a8f in LinuxAIOHandler::collect (this=0xaddfa0e0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1809
#3  0x80d24089 in LinuxAIOHandler::poll (this=0xaddfa0e0, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1968
#4  0x80d241ed in os_aio_linux_handler (global_segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:2022
#5  0x80d290e5 in os_aio_handler (segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:5561
#6  0x80f6d284 in fil_aio_wait (segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4355
#7  0x80e17243 in io_handler_thread (arg=0x823ef6f0 <n+16>) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/srv/srv0start.cc:324
#8  0xb73ed2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb712916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 3 (Thread 0xacdf8b40 (LWP 4438)):
#0  0xb778fc31 in __kernel_vsyscall ()
#1  0xb73f2a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80d31854 in os_event::wait (this=0x843ff698) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0event.cc:158
#3  0x80d312b5 in os_event::wait_low (this=0x843ff698, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0event.cc:325
#4  0x80d315fb in os_event_wait_low (event=0x843ff698, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0event.cc:502
#5  0x80efbd6a in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/buf/buf0flu.cc:3045
#6  0xb73ed2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb712916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 2 (Thread 0xb17ffb40 (LWP 4431)):
#0  0xb778fc31 in __kernel_vsyscall ()
#1  0xb73f2e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x810bac4c in safe_cond_timedwait (cond=0x823bb6e0 <COND_checkpoint>, mp=0x823bb660 <LOCK_checkpoint>, abstime=0xb17ff19c, file=0x813b9ae0 "/home/buildbot/buildbot/build/mariadb-10.4.7/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.7/mysys/thr_mutex.c:546
#3  0x80bbe4ac in inline_mysql_cond_timedwait (that=0x823bb6e0 <COND_checkpoint>, mutex=0x823bb660 <LOCK_checkpoint>, abstime=0xb17ff19c, src_file=0x813b9b30 "/home/buildbot/buildbot/build/mariadb-10.4.7/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/buildbot/build/mariadb-10.4.7/include/mysql/psi/mysql_thread.h:1204
#4  0x80bbe9ef in my_service_thread_sleep (control=0x819dc338 <checkpoint_control>, sleep_time=29000000000) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/maria/ma_servicethread.c:115
#5  0x80bb28e9 in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/maria/ma_checkpoint.c:707
#6  0x80b5e064 in pfs_spawn_thread (arg=0x842bcb38) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/perfschema/pfs.cc:1862
#7  0xb73ed2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#8  0xb712916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 1 (Thread 0xad5f9b40 (LWP 4437)):
#0  0x810b189a in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.7/mysys/my_read.c:63
#1  0x8091ef8f in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/signal_handler.cc:66
#2  0x8091f783 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb778fc31 in __kernel_vsyscall ()
#5  0xb706de89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb706f3e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb7066d07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb7066d8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x80f6fa62 in Check::validate (space=0x84447738) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4567
#10 0x80f6db68 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4598
#11 0x80f5fcf4 in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:216
#12 0x80f6d233 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4353
#13 0x80e17243 in io_handler_thread (arg=0x823ef6f4 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/srv/srv0start.cc:324
#14 0xb73ed2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb712916e in clone () from /lib/i386-linux-gnu/libc.so.6



 Comments   
Comment by Elena Stepanova [ 2019-07-30 ]

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/18799

10.4 83d368a062f017bd28536639cb2041a6

binlog_encryption.multisource 'innodb'   w2 [ fail ]
        Test ended at 2019-07-29 10:45:44
 
CURRENT_TEST: binlog_encryption.multisource
 
 
Failed to start mysqld.2
mysqltest failed but provided no output
 
 
 - found 'core' (0/0)
 
Trying 'dbx' to get a backtrace
 
Trying 'gdb' to get a backtrace from coredump /mnt/buildbot/build/mariadb-10.4.7/mysql-test/var/2/log/binlog_encryption.multisource-innodb/mysqld.2/data/core
Core generated by '/mnt/buildbot/build/mariadb-10.4.7/sql/mysqld'
Output from gdb follows. The first stack trace is from the failing thread.
The following stack traces are from all threads (so the failing one is
duplicated).
--------------------------
[New LWP 2266]
[New LWP 2248]
[New LWP 2230]
[New LWP 2256]
[New LWP 2261]
[New LWP 2262]
[New LWP 2263]
[New LWP 2264]
[New LWP 2265]
[New LWP 2267]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `/mnt/buildbot/build/mariadb-10.4.7/sql/mysqld --defaults-group-suffix=.2 --defa'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x8102720c in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.7/mysys/my_read.c:63
63	      int got_errno= my_errno= errno;
[Current thread is 1 (Thread 0xad5f9b40 (LWP 2266))]
#0  0x8102720c in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.7/mysys/my_read.c:63
#1  0x8089471d in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/signal_handler.cc:66
#2  0x80894f02 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb772bc31 in __kernel_vsyscall ()
#5  0xb7009e89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb700b3e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb7002d07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb7002d8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x80ee53d6 in Check::validate (space=0x8376c8a8) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4567
#10 0x80ee34dc in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4598
#11 0x80ed5668 in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:216
#12 0x80ee2ba7 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4353
#13 0x80d8cba1 in io_handler_thread (arg=0x823656f4 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/srv/srv0start.cc:324
#14 0xb73892b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb70c516e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 10 (Thread 0xacdf8b40 (LWP 2267)):
#0  0xb772bc31 in __kernel_vsyscall ()
#1  0xb738ea8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80ca7176 in os_event::wait (this=0x837247b0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0event.cc:158
#3  0x80ca6bd7 in os_event::wait_low (this=0x837247b0, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0event.cc:325
#4  0x80ca6f1d in os_event_wait_low (event=0x837247b0, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0event.cc:502
#5  0x80e716de in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/buf/buf0flu.cc:3045
#6  0xb73892b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb70c516e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 9 (Thread 0xaddfab40 (LWP 2265)):
#0  0xb76f7570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76f75e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80c993b1 in LinuxAIOHandler::collect (this=0xaddfa0e0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1809
#3  0x80c999ab in LinuxAIOHandler::poll (this=0xaddfa0e0, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1968
#4  0x80c99b0f in os_aio_linux_handler (global_segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:2022
#5  0x80c9ea07 in os_aio_handler (segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:5561
#6  0x80ee2bf8 in fil_aio_wait (segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4355
#7  0x80d8cba1 in io_handler_thread (arg=0x823656f0 <n+16>) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/srv/srv0start.cc:324
#8  0xb73892b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb70c516e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 8 (Thread 0xae5fbb40 (LWP 2264)):
#0  0xb76f7570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76f75e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80c993b1 in LinuxAIOHandler::collect (this=0xae5fb0e0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1809
#3  0x80c999ab in LinuxAIOHandler::poll (this=0xae5fb0e0, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1968
#4  0x80c99b0f in os_aio_linux_handler (global_segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:2022
#5  0x80c9ea07 in os_aio_handler (segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:5561
#6  0x80ee2bf8 in fil_aio_wait (segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4355
#7  0x80d8cba1 in io_handler_thread (arg=0x823656ec <n+12>) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/srv/srv0start.cc:324
#8  0xb73892b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb70c516e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 7 (Thread 0xaedfcb40 (LWP 2263)):
#0  0xb76f7570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76f75e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80c993b1 in LinuxAIOHandler::collect (this=0xaedfc0e0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1809
#3  0x80c999ab in LinuxAIOHandler::poll (this=0xaedfc0e0, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1968
#4  0x80c99b0f in os_aio_linux_handler (global_segment=2, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:2022
#5  0x80c9ea07 in os_aio_handler (segment=2, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:5561
#6  0x80ee2bf8 in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4355
#7  0x80d8cba1 in io_handler_thread (arg=0x823656e8 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/srv/srv0start.cc:324
#8  0xb73892b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb70c516e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 6 (Thread 0xaf5fdb40 (LWP 2262)):
#0  0xb76f7570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76f75e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80c993b1 in LinuxAIOHandler::collect (this=0xaf5fd0e0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1809
#3  0x80c999ab in LinuxAIOHandler::poll (this=0xaf5fd0e0, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1968
#4  0x80c99b0f in os_aio_linux_handler (global_segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:2022
#5  0x80c9ea07 in os_aio_handler (segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:5561
#6  0x80ee2bf8 in fil_aio_wait (segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4355
#7  0x80d8cba1 in io_handler_thread (arg=0x823656e4 <n+4>) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/srv/srv0start.cc:324
#8  0xb73892b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb70c516e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 5 (Thread 0xafdfeb40 (LWP 2261)):
#0  0xb76f7570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76f75e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80c993b1 in LinuxAIOHandler::collect (this=0xafdfe0e0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1809
#3  0x80c999ab in LinuxAIOHandler::poll (this=0xafdfe0e0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:1968
#4  0x80c99b0f in os_aio_linux_handler (global_segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:2022
#5  0x80c9ea07 in os_aio_handler (segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0file.cc:5561
#6  0x80ee2bf8 in fil_aio_wait (segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4355
#7  0x80d8cba1 in io_handler_thread (arg=0x823656e0 <n>) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/srv/srv0start.cc:324
#8  0xb73892b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb70c516e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 4 (Thread 0xb1743b40 (LWP 2256)):
#0  0xb772bc31 in __kernel_vsyscall ()
#1  0xb738ee56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x810305be in safe_cond_timedwait (cond=0x823316e0 <COND_checkpoint>, mp=0x82331660 <LOCK_checkpoint>, abstime=0xb174319c, file=0x8132f1c0 "/home/buildbot/buildbot/build/mariadb-10.4.7/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.7/mysys/thr_mutex.c:546
#3  0x80b33dce in inline_mysql_cond_timedwait (that=0x823316e0 <COND_checkpoint>, mutex=0x82331660 <LOCK_checkpoint>, abstime=0xb174319c, src_file=0x8132f210 "/home/buildbot/buildbot/build/mariadb-10.4.7/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/buildbot/build/mariadb-10.4.7/include/mysql/psi/mysql_thread.h:1204
#4  0x80b34311 in my_service_thread_sleep (control=0x81952338 <checkpoint_control>, sleep_time=29000000000) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/maria/ma_servicethread.c:115
#5  0x80b2820b in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/maria/ma_checkpoint.c:707
#6  0x80ad3986 in pfs_spawn_thread (arg=0x835e1cc0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/perfschema/pfs.cc:1862
#7  0xb73892b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#8  0xb70c516e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 3 (Thread 0xb6e4b7c0 (LWP 2230)):
#0  0xb772bc31 in __kernel_vsyscall ()
#1  0xb738ea8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80ca7176 in os_event::wait (this=0x836684c0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0event.cc:158
#3  0x80ca6bd7 in os_event::wait_low (this=0x836684c0, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0event.cc:325
#4  0x80ca6f1d in os_event_wait_low (event=0x836684c0, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/os/os0event.cc:502
#5  0x80d9542d in sync_array_wait_event (arr=0x835f8650, cell=@0xbfd33c00: 0x835f86e0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/sync/sync0arr.cc:471
#6  0x80bef2cf in TTASEventMutex<GenericPolicy>::enter (this=0x8195c870 <fil_system+16>, max_spins=60, max_delay=4, filename=0x81465fb8 "/home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc", line=4586) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/include/ib0mutex.h:478
#7  0x80bed5cc in PolicyMutex<TTASEventMutex<GenericPolicy> >::enter (this=0x8195c870 <fil_system+16>, n_spins=30, n_delay=4, name=0x81465fb8 "/home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc", line=4586) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/include/ib0mutex.h:595
#8  0x80ee348b in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4586
#9  0x80f047d4 in SysTablespace::open_or_create (this=0x82372980 <srv_sys_space>, is_temp=false, create_new_db=false, sum_new_sizes=0xbfd33d88, flush_lsn=0xbfd33dd8) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fsp/fsp0sysspace.cc:924
#10 0x80d90efd in srv_start (create_new_db=false) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/srv/srv0start.cc:1583
#11 0x80bc75ec in innodb_init (p=0x835e1cc0) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/handler/ha_innodb.cc:4266
#12 0x80898890 in ha_initialize_handlerton (plugin=0x83587670) at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/handler.cc:557
#13 0x805851ac in plugin_initialize (tmp_root=0xbfd38fa8, plugin=0x83587670, argc=0x81b040b4 <remaining_argc>, argv=0x83423ea8, options_only=false) at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/sql_plugin.cc:1438
#14 0x80585d4a in plugin_init (argc=0x81b040b4 <remaining_argc>, argv=0x83423ea8, flags=0) at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/sql_plugin.cc:1720
#15 0x80442d85 in init_server_components () at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/mysqld.cc:5203
#16 0x80444265 in mysqld_main (argc=150, argv=0x83423ea8) at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/mysqld.cc:5726
#17 0x804385ab in main (argc=23, argv=0xbfd39c94) at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/main.cc:25
 
Thread 2 (Thread 0xb2bd8b40 (LWP 2248)):
#0  0xb772bc31 in __kernel_vsyscall ()
#1  0xb738ee56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x810305be in safe_cond_timedwait (cond=0x8237a100 <COND_timer>, mp=0x8237a080 <LOCK_timer>, abstime=0xb2bd8250, file=0x814b95f0 "/home/buildbot/buildbot/build/mariadb-10.4.7/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.7/mysys/thr_mutex.c:546
#3  0x810314cb in inline_mysql_cond_timedwait (that=0x8237a100 <COND_timer>, mutex=0x8237a080 <LOCK_timer>, abstime=0xb2bd8250, src_file=0x814b9640 "/home/buildbot/buildbot/build/mariadb-10.4.7/mysys/thr_timer.c", src_line=292) at /home/buildbot/buildbot/build/mariadb-10.4.7/include/mysql/psi/mysql_thread.h:1204
#4  0x8103203b in timer_handler (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.7/mysys/thr_timer.c:292
#5  0x80ad3986 in pfs_spawn_thread (arg=0x83564d58) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/perfschema/pfs.cc:1862
#6  0xb73892b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb70c516e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 1 (Thread 0xad5f9b40 (LWP 2266)):
#0  0x8102720c in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.7/mysys/my_read.c:63
#1  0x8089471d in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/signal_handler.cc:66
#2  0x80894f02 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.7/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb772bc31 in __kernel_vsyscall ()
#5  0xb7009e89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb700b3e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb7002d07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb7002d8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x80ee53d6 in Check::validate (space=0x8376c8a8) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4567
#10 0x80ee34dc in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4598
#11 0x80ed5668 in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:216
#12 0x80ee2ba7 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/fil/fil0fil.cc:4353
#13 0x80d8cba1 in io_handler_thread (arg=0x823656f4 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.7/storage/innobase/srv/srv0start.cc:324
#14 0xb73892b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb70c516e in clone () from /lib/i386-linux-gnu/libc.so.6

Comment by Elena Stepanova [ 2019-09-08 ]

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/18983/steps/mtr_ps/logs/stdio

10.4 136cf0400fedebf1b8521e77b016a096

rpl.rpl_lcase_tblnames_rewrite_db 'mix'  w2 [ fail ]
        Test ended at 2019-08-15 14:11:30
 
CURRENT_TEST: rpl.rpl_lcase_tblnames_rewrite_db
 
 
Failed to start mysqld.2
mysqltest failed but provided no output
 
 
 - found 'core' (0/0)
 
Trying 'dbx' to get a backtrace
 
Trying 'gdb' to get a backtrace from coredump /mnt/buildbot/build/mariadb-10.4.8/mysql-test/var/2/log/rpl.rpl_lcase_tblnames_rewrite_db-mix/mysqld.2/data/core
Core generated by '/mnt/buildbot/build/mariadb-10.4.8/sql/mysqld'
Output from gdb follows. The first stack trace is from the failing thread.
The following stack traces are from all threads (so the failing one is
duplicated).
--------------------------
[New LWP 24825]
[New LWP 24811]
[New LWP 24824]
[New LWP 24826]
[New LWP 24823]
[New LWP 24822]
[New LWP 24821]
[New LWP 24820]
[New LWP 24818]
[New LWP 24819]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `/mnt/buildbot/build/mariadb-10.4.8/sql/mysqld --defaults-group-suffix=.2 --defa'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x81033b9c in my_read (Filedes=7, Buffer=0xad6f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.8/mysys/my_read.c:63
63	      int got_errno= my_errno= errno;
[Current thread is 1 (Thread 0xad6f9b40 (LWP 24825))]
#0  0x81033b9c in my_read (Filedes=7, Buffer=0xad6f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.8/mysys/my_read.c:63
#1  0x808a0bdb in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/signal_handler.cc:66
#2  0x808a13c0 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb77acc31 in __kernel_vsyscall ()
#5  0xb708ae89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb708c3e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb7083d07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb7083d8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x80ef1cf6 in Check::validate (space=0x830c0f08) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4567
#10 0x80eefdfc in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4598
#11 0x80ee1f88 in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:216
#12 0x80eef4c7 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4353
#13 0x80d99271 in io_handler_thread (arg=0x823746f4 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/srv/srv0start.cc:324
#14 0xb740a2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb714616e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 10 (Thread 0xb180eb40 (LWP 24819)):
#0  0xb77acc31 in __kernel_vsyscall ()
#1  0xb740fe56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x8103cf4e in safe_cond_timedwait (cond=0x823406e0 <COND_checkpoint>, mp=0x82340660 <LOCK_checkpoint>, abstime=0xb180e19c, file=0x8133dac0 "/home/buildbot/buildbot/build/mariadb-10.4.8/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.8/mysys/thr_mutex.c:546
#3  0x80b40530 in inline_mysql_cond_timedwait (that=0x823406e0 <COND_checkpoint>, mutex=0x82340660 <LOCK_checkpoint>, abstime=0xb180e19c, src_file=0x8133db10 "/home/buildbot/buildbot/build/mariadb-10.4.8/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/buildbot/build/mariadb-10.4.8/include/mysql/psi/mysql_thread.h:1204
#4  0x80b40a73 in my_service_thread_sleep (control=0x81961358 <checkpoint_control>, sleep_time=29000000000) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/maria/ma_servicethread.c:115
#5  0x80b3496d in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/maria/ma_checkpoint.c:707
#6  0x80ae00e8 in pfs_spawn_thread (arg=0x82f362e8) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/perfschema/pfs.cc:1862
#7  0xb740a2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#8  0xb714616e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 9 (Thread 0xb2c59b40 (LWP 24818)):
#0  0xb77acc31 in __kernel_vsyscall ()
#1  0xb740fe56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x8103cf4e in safe_cond_timedwait (cond=0x82389100 <COND_timer>, mp=0x82389080 <LOCK_timer>, abstime=0xb2c59250, file=0x814c8510 "/home/buildbot/buildbot/build/mariadb-10.4.8/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.8/mysys/thr_mutex.c:546
#3  0x8103de5b in inline_mysql_cond_timedwait (that=0x82389100 <COND_timer>, mutex=0x82389080 <LOCK_timer>, abstime=0xb2c59250, src_file=0x814c8560 "/home/buildbot/buildbot/build/mariadb-10.4.8/mysys/thr_timer.c", src_line=292) at /home/buildbot/buildbot/build/mariadb-10.4.8/include/mysql/psi/mysql_thread.h:1204
#4  0x8103e9cb in timer_handler (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.8/mysys/thr_timer.c:292
#5  0x80ae00e8 in pfs_spawn_thread (arg=0x82eb9380) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/perfschema/pfs.cc:1862
#6  0xb740a2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb714616e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 8 (Thread 0xafefeb40 (LWP 24820)):
#0  0xb7778570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb77785e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80ca5fff in LinuxAIOHandler::collect (this=0xafefe0e0) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1809
#3  0x80ca65f9 in LinuxAIOHandler::poll (this=0xafefe0e0, m1=0xafefe150, m2=0xafefe154, request=0xafefe178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1968
#4  0x80ca675d in os_aio_linux_handler (global_segment=0, m1=0xafefe150, m2=0xafefe154, request=0xafefe178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:2022
#5  0x80cab655 in os_aio_handler (segment=0, m1=0xafefe150, m2=0xafefe154, request=0xafefe178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:5561
#6  0x80eef518 in fil_aio_wait (segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4355
#7  0x80d99271 in io_handler_thread (arg=0x823746e0 <n>) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/srv/srv0start.cc:324
#8  0xb740a2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb714616e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 7 (Thread 0xaf6fdb40 (LWP 24821)):
#0  0xb7778570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb77785e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80ca5fff in LinuxAIOHandler::collect (this=0xaf6fd0e0) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1809
#3  0x80ca65f9 in LinuxAIOHandler::poll (this=0xaf6fd0e0, m1=0xaf6fd150, m2=0xaf6fd154, request=0xaf6fd178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1968
#4  0x80ca675d in os_aio_linux_handler (global_segment=1, m1=0xaf6fd150, m2=0xaf6fd154, request=0xaf6fd178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:2022
#5  0x80cab655 in os_aio_handler (segment=1, m1=0xaf6fd150, m2=0xaf6fd154, request=0xaf6fd178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:5561
#6  0x80eef518 in fil_aio_wait (segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4355
#7  0x80d99271 in io_handler_thread (arg=0x823746e4 <n+4>) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/srv/srv0start.cc:324
#8  0xb740a2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb714616e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 6 (Thread 0xaeefcb40 (LWP 24822)):
#0  0xb7778570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb77785e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80ca5fff in LinuxAIOHandler::collect (this=0xaeefc0e0) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1809
#3  0x80ca65f9 in LinuxAIOHandler::poll (this=0xaeefc0e0, m1=0xaeefc150, m2=0xaeefc154, request=0xaeefc178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1968
#4  0x80ca675d in os_aio_linux_handler (global_segment=2, m1=0xaeefc150, m2=0xaeefc154, request=0xaeefc178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:2022
#5  0x80cab655 in os_aio_handler (segment=2, m1=0xaeefc150, m2=0xaeefc154, request=0xaeefc178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:5561
#6  0x80eef518 in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4355
#7  0x80d99271 in io_handler_thread (arg=0x823746e8 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/srv/srv0start.cc:324
#8  0xb740a2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb714616e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 5 (Thread 0xae6fbb40 (LWP 24823)):
#0  0xb7778570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb77785e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80ca5fff in LinuxAIOHandler::collect (this=0xae6fb0e0) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1809
#3  0x80ca65f9 in LinuxAIOHandler::poll (this=0xae6fb0e0, m1=0xae6fb150, m2=0xae6fb154, request=0xae6fb178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1968
#4  0x80ca675d in os_aio_linux_handler (global_segment=3, m1=0xae6fb150, m2=0xae6fb154, request=0xae6fb178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:2022
#5  0x80cab655 in os_aio_handler (segment=3, m1=0xae6fb150, m2=0xae6fb154, request=0xae6fb178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:5561
#6  0x80eef518 in fil_aio_wait (segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4355
#7  0x80d99271 in io_handler_thread (arg=0x823746ec <n+12>) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/srv/srv0start.cc:324
#8  0xb740a2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb714616e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 4 (Thread 0xacef8b40 (LWP 24826)):
#0  0xb77acc31 in __kernel_vsyscall ()
#1  0xb740fa8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80cb3dc4 in os_event::wait (this=0x83078e10) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0event.cc:158
#3  0x80cb3825 in os_event::wait_low (this=0x83078e10, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0event.cc:325
#4  0x80cb3b6b in os_event_wait_low (event=0x83078e10, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0event.cc:502
#5  0x80e7dffe in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/buf/buf0flu.cc:3045
#6  0xb740a2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb714616e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 3 (Thread 0xadefab40 (LWP 24824)):
#0  0xb7778570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb77785e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80ca5fff in LinuxAIOHandler::collect (this=0xadefa0e0) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1809
#3  0x80ca65f9 in LinuxAIOHandler::poll (this=0xadefa0e0, m1=0xadefa150, m2=0xadefa154, request=0xadefa178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1968
#4  0x80ca675d in os_aio_linux_handler (global_segment=4, m1=0xadefa150, m2=0xadefa154, request=0xadefa178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:2022
#5  0x80cab655 in os_aio_handler (segment=4, m1=0xadefa150, m2=0xadefa154, request=0xadefa178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:5561
#6  0x80eef518 in fil_aio_wait (segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4355
#7  0x80d99271 in io_handler_thread (arg=0x823746f0 <n+16>) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/srv/srv0start.cc:324
#8  0xb740a2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb714616e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 2 (Thread 0xb6ecc7c0 (LWP 24811)):
#0  0xb77acc31 in __kernel_vsyscall ()
#1  0xb740fa8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80cb3dc4 in os_event::wait (this=0x82fbcb20) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0event.cc:158
#3  0x80cb3825 in os_event::wait_low (this=0x82fbcb20, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0event.cc:325
#4  0x80cb3b6b in os_event_wait_low (event=0x82fbcb20, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0event.cc:502
#5  0x80da1aef in sync_array_wait_event (arr=0x82f4cc78, cell=@0xbf9014c0: 0x82f4cd08) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/sync/sync0arr.cc:471
#6  0x80bfba11 in TTASEventMutex<GenericPolicy>::enter (this=0x8196b870 <fil_system+16>, max_spins=60, max_delay=4, filename=0x81474d44 "/home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc", line=4586) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/include/ib0mutex.h:478
#7  0x80bf9d0e in PolicyMutex<TTASEventMutex<GenericPolicy> >::enter (this=0x8196b870 <fil_system+16>, n_spins=30, n_delay=4, name=0x81474d44 "/home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc", line=4586) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/include/ib0mutex.h:595
#8  0x80eefdab in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4586
#9  0x80f110f4 in SysTablespace::open_or_create (this=0x82381980 <srv_sys_space>, is_temp=false, create_new_db=false, sum_new_sizes=0xbf901648, flush_lsn=0xbf901698) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fsp/fsp0sysspace.cc:924
#10 0x80d9d5cd in srv_start (create_new_db=false) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/srv/srv0start.cc:1583
#11 0x80bd3d2e in innodb_init (p=0x82f362e8) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/handler/ha_innodb.cc:4258
#12 0x808a4d4e in ha_initialize_handlerton (plugin=0x82edbc98) at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/handler.cc:557
#13 0x80590d64 in plugin_initialize (tmp_root=0xbf906868, plugin=0x82edbc98, argc=0x81b130b4 <remaining_argc>, argv=0x82d75128, options_only=false) at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/sql_plugin.cc:1438
#14 0x80591902 in plugin_init (argc=0x81b130b4 <remaining_argc>, argv=0x82d75128, flags=0) at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/sql_plugin.cc:1720
#15 0x8044e825 in init_server_components () at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/mysqld.cc:5203
#16 0x8044fd05 in mysqld_main (argc=143, argv=0x82d75128) at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/mysqld.cc:5726
#17 0x8044404b in main (argc=10, argv=0xbf907554) at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/main.cc:25
 
Thread 1 (Thread 0xad6f9b40 (LWP 24825)):
#0  0x81033b9c in my_read (Filedes=7, Buffer=0xad6f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.8/mysys/my_read.c:63
#1  0x808a0bdb in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/signal_handler.cc:66
#2  0x808a13c0 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb77acc31 in __kernel_vsyscall ()
#5  0xb708ae89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb708c3e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb7083d07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb7083d8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x80ef1cf6 in Check::validate (space=0x830c0f08) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4567
#10 0x80eefdfc in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4598
#11 0x80ee1f88 in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:216
#12 0x80eef4c7 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4353
#13 0x80d99271 in io_handler_thread (arg=0x823746f4 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/srv/srv0start.cc:324
#14 0xb740a2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb714616e in clone () from /lib/i386-linux-gnu/libc.so.6

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/19218

10.4 29b13d114e4c4a8a880ddf052351f7bf

rpl.rpl_domain_id_filter 'innodb,stmt'   w1 [ fail ]
        Test ended at 2019-09-01 19:22:19
 
CURRENT_TEST: rpl.rpl_domain_id_filter
 
 
Failed to start mysqld.2
mysqltest failed but provided no output
 
 
 - found 'core' (0/0)
 
Trying 'dbx' to get a backtrace
 
Trying 'gdb' to get a backtrace from coredump /mnt/buildbot/build/mariadb-10.4.8/mysql-test/var/1/log/rpl.rpl_domain_id_filter-innodb,stmt/mysqld.2/data/core
Core generated by '/mnt/buildbot/build/mariadb-10.4.8/sql/mysqld'
Output from gdb follows. The first stack trace is from the failing thread.
The following stack traces are from all threads (so the failing one is
duplicated).
--------------------------
[New LWP 26986]
[New LWP 26962]
[New LWP 26975]
[New LWP 26981]
[New LWP 26982]
[New LWP 26983]
[New LWP 26984]
[New LWP 26985]
[New LWP 26987]
[New LWP 26976]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `/mnt/buildbot/build/mariadb-10.4.8/sql/mysqld --defaults-group-suffix=.2 --defa'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x81154308 in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.8/mysys/my_read.c:63
63	      int got_errno= my_errno= errno;
[Current thread is 1 (Thread 0xad5f9b40 (LWP 26986))]
#0  0x81154308 in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.8/mysys/my_read.c:63
#1  0x808d65cf in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/signal_handler.cc:66
#2  0x808d6dc3 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb7741c31 in __kernel_vsyscall ()
#5  0xb701fe89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb70213e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb7018d07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb7018d8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x80ff58ea in Check::validate (space=0x846faf48) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4573
#10 0x80ff33a7 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4604
#11 0x80fe2e74 in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:216
#12 0x80ff2884 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4359
#13 0x80e5bac8 in io_handler_thread (arg=0x824ad6f4 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/srv/srv0start.cc:324
#14 0xb769b2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb70db16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 10 (Thread 0xb17ccb40 (LWP 26976)):
#0  0xb7741c31 in __kernel_vsyscall ()
#1  0xb76a0e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x8115da0e in safe_cond_timedwait (cond=0x824796e0 <COND_checkpoint>, mp=0x82479660 <LOCK_checkpoint>, abstime=0xb17cc19c, file=0x8146c7bc "/home/buildbot/buildbot/build/mariadb-10.4.8/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.8/mysys/thr_mutex.c:546
#3  0x80ba2488 in inline_mysql_cond_timedwait (that=0x824796e0 <COND_checkpoint>, mutex=0x82479660 <LOCK_checkpoint>, abstime=0xb17cc19c, src_file=0x8146c80c "/home/buildbot/buildbot/build/mariadb-10.4.8/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/buildbot/build/mariadb-10.4.8/include/mysql/psi/mysql_thread.h:1204
#4  0x80ba2a03 in my_service_thread_sleep (control=0x81a9a3b8 <checkpoint_control>, sleep_time=29000000000) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/maria/ma_servicethread.c:115
#5  0x80b95ee0 in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/maria/ma_checkpoint.c:707
#6  0x80b37889 in pfs_spawn_thread (arg=0x84570270) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/perfschema/pfs.cc:1862
#7  0xb769b2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#8  0xb70db16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 9 (Thread 0xacdf8b40 (LWP 26987)):
#0  0xb7741c31 in __kernel_vsyscall ()
#1  0xb76a0a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80d42b9c in os_event::wait (this=0x846b2e50) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0event.cc:158
#3  0x80d425fd in os_event::wait_low (this=0x846b2e50, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0event.cc:325
#4  0x80d42943 in os_event_wait_low (event=0x846b2e50, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0event.cc:502
#5  0x80f6c978 in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/buf/buf0flu.cc:3045
#6  0xb769b2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb70db16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 8 (Thread 0xaddfab40 (LWP 26985)):
#0  0xb770d570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb770d5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d33da0 in LinuxAIOHandler::collect (this=0xaddfa0e0) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1809
#3  0x80d34410 in LinuxAIOHandler::poll (this=0xaddfa0e0, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1968
#4  0x80d34574 in os_aio_linux_handler (global_segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:2022
#5  0x80d39771 in os_aio_handler (segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:5561
#6  0x80ff2910 in fil_aio_wait (segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4361
#7  0x80e5bac8 in io_handler_thread (arg=0x824ad6f0 <n+16>) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/srv/srv0start.cc:324
#8  0xb769b2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb70db16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 7 (Thread 0xae5fbb40 (LWP 26984)):
#0  0xb770d570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb770d5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d33da0 in LinuxAIOHandler::collect (this=0xae5fb0e0) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1809
#3  0x80d34410 in LinuxAIOHandler::poll (this=0xae5fb0e0, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1968
#4  0x80d34574 in os_aio_linux_handler (global_segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:2022
#5  0x80d39771 in os_aio_handler (segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:5561
#6  0x80ff2910 in fil_aio_wait (segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4361
#7  0x80e5bac8 in io_handler_thread (arg=0x824ad6ec <n+12>) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/srv/srv0start.cc:324
#8  0xb769b2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb70db16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 6 (Thread 0xaedfcb40 (LWP 26983)):
#0  0xb770d570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb770d5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d33da0 in LinuxAIOHandler::collect (this=0xaedfc0e0) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1809
#3  0x80d34410 in LinuxAIOHandler::poll (this=0xaedfc0e0, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1968
#4  0x80d34574 in os_aio_linux_handler (global_segment=2, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:2022
#5  0x80d39771 in os_aio_handler (segment=2, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:5561
#6  0x80ff2910 in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4361
#7  0x80e5bac8 in io_handler_thread (arg=0x824ad6e8 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/srv/srv0start.cc:324
#8  0xb769b2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb70db16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 5 (Thread 0xaf5fdb40 (LWP 26982)):
#0  0xb770d570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb770d5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d33da0 in LinuxAIOHandler::collect (this=0xaf5fd0e0) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1809
#3  0x80d34410 in LinuxAIOHandler::poll (this=0xaf5fd0e0, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1968
#4  0x80d34574 in os_aio_linux_handler (global_segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:2022
#5  0x80d39771 in os_aio_handler (segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:5561
#6  0x80ff2910 in fil_aio_wait (segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4361
#7  0x80e5bac8 in io_handler_thread (arg=0x824ad6e4 <n+4>) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/srv/srv0start.cc:324
#8  0xb769b2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb70db16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 4 (Thread 0xafdfeb40 (LWP 26981)):
#0  0xb770d570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb770d5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d33da0 in LinuxAIOHandler::collect (this=0xafdfe0e0) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1809
#3  0x80d34410 in LinuxAIOHandler::poll (this=0xafdfe0e0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:1968
#4  0x80d34574 in os_aio_linux_handler (global_segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:2022
#5  0x80d39771 in os_aio_handler (segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0file.cc:5561
#6  0x80ff2910 in fil_aio_wait (segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4361
#7  0x80e5bac8 in io_handler_thread (arg=0x824ad6e0 <n>) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/srv/srv0start.cc:324
#8  0xb769b2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb70db16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 3 (Thread 0xb2beeb40 (LWP 26975)):
#0  0xb7741c31 in __kernel_vsyscall ()
#1  0xb76a0e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x8115da0e in safe_cond_timedwait (cond=0x824c2100 <COND_timer>, mp=0x824c2080 <LOCK_timer>, abstime=0xb2bee250, file=0x815fc1e4 "/home/buildbot/buildbot/build/mariadb-10.4.8/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.8/mysys/thr_mutex.c:546
#3  0x8115ea49 in inline_mysql_cond_timedwait (that=0x824c2100 <COND_timer>, mutex=0x824c2080 <LOCK_timer>, abstime=0xb2bee250, src_file=0x815fc234 "/home/buildbot/buildbot/build/mariadb-10.4.8/mysys/thr_timer.c", src_line=292) at /home/buildbot/buildbot/build/mariadb-10.4.8/include/mysql/psi/mysql_thread.h:1204
#4  0x8115f66a in timer_handler (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.8/mysys/thr_timer.c:292
#5  0x80b37889 in pfs_spawn_thread (arg=0x844f3350) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/perfschema/pfs.cc:1862
#6  0xb769b2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb70db16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 2 (Thread 0xb6e617c0 (LWP 26962)):
#0  0xb7741c31 in __kernel_vsyscall ()
#1  0xb76a0a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80d42b9c in os_event::wait (this=0x845f6b60) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0event.cc:158
#3  0x80d425fd in os_event::wait_low (this=0x845f6b60, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0event.cc:325
#4  0x80d42943 in os_event_wait_low (event=0x845f6b60, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/os/os0event.cc:502
#5  0x80e64e05 in sync_array_wait_event (arr=0x84586c90, cell=@0xbfc33ff0: 0x84586d20) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/sync/sync0arr.cc:471
#6  0x80c685ad in TTASEventMutex<GenericPolicy>::enter (this=0x81aa4870 <fil_system+16>, max_spins=60, max_delay=4, filename=0x815a6f70 "/home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc", line=4592) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/include/ib0mutex.h:478
#7  0x80c66710 in PolicyMutex<TTASEventMutex<GenericPolicy> >::enter (this=0x81aa4870 <fil_system+16>, n_spins=30, n_delay=4, name=0x815a6f70 "/home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc", line=4592) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/include/ib0mutex.h:595
#8  0x80ff3356 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4592
#9  0x810193d6 in SysTablespace::open_or_create (this=0x824ba980 <srv_sys_space>, is_temp=false, create_new_db=false, sum_new_sizes=0xbfc34178, flush_lsn=0xbfc341c8) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fsp/fsp0sysspace.cc:924
#10 0x80e60012 in srv_start (create_new_db=false) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/srv/srv0start.cc:1583
#11 0x80c3d5e6 in innodb_init (p=0x84570270) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/handler/ha_innodb.cc:4259
#12 0x808dabc6 in ha_initialize_handlerton (plugin=0x84515c20) at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/handler.cc:557
#13 0x805a3dc7 in plugin_initialize (tmp_root=0xbfc39398, plugin=0x84515c20, argc=0x81c4c0b4 <remaining_argc>, argv=0x843af128, options_only=false) at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/sql_plugin.cc:1438
#14 0x805a4a55 in plugin_init (argc=0x81c4c0b4 <remaining_argc>, argv=0x843af128, flags=0) at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/sql_plugin.cc:1720
#15 0x80451c1c in init_server_components () at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/mysqld.cc:5203
#16 0x80453194 in mysqld_main (argc=158, argv=0x843af128) at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/mysqld.cc:5726
#17 0x80446f2b in main (argc=25, argv=0xbfc3a084) at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/main.cc:25
 
Thread 1 (Thread 0xad5f9b40 (LWP 26986)):
#0  0x81154308 in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.8/mysys/my_read.c:63
#1  0x808d65cf in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/signal_handler.cc:66
#2  0x808d6dc3 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.8/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb7741c31 in __kernel_vsyscall ()
#5  0xb701fe89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb70213e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb7018d07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb7018d8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x80ff58ea in Check::validate (space=0x846faf48) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4573
#10 0x80ff33a7 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4604
#11 0x80fe2e74 in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:216
#12 0x80ff2884 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/fil/fil0fil.cc:4359
#13 0x80e5bac8 in io_handler_thread (arg=0x824ad6f4 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.8/storage/innobase/srv/srv0start.cc:324
#14 0xb769b2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb70db16e in clone () from /lib/i386-linux-gnu/libc.so.6

Comment by Elena Stepanova [ 2019-11-03 ]

10.4 bf617c36540de99cc3597d339ff80dcf

rpl.rpl_dual_pos_advance 'innodb,stmt'   w3 [ fail ]
        Test ended at 2019-09-18 05:31:41
 
CURRENT_TEST: rpl.rpl_dual_pos_advance
 
 
Failed to start mysqld.1
mysqltest failed but provided no output
 
 
 - found 'core' (0/0)
 
Trying 'dbx' to get a backtrace
 
Trying 'gdb' to get a backtrace from coredump /mnt/buildbot/build/mariadb-10.4.9/mysql-test/var/3/log/rpl.rpl_dual_pos_advance-innodb,stmt/mysqld.1/data/core
Core generated by '/mnt/buildbot/build/mariadb-10.4.9/sql/mysqld'
Output from gdb follows. The first stack trace is from the failing thread.
The following stack traces are from all threads (so the failing one is
duplicated).
--------------------------
[New LWP 12813]
[New LWP 12802]
[New LWP 12812]
[New LWP 12811]
[New LWP 12810]
[New LWP 12809]
[New LWP 12808]
[New LWP 12814]
[New LWP 12804]
[New LWP 12793]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `/mnt/buildbot/build/mariadb-10.4.9/sql/mysqld --defaults-group-suffix=.1 --defa'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x81188004 in my_read (Filedes=7, Buffer=0xad4f690c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.9/mysys/my_read.c:63
63	      int got_errno= my_errno= errno;
[Current thread is 1 (Thread 0xad4f8b40 (LWP 12813))]
#0  0x81188004 in my_read (Filedes=7, Buffer=0xad4f690c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.9/mysys/my_read.c:63
#1  0x8090909f in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/signal_handler.cc:66
#2  0x80909893 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb7704c31 in __kernel_vsyscall ()
#5  0xb6fe2e89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb6fe43e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb6fdbd07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb6fdbd8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x810295d2 in Check::validate (space=0x8419c988) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4573
#10 0x81027059 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4604
#11 0x81016b26 in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:216
#12 0x81026536 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4359
#13 0x80e8ef2e in io_handler_thread (arg=0x824e3734 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/srv/srv0start.cc:324
#14 0xb765e2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb709e16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 10 (Thread 0xb6e247c0 (LWP 12793)):
#0  0xb7704c31 in __kernel_vsyscall ()
#1  0xb7663a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80d765e4 in os_event::wait (this=0x840985f8) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0event.cc:158
#3  0x80d76045 in os_event::wait_low (this=0x840985f8, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0event.cc:325
#4  0x80d7638b in os_event_wait_low (event=0x840985f8, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0event.cc:502
#5  0x80e9826b in sync_array_wait_event (arr=0x840287b8, cell=@0xbfa5d820: 0x84028848) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/sync/sync0arr.cc:471
#6  0x80c9c581 in TTASEventMutex<GenericPolicy>::enter (this=0x81ada870 <fil_system+16>, max_spins=60, max_delay=4, filename=0x815db188 "/home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc", line=4592) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/include/ib0mutex.h:478
#7  0x80c9a6e4 in PolicyMutex<TTASEventMutex<GenericPolicy> >::enter (this=0x81ada870 <fil_system+16>, n_spins=30, n_delay=4, name=0x815db188 "/home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc", line=4592) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/include/ib0mutex.h:595
#8  0x81027008 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4592
#9  0x8104d0be in SysTablespace::open_or_create (this=0x824f09c0 <srv_sys_space>, is_temp=false, create_new_db=false, sum_new_sizes=0xbfa5d9a8, flush_lsn=0xbfa5d9f8) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fsp/fsp0sysspace.cc:924
#10 0x80e93478 in srv_start (create_new_db=false) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/srv/srv0start.cc:1583
#11 0x80c715ba in innodb_init (p=0x84011e48) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/handler/ha_innodb.cc:4259
#12 0x8090d696 in ha_initialize_handlerton (plugin=0x83fb76d8) at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/handler.cc:557
#13 0x805d5d7f in plugin_initialize (tmp_root=0xbfa62bc8, plugin=0x83fb76d8, argc=0x81c820b4 <remaining_argc>, argv=0x83e53e68, options_only=false) at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/sql_plugin.cc:1438
#14 0x805d6a0d in plugin_init (argc=0x81c820b4 <remaining_argc>, argv=0x83e53e68, flags=0) at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/sql_plugin.cc:1720
#15 0x80483935 in init_server_components () at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/mysqld.cc:5205
#16 0x80484ead in mysqld_main (argc=149, argv=0x83e53e68) at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/mysqld.cc:5728
#17 0x80478c3b in main (argc=25, argv=0xbfa638b4) at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/main.cc:25
 
Thread 9 (Thread 0xb14ffb40 (LWP 12804)):
#0  0xb7704c31 in __kernel_vsyscall ()
#1  0xb7663e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x8119170a in safe_cond_timedwait (cond=0x824af720 <COND_checkpoint>, mp=0x824af6a0 <LOCK_checkpoint>, abstime=0xb14ff19c, file=0x814a077c "/home/buildbot/buildbot/build/mariadb-10.4.9/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.9/mysys/thr_mutex.c:546
#3  0x80bd6520 in inline_mysql_cond_timedwait (that=0x824af720 <COND_checkpoint>, mutex=0x824af6a0 <LOCK_checkpoint>, abstime=0xb14ff19c, src_file=0x814a07cc "/home/buildbot/buildbot/build/mariadb-10.4.9/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/buildbot/build/mariadb-10.4.9/include/mysql/psi/mysql_thread.h:1204
#4  0x80bd6a9b in my_service_thread_sleep (control=0x81ad03b8 <checkpoint_control>, sleep_time=29000000000) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/maria/ma_servicethread.c:115
#5  0x80bc9f78 in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/maria/ma_checkpoint.c:707
#6  0x80b6b829 in pfs_spawn_thread (arg=0x84011e48) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/perfschema/pfs.cc:1862
#7  0xb765e2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#8  0xb709e16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 8 (Thread 0xaccf7b40 (LWP 12814)):
#0  0xb7704c31 in __kernel_vsyscall ()
#1  0xb7663a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80d765e4 in os_event::wait (this=0x841548e8) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0event.cc:158
#3  0x80d76045 in os_event::wait_low (this=0x841548e8, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0event.cc:325
#4  0x80d7638b in os_event_wait_low (event=0x841548e8, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0event.cc:502
#5  0x80fa06f0 in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/buf/buf0flu.cc:3045
#6  0xb765e2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb709e16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 7 (Thread 0xafcfdb40 (LWP 12808)):
#0  0xb76d0570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76d05e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d67870 in LinuxAIOHandler::collect (this=0xafcfd0e0) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1809
#3  0x80d67ee0 in LinuxAIOHandler::poll (this=0xafcfd0e0, m1=0xafcfd150, m2=0xafcfd154, request=0xafcfd178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1968
#4  0x80d68044 in os_aio_linux_handler (global_segment=0, m1=0xafcfd150, m2=0xafcfd154, request=0xafcfd178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:2022
#5  0x80d6d241 in os_aio_handler (segment=0, m1=0xafcfd150, m2=0xafcfd154, request=0xafcfd178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:5561
#6  0x810265c2 in fil_aio_wait (segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4361
#7  0x80e8ef2e in io_handler_thread (arg=0x824e3720 <n>) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/srv/srv0start.cc:324
#8  0xb765e2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb709e16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 6 (Thread 0xaf4fcb40 (LWP 12809)):
#0  0xb76d0570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76d05e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d67870 in LinuxAIOHandler::collect (this=0xaf4fc0e0) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1809
#3  0x80d67ee0 in LinuxAIOHandler::poll (this=0xaf4fc0e0, m1=0xaf4fc150, m2=0xaf4fc154, request=0xaf4fc178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1968
#4  0x80d68044 in os_aio_linux_handler (global_segment=1, m1=0xaf4fc150, m2=0xaf4fc154, request=0xaf4fc178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:2022
#5  0x80d6d241 in os_aio_handler (segment=1, m1=0xaf4fc150, m2=0xaf4fc154, request=0xaf4fc178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:5561
#6  0x810265c2 in fil_aio_wait (segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4361
#7  0x80e8ef2e in io_handler_thread (arg=0x824e3724 <n+4>) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/srv/srv0start.cc:324
#8  0xb765e2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb709e16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 5 (Thread 0xaecfbb40 (LWP 12810)):
#0  0xb76d0570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76d05e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d67870 in LinuxAIOHandler::collect (this=0xaecfb0e0) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1809
#3  0x80d67ee0 in LinuxAIOHandler::poll (this=0xaecfb0e0, m1=0xaecfb150, m2=0xaecfb154, request=0xaecfb178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1968
#4  0x80d68044 in os_aio_linux_handler (global_segment=2, m1=0xaecfb150, m2=0xaecfb154, request=0xaecfb178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:2022
#5  0x80d6d241 in os_aio_handler (segment=2, m1=0xaecfb150, m2=0xaecfb154, request=0xaecfb178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:5561
#6  0x810265c2 in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4361
#7  0x80e8ef2e in io_handler_thread (arg=0x824e3728 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/srv/srv0start.cc:324
#8  0xb765e2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb709e16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 4 (Thread 0xae4fab40 (LWP 12811)):
#0  0xb76d0570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76d05e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d67870 in LinuxAIOHandler::collect (this=0xae4fa0e0) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1809
#3  0x80d67ee0 in LinuxAIOHandler::poll (this=0xae4fa0e0, m1=0xae4fa150, m2=0xae4fa154, request=0xae4fa178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1968
#4  0x80d68044 in os_aio_linux_handler (global_segment=3, m1=0xae4fa150, m2=0xae4fa154, request=0xae4fa178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:2022
#5  0x80d6d241 in os_aio_handler (segment=3, m1=0xae4fa150, m2=0xae4fa154, request=0xae4fa178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:5561
#6  0x810265c2 in fil_aio_wait (segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4361
#7  0x80e8ef2e in io_handler_thread (arg=0x824e372c <n+12>) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/srv/srv0start.cc:324
#8  0xb765e2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb709e16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 3 (Thread 0xadcf9b40 (LWP 12812)):
#0  0xb76d0570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76d05e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d67870 in LinuxAIOHandler::collect (this=0xadcf90e0) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1809
#3  0x80d67ee0 in LinuxAIOHandler::poll (this=0xadcf90e0, m1=0xadcf9150, m2=0xadcf9154, request=0xadcf9178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1968
#4  0x80d68044 in os_aio_linux_handler (global_segment=4, m1=0xadcf9150, m2=0xadcf9154, request=0xadcf9178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:2022
#5  0x80d6d241 in os_aio_handler (segment=4, m1=0xadcf9150, m2=0xadcf9154, request=0xadcf9178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:5561
#6  0x810265c2 in fil_aio_wait (segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4361
#7  0x80e8ef2e in io_handler_thread (arg=0x824e3730 <n+16>) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/srv/srv0start.cc:324
#8  0xb765e2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb709e16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 2 (Thread 0xb2bb1b40 (LWP 12802)):
#0  0xb7704c31 in __kernel_vsyscall ()
#1  0xb7663e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x8119170a in safe_cond_timedwait (cond=0x824f8140 <COND_timer>, mp=0x824f80c0 <LOCK_timer>, abstime=0xb2bb1250, file=0x8163039c "/home/buildbot/buildbot/build/mariadb-10.4.9/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.9/mysys/thr_mutex.c:546
#3  0x81192745 in inline_mysql_cond_timedwait (that=0x824f8140 <COND_timer>, mutex=0x824f80c0 <LOCK_timer>, abstime=0xb2bb1250, src_file=0x816303ec "/home/buildbot/buildbot/build/mariadb-10.4.9/mysys/thr_timer.c", src_line=292) at /home/buildbot/buildbot/build/mariadb-10.4.9/include/mysql/psi/mysql_thread.h:1204
#4  0x81193366 in timer_handler (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.9/mysys/thr_timer.c:292
#5  0x80b6b829 in pfs_spawn_thread (arg=0x83f94d08) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/perfschema/pfs.cc:1862
#6  0xb765e2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb709e16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 1 (Thread 0xad4f8b40 (LWP 12813)):
#0  0x81188004 in my_read (Filedes=7, Buffer=0xad4f690c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.9/mysys/my_read.c:63
#1  0x8090909f in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/signal_handler.cc:66
#2  0x80909893 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb7704c31 in __kernel_vsyscall ()
#5  0xb6fe2e89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb6fe43e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb6fdbd07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb6fdbd8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x810295d2 in Check::validate (space=0x8419c988) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4573
#10 0x81027059 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4604
#11 0x81016b26 in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:216
#12 0x81026536 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4359
#13 0x80e8ef2e in io_handler_thread (arg=0x824e3734 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/srv/srv0start.cc:324
#14 0xb765e2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb709e16e in clone () from /lib/i386-linux-gnu/libc.so.6

Comment by Elena Stepanova [ 2019-11-03 ]

10.4 a39c4b5f9f7f4d2766f1ddb9487de272

innodb_zip.restart '4k,innodb,strict_full_crc32' w3 [ fail ]
        Test ended at 2019-09-24 08:14:39
 
CURRENT_TEST: innodb_zip.restart
 
worker[3] > Restart  - not started
 
Server [mysqld.1 - pid: 32695, winpid: 32695, exit: 256] failed during test run
Server log from this test:
----------SERVER LOG START-----------
...
2019-09-24  8:14:33 0 [Note] InnoDB: Completed initialization of buffer pool
2019-09-24  8:14:33 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
mysqld: /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4573: static ulint Check::validate(const fil_space_t*): Assertion `space->id != TRX_SYS_SPACE || space == fil_system.sys_space' failed.
190924  8:14:33 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 10.4.9-MariaDB-debug-log
key_buffer_size=1048576
read_buffer_size=131072
max_used_connections=0
max_threads=153
thread_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 62208 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x49000
/mnt/buildbot/build/mariadb-10.4.9/sql/mysqld(my_print_stacktrace+0x3c)[0x8114995c]
/mnt/buildbot/build/mariadb-10.4.9/sql/mysqld(handle_fatal_signal+0x391)[0x808d56bd]
addr2line: '': No such file
[0xb7702c14]
[0xb7702c31]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x39)[0xb6fe0e89]
/lib/i386-linux-gnu/libc.so.6(abort+0x157)[0xb6fe23e7]
/lib/i386-linux-gnu/libc.so.6(+0x24d07)[0xb6fd9d07]
/lib/i386-linux-gnu/libc.so.6(+0x24d8b)[0xb6fd9d8b]
/mnt/buildbot/build/mariadb-10.4.9/sql/mysqld(+0xf77a8f)[0x80fe8a8f]
/mnt/buildbot/build/mariadb-10.4.9/sql/mysqld(+0xf7554e)[0x80fe654e]
/mnt/buildbot/build/mariadb-10.4.9/sql/mysqld(+0xf652da)[0x80fd62da]
fil/fil0fil.cc:4575(Check::validate(fil_space_t const*))[0x80fe5a6e]
fil/fil0fil.cc:4604(fil_validate())[0x80e51604]
/lib/i386-linux-gnu/libpthread.so.0(+0x62b5)[0xb765c2b5]
/lib/i386-linux-gnu/libc.so.6(clone+0x6e)[0xb709c16e]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Writing a core file...
Working directory at /dev/shm/var_auto_tpAR/3/mysqld.1/data
Resource Limits:
Fatal signal 11 while backtracing
----------SERVER LOG END-------------
mysqltest failed but provided no output
The result from queries just before the failure was:
< snip >
t66_restart.par
t77_restart#p#p0#sp#s0.isl
t77_restart#p#p0#sp#s1.isl
t77_restart#p#p1#sp#s2.isl
t77_restart#p#p1#sp#s3.isl
t77_restart.frm
t77_restart.par
---- MYSQL_TMP_DIR/alt_dir/test
t55_restart.ibd
t66_restart#p#p0.ibd
t66_restart#p#p1.ibd
t66_restart#p#p2.ibd
t77_restart#p#p0#sp#s0.ibd
t77_restart#p#p0#sp#s1.ibd
t77_restart#p#p1#sp#s2.ibd
t77_restart#p#p1#sp#s3.ibd
#
# Restart the server
#
# restart
 
 
 
 - found 'core' (0/0)
 
Trying 'dbx' to get a backtrace
 
Trying 'gdb' to get a backtrace from coredump /mnt/buildbot/build/mariadb-10.4.9/mysql-test/var/3/log/innodb_zip.restart-4k,innodb,strict_full_crc32/mysqld.1/data/core
Core generated by '/mnt/buildbot/build/mariadb-10.4.9/sql/mysqld'
Output from gdb follows. The first stack trace is from the failing thread.
The following stack traces are from all threads (so the failing one is
duplicated).
--------------------------
[New LWP 32726]
[New LWP 32696]
[New LWP 32717]
[New LWP 32721]
[New LWP 32722]
[New LWP 32723]
[New LWP 32724]
[New LWP 32725]
[New LWP 32727]
[New LWP 32715]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `/mnt/buildbot/build/mariadb-10.4.9/sql/mysqld --defaults-group-suffix=.1 --defa'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x811453d2 in my_read (Filedes=6, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.9/mysys/my_read.c:63
63	      int got_errno= my_errno= errno;
[Current thread is 1 (Thread 0xad5f9b40 (LWP 32726))]
#0  0x811453d2 in my_read (Filedes=6, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.9/mysys/my_read.c:63
#1  0x808d5251 in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/signal_handler.cc:66
#2  0x808d5a36 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb7702c31 in __kernel_vsyscall ()
#5  0xb6fe0e89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb6fe23e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb6fd9d07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb6fd9d8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x80fe8a8f in Check::validate (space=0x834d5248) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4573
#10 0x80fe654e in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4604
#11 0x80fd62da in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:216
#12 0x80fe5a6e in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4359
#13 0x80e51604 in io_handler_thread (arg=0x82496734 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/srv/srv0start.cc:324
#14 0xb765c2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb709c16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 10 (Thread 0xb2bafb40 (LWP 32715)):
#0  0xb7702c31 in __kernel_vsyscall ()
#1  0xb7661e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x8114e9a2 in safe_cond_timedwait (cond=0x824ab140 <COND_timer>, mp=0x824ab0c0 <LOCK_timer>, abstime=0xb2baf250, file=0x815eb9d0 "/home/buildbot/buildbot/build/mariadb-10.4.9/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.9/mysys/thr_mutex.c:546
#3  0x8114f954 in inline_mysql_cond_timedwait (that=0x824ab140 <COND_timer>, mutex=0x824ab0c0 <LOCK_timer>, abstime=0xb2baf250, src_file=0x815eba20 "/home/buildbot/buildbot/build/mariadb-10.4.9/mysys/thr_timer.c", src_line=292) at /home/buildbot/buildbot/build/mariadb-10.4.9/include/mysql/psi/mysql_thread.h:1204
#4  0x81150533 in timer_handler (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.9/mysys/thr_timer.c:292
#5  0x80b34d4f in pfs_spawn_thread (arg=0x831fdd08) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/perfschema/pfs.cc:1862
#6  0xb765c2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb709c16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 9 (Thread 0xacdf8b40 (LWP 32727)):
#0  0xb7702c31 in __kernel_vsyscall ()
#1  0xb7661a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80d3a024 in os_event::wait (this=0x834764b8) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0event.cc:158
#3  0x80d39a85 in os_event::wait_low (this=0x834764b8, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0event.cc:325
#4  0x80d39dcb in os_event_wait_low (event=0x834764b8, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0event.cc:502
#5  0x80f60a44 in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/buf/buf0flu.cc:3045
#6  0xb765c2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb709c16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 8 (Thread 0xaddfab40 (LWP 32725)):
#0  0xb76ce570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76ce5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d2b407 in LinuxAIOHandler::collect (this=0xaddfa0e0) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1809
#3  0x80d2ba6d in LinuxAIOHandler::poll (this=0xaddfa0e0, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1968
#4  0x80d2bbd1 in os_aio_linux_handler (global_segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:2022
#5  0x80d30d5f in os_aio_handler (segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:5561
#6  0x80fe5aef in fil_aio_wait (segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4361
#7  0x80e51604 in io_handler_thread (arg=0x82496730 <n+16>) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/srv/srv0start.cc:324
#8  0xb765c2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb709c16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 7 (Thread 0xae5fbb40 (LWP 32724)):
#0  0xb76ce570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76ce5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d2b407 in LinuxAIOHandler::collect (this=0xae5fb0e0) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1809
#3  0x80d2ba6d in LinuxAIOHandler::poll (this=0xae5fb0e0, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1968
#4  0x80d2bbd1 in os_aio_linux_handler (global_segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:2022
#5  0x80d30d5f in os_aio_handler (segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:5561
#6  0x80fe5aef in fil_aio_wait (segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4361
#7  0x80e51604 in io_handler_thread (arg=0x8249672c <n+12>) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/srv/srv0start.cc:324
#8  0xb765c2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb709c16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 6 (Thread 0xaedfcb40 (LWP 32723)):
#0  0xb76ce570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76ce5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d2b407 in LinuxAIOHandler::collect (this=0xaedfc0e0) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1809
#3  0x80d2ba6d in LinuxAIOHandler::poll (this=0xaedfc0e0, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1968
#4  0x80d2bbd1 in os_aio_linux_handler (global_segment=2, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:2022
#5  0x80d30d5f in os_aio_handler (segment=2, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:5561
#6  0x80fe5aef in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4361
#7  0x80e51604 in io_handler_thread (arg=0x82496728 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/srv/srv0start.cc:324
#8  0xb765c2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb709c16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 5 (Thread 0xaf5fdb40 (LWP 32722)):
#0  0xb76ce570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76ce5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d2b407 in LinuxAIOHandler::collect (this=0xaf5fd0e0) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1809
#3  0x80d2ba6d in LinuxAIOHandler::poll (this=0xaf5fd0e0, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1968
#4  0x80d2bbd1 in os_aio_linux_handler (global_segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:2022
#5  0x80d30d5f in os_aio_handler (segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:5561
#6  0x80fe5aef in fil_aio_wait (segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4361
#7  0x80e51604 in io_handler_thread (arg=0x82496724 <n+4>) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/srv/srv0start.cc:324
#8  0xb765c2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb709c16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 4 (Thread 0xafdfeb40 (LWP 32721)):
#0  0xb76ce570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76ce5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d2b407 in LinuxAIOHandler::collect (this=0xafdfe0e0) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1809
#3  0x80d2ba6d in LinuxAIOHandler::poll (this=0xafdfe0e0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:1968
#4  0x80d2bbd1 in os_aio_linux_handler (global_segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:2022
#5  0x80d30d5f in os_aio_handler (segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0file.cc:5561
#6  0x80fe5aef in fil_aio_wait (segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4361
#7  0x80e51604 in io_handler_thread (arg=0x82496720 <n>) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/srv/srv0start.cc:324
#8  0xb765c2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb709c16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 3 (Thread 0xb1743b40 (LWP 32717)):
#0  0xb7702c31 in __kernel_vsyscall ()
#1  0xb7661e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x8114e9a2 in safe_cond_timedwait (cond=0x82462720 <COND_checkpoint>, mp=0x824626a0 <LOCK_checkpoint>, abstime=0xb174319c, file=0x8145c458 "/home/buildbot/buildbot/build/mariadb-10.4.9/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.9/mysys/thr_mutex.c:546
#3  0x80b9d854 in inline_mysql_cond_timedwait (that=0x82462720 <COND_checkpoint>, mutex=0x824626a0 <LOCK_checkpoint>, abstime=0xb174319c, src_file=0x8145c4a8 "/home/buildbot/buildbot/build/mariadb-10.4.9/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/buildbot/build/mariadb-10.4.9/include/mysql/psi/mysql_thread.h:1204
#4  0x80b9ddb9 in my_service_thread_sleep (control=0x81a833b8 <checkpoint_control>, sleep_time=29000000000) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/maria/ma_servicethread.c:115
#5  0x80b9161a in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/maria/ma_checkpoint.c:707
#6  0x80b34d4f in pfs_spawn_thread (arg=0x83278ac8) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/perfschema/pfs.cc:1862
#7  0xb765c2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#8  0xb709c16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 2 (Thread 0xb6e227c0 (LWP 32696)):
#0  0xb7702c31 in __kernel_vsyscall ()
#1  0xb7661a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80d3a024 in os_event::wait (this=0x832ff488) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0event.cc:158
#3  0x80d39a85 in os_event::wait_low (this=0x832ff488, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0event.cc:325
#4  0x80d39dcb in os_event_wait_low (event=0x832ff488, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/os/os0event.cc:502
#5  0x80e5a85a in sync_array_wait_event (arr=0x8328f610, cell=@0xbf974300: 0x8328f6a0) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/sync/sync0arr.cc:471
#6  0x80c61a05 in TTASEventMutex<GenericPolicy>::enter (this=0x81a8d870 <fil_system+16>, max_spins=60, max_delay=4, filename=0x81596c40 "/home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc", line=4592) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/include/ib0mutex.h:478
#7  0x80c5fb68 in PolicyMutex<TTASEventMutex<GenericPolicy> >::enter (this=0x81a8d870 <fil_system+16>, n_spins=30, n_delay=4, name=0x81596c40 "/home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc", line=4592) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/include/ib0mutex.h:595
#8  0x80fe64fd in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4592
#9  0x8100c43e in SysTablespace::open_or_create (this=0x824a39c0 <srv_sys_space>, is_temp=false, create_new_db=false, sum_new_sizes=0xbf974488, flush_lsn=0xbf9744d8) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fsp/fsp0sysspace.cc:924
#10 0x80e55b0a in srv_start (create_new_db=false) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/srv/srv0start.cc:1583
#11 0x80c36c78 in innodb_init (p=0x83278ac8) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/handler/ha_innodb.cc:4259
#12 0x808d9806 in ha_initialize_handlerton (plugin=0x8321e3e8) at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/handler.cc:557
#13 0x805a4e17 in plugin_initialize (tmp_root=0xbf9796a8, plugin=0x8321e3e8, argc=0x81c350b4 <remaining_argc>, argv=0x830bce88, options_only=false) at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/sql_plugin.cc:1438
#14 0x805a5aa9 in plugin_init (argc=0x81c350b4 <remaining_argc>, argv=0x830bce88, flags=0) at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/sql_plugin.cc:1720
#15 0x804538a7 in init_server_components () at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/mysqld.cc:5205
#16 0x80454de9 in mysqld_main (argc=153, argv=0x830bce88) at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/mysqld.cc:5728
#17 0x80448c0b in main (argc=29, argv=0xbf97a394) at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/main.cc:25
 
Thread 1 (Thread 0xad5f9b40 (LWP 32726)):
#0  0x811453d2 in my_read (Filedes=6, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.9/mysys/my_read.c:63
#1  0x808d5251 in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/signal_handler.cc:66
#2  0x808d5a36 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.9/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb7702c31 in __kernel_vsyscall ()
#5  0xb6fe0e89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb6fe23e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb6fd9d07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb6fd9d8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x80fe8a8f in Check::validate (space=0x834d5248) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4573
#10 0x80fe654e in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4604
#11 0x80fd62da in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:216
#12 0x80fe5a6e in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4359
#13 0x80e51604 in io_handler_thread (arg=0x82496734 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/srv/srv0start.cc:324
#14 0xb765c2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb709c16e in clone () from /lib/i386-linux-gnu/libc.so.6

Comment by Elena Stepanova [ 2019-11-03 ]

http://buildbot.askmonty.org/buildbot/builders/kvm-asan/builds/2263

10.4 82f22d2f2538fd53eae071a3694c7c99

innodb.innodb_prefix_index_restart_server 'innodb' w3 [ fail ]
        Test ended at 2019-10-23 04:43:40
 
CURRENT_TEST: innodb.innodb_prefix_index_restart_server
...
2019-10-23  4:43:39 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
mysqld: /home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/fil/fil0fil.cc:4571: static ulint Check::validate(const fil_space_t*): Assertion `space->id != 0xFFFFFFFEU || space == fil_system.temp_space' failed.
191023  4:43:39 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 10.4.9-MariaDB-debug-log
key_buffer_size=1048576
read_buffer_size=131072
max_used_connections=0
max_threads=153
thread_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63617 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x5fc00
/usr/lib/x86_64-linux-gnu/libasan.so.2(+0x4a077)[0x7f5a3aa47077]
/home/buildbot/buildbot/build/mariadb-10.4.9/sql/mysqld(my_print_stacktrace+0xce)[0x23252d7]
dict/dict0crea.cc:1223(dict_create_index_step(que_thr_t*))[0x1133002]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f5a39b72390]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7f5a3867b428]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7f5a3867d02a]
/lib/x86_64-linux-gnu/libc.so.6(+0x2dbd7)[0x7f5a38673bd7]
/lib/x86_64-linux-gnu/libc.so.6(+0x2dc82)[0x7f5a38673c82]
/home/buildbot/buildbot/build/mariadb-10.4.9/sql/mysqld[0x1fd0961]
row/row0ftsort.cc:1227(row_merge_write_fts_word(fts_psort_insert*, fts_tokenizer_word_t*))[0x1fcc315]
row/row0ftsort.cc:595(row_merge_fts_doc_tokenize(row_merge_buf_t**, unsigned long, fts_doc_t*, merge_file_t**, unsigned long, fts_tokenize_ctx*))[0x1fb1b77]
rem/rem0rec.cc:906(rec_get_offsets_func(unsigned char const*, dict_index_t const*, unsigned long*, bool, unsigned long, char const*, unsigned int, mem_block_info_t**))[0x1fcadf2]
include/ut0new.h:348(ut_allocator<std::_Rb_tree_node<std::pair<unsigned long const, file_name_t> >, true>::allocate(unsigned long, std::_Rb_tree_node<std::pair<unsigned long const, file_name_t> > const*, char const*, bool, bool))[0x1f06950]
log/log0crypt.cc:92(init_crypt_key(crypt_info_t*, bool))[0x1f07a9e]
lock/lock0lock.cc:3810(lock_table_other_has_incompatible(trx_t const*, unsigned long, dict_table_t const*, lock_mode))[0x1eacd50]
handler/i_s.cc:5872(i_s_common_deinit(void*))[0x1e55364]
handler/i_s.cc:6052(i_s_sys_tables_fill_table(THD*, TABLE_LIST*, Item*))[0x1e56143]
handler/i_s.cc:6072(i_s_sys_tables_fill_table(THD*, TABLE_LIST*, Item*))[0x1e562ba]
include/row0row.ic:73(row_get_rec_trx_id(unsigned char const*, dict_index_t const*, unsigned long const*))[0x20bdafd]
include/eval0eval.ic:73(eval_node_ensure_val_buf(void*, unsigned long))[0x20bde8c]
maria/ma_check.c:1110(isam_key_length)[0x1cace76]
maria/ma_check.c:1120(record_pos_to_txt)[0x1cacfce]
maria/ma_sort.c:739(write_keys)[0x1ce0c3f]
row/row0sel.cc:4387(row_search_mvcc(unsigned char*, page_cur_mode_t, row_prebuilt_t*, unsigned long, unsigned long))[0x20d376c]
row/row0sel.cc:4470(row_search_mvcc(unsigned char*, page_cur_mode_t, row_prebuilt_t*, unsigned long, unsigned long))[0x20d3bc6]
maria/ma_sort.c:1083(merge_buffers)[0x1ce34d9]
maria/ma_sort.c:1007(merge_buffers)[0x1ce39fd]
maria/ma_pagecache.c:2375(pagecache_wait_lock)[0x1b99123]
maria/ma_pagecache.c:2421(get_wrlock)[0x1b995ae]
maria/ma_pagecache.c:2421(get_wrlock)[0x1b99965]
handler/ha_innodb.cc:20734(innobase_get_computed_value(dtuple_t*, dict_v_col_t const*, dict_index_t const*, mem_block_info_t**, mem_block_info_t*, dict_field_t const*, THD*, TABLE*, unsigned char*, dict_table_t const*, upd_t*, dict_foreign_t*))[0x1d990a9]
handler/ha_innodb.cc:20853(ha_innobase::can_convert_varstring(Field_varstring const*, Column_definition const&) const)[0x1d9a2ec]
handler/ha_innodb.cc:20900(ha_innobase::can_convert_blob(Field_blob const*, Column_definition const&) const)[0x1d9aa1e]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7f5a39b686ba]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f5a3874c82d]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Writing a core file...
Working directory at /dev/shm/var/3/mysqld.1/data
Resource Limits:
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        0                    0                    bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             23715                23715                processes 
Max open files            1024                 1024                 files     
Max locked memory         65536                65536                bytes     
Max address space         unlimited            unlimited            bytes     
Max file locks            unlimited            unlimited            locks     
Max pending signals       23715                23715                signals   
Max msgqueue size         819200               819200               bytes     
Max nice priority         0                    0                    
Max realtime priority     0                    0                    
Max realtime timeout      unlimited            unlimited            us        
Core pattern: |/usr/share/apport/apport %p %s %c %P

Comment by Elena Stepanova [ 2019-11-03 ]

At least a dozen different MTR tests have already failed due to this problem, and new ones keep showing up.

Comment by Elena Stepanova [ 2019-12-10 ]

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/20177

10.4 89ae01fd0085cf0d1af272eca545e49f

binlog.binlog_row_binlog 'innodb,row'    w4 [ fail ]
        Test ended at 2019-11-14 12:37:13
 
CURRENT_TEST: binlog.binlog_row_binlog
 
 
Failed to start mysqld.1
mysqltest failed but provided no output
 
 
 - found 'core' (0/0)
 
Trying 'dbx' to get a backtrace
 
Trying 'gdb' to get a backtrace from coredump /mnt/buildbot/build/mariadb-10.4.11/mysql-test/var/4/log/binlog.binlog_row_binlog-innodb,row/mysqld.1/data/core
worker[2] > Restart [mysqld.1 - pid: 7616, winpid: 7616] - running with different options '--innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --log-bin=master-bin --binlog-format=row --log-bin=master-bin --loose-innodb-lock-wait-timeout=2 --default-storage-engine=MyISAM' != '--innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --binlog-format=row --log-bin=master-bin --binlog_cache_size=32768 --binlog_stmt_cache_size=32768'
Core generated by '/mnt/buildbot/build/mariadb-10.4.11/sql/mysqld'
Output from gdb follows. The first stack trace is from the failing thread.
The following stack traces are from all threads (so the failing one is
duplicated).
--------------------------
[New LWP 7708]
[New LWP 7698]
[New LWP 7711]
[New LWP 7710]
[New LWP 7709]
[New LWP 7706]
[New LWP 7712]
[New LWP 7707]
[New LWP 7701]
[New LWP 7691]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `/mnt/buildbot/build/mariadb-10.4.11/sql/mysqld --defaults-group-suffix=.1 --def'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x81160e2e in my_read (Filedes=7, Buffer=0xaedfa90c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.11/mysys/my_read.c:63
63	      int got_errno= my_errno= errno;
[Current thread is 1 (Thread 0xaedfcb40 (LWP 7708))]
#0  0x81160e2e in my_read (Filedes=7, Buffer=0xaedfa90c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.11/mysys/my_read.c:63
#1  0x808ef979 in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/signal_handler.cc:66
#2  0x808f0161 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb7786c31 in __kernel_vsyscall ()
#5  0xb7064e89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb70663e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb705dd07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb705dd8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x8100471f in Check::validate (space=0x83f67970) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4569
#10 0x810021de in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4600
#11 0x80ff1f6a in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:216
#12 0x810016fe in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4355
#13 0x80e6e688 in io_handler_thread (arg=0x824b9628 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/srv/srv0start.cc:324
#14 0xb76e02b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb712016e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 10 (Thread 0xb6ea67c0 (LWP 7691)):
#0  0xb7786c31 in __kernel_vsyscall ()
#1  0xb76e5a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80d5433c in os_event::wait (this=0x83e71cf8) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0event.cc:158
#3  0x80d53d9d in os_event::wait_low (this=0x83e71cf8, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0event.cc:325
#4  0x80d540e3 in os_event_wait_low (event=0x83e71cf8, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0event.cc:502
#5  0x80e778d4 in sync_array_wait_event (arr=0x83e01eb8, cell=@0xbfa0fd80: 0x83e01f48) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/sync/sync0arr.cc:471
#6  0x80c7d961 in TTASEventMutex<GenericPolicy>::enter (this=0x81ab06f0 <fil_system+16>, max_spins=60, max_delay=4, filename=0x815b6fc4 "/home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc", line=4588) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/include/ib0mutex.h:478
#7  0x80c7b9ce in PolicyMutex<TTASEventMutex<GenericPolicy> >::enter (this=0x81ab06f0 <fil_system+16>, n_spins=30, n_delay=4, name=0x815b6fc4 "/home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc", line=4588) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/include/ib0mutex.h:595
#8  0x8100218d in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4588
#9  0x81028028 in SysTablespace::open_or_create (this=0x824c68c0 <srv_sys_space>, is_temp=false, create_new_db=false, sum_new_sizes=0xbfa0ff10, flush_lsn=0xbfa0ff60) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fsp/fsp0sysspace.cc:924
#10 0x80e72b84 in srv_start (create_new_db=false) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/srv/srv0start.cc:1583
#11 0x80c51fc2 in innodb_init (p=0x83deb378) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/handler/ha_innodb.cc:4258
#12 0x808f3f38 in ha_initialize_handlerton (plugin=0x83d90ce0) at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/handler.cc:557
#13 0x805bd37f in plugin_initialize (tmp_root=0xbfa15128, plugin=0x83d90ce0, argc=0x81c57fb4 <remaining_argc>, argv=0x83c2a370, options_only=false) at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/sql_plugin.cc:1438
#14 0x805be011 in plugin_init (argc=0x81c57fb4 <remaining_argc>, argv=0x83c2a370, flags=0) at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/sql_plugin.cc:1720
#15 0x8046b4b1 in init_server_components () at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/mysqld.cc:5199
#16 0x8046c9f3 in mysqld_main (argc=152, argv=0x83c2a370) at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/mysqld.cc:5722
#17 0x8046082b in main (argc=28, argv=0xbfa15e14) at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/main.cc:25
 
Thread 9 (Thread 0xb17ffb40 (LWP 7701)):
#0  0xb7786c31 in __kernel_vsyscall ()
#1  0xb76e5e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x8116a3fe in safe_cond_timedwait (cond=0x82485620 <COND_checkpoint>, mp=0x824855a0 <LOCK_checkpoint>, abstime=0xb17ff19c, file=0x8147a5f8 "/home/buildbot/buildbot/build/mariadb-10.4.11/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.11/mysys/thr_mutex.c:546
#3  0x80bb846d in inline_mysql_cond_timedwait (that=0x82485620 <COND_checkpoint>, mutex=0x824855a0 <LOCK_checkpoint>, abstime=0xb17ff19c, src_file=0x8147a648 "/home/buildbot/buildbot/build/mariadb-10.4.11/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/buildbot/build/mariadb-10.4.11/include/mysql/psi/mysql_thread.h:1204
#4  0x80bb89d2 in my_service_thread_sleep (control=0x81aa53b8 <checkpoint_control>, sleep_time=29000000000) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/maria/ma_servicethread.c:115
#5  0x80bac210 in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/maria/ma_checkpoint.c:707
#6  0x80b4f81d in pfs_spawn_thread (arg=0x83deb378) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/perfschema/pfs.cc:1862
#7  0xb76e02b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#8  0xb712016e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 8 (Thread 0xaf5fdb40 (LWP 7707)):
#0  0xb7752570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb77525e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d4571f in LinuxAIOHandler::collect (this=0xaf5fd0e0) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1809
#3  0x80d45d85 in LinuxAIOHandler::poll (this=0xaf5fd0e0, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1968
#4  0x80d45ee9 in os_aio_linux_handler (global_segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:2022
#5  0x80d4b077 in os_aio_handler (segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:5565
#6  0x8100177f in fil_aio_wait (segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4357
#7  0x80e6e688 in io_handler_thread (arg=0x824b9624 <n+4>) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/srv/srv0start.cc:324
#8  0xb76e02b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb712016e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 7 (Thread 0xacdf8b40 (LWP 7712)):
#0  0xb7786c31 in __kernel_vsyscall ()
#1  0xb76e5a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80d5433c in os_event::wait (this=0x83f2dfd8) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0event.cc:158
#3  0x80d53d9d in os_event::wait_low (this=0x83f2dfd8, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0event.cc:325
#4  0x80d540e3 in os_event_wait_low (event=0x83f2dfd8, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0event.cc:502
#5  0x80f7d734 in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/buf/buf0flu.cc:3047
#6  0xb76e02b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb712016e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 6 (Thread 0xafdfeb40 (LWP 7706)):
#0  0xb7752570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb77525e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d4571f in LinuxAIOHandler::collect (this=0xafdfe0e0) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1809
#3  0x80d45d85 in LinuxAIOHandler::poll (this=0xafdfe0e0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1968
#4  0x80d45ee9 in os_aio_linux_handler (global_segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:2022
#5  0x80d4b077 in os_aio_handler (segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:5565
#6  0x8100177f in fil_aio_wait (segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4357
#7  0x80e6e688 in io_handler_thread (arg=0x824b9620 <n>) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/srv/srv0start.cc:324
#8  0xb76e02b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb712016e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 5 (Thread 0xae5fbb40 (LWP 7709)):
#0  0xb7752570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb77525e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d4571f in LinuxAIOHandler::collect (this=0xae5fb0e0) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1809
#3  0x80d45d85 in LinuxAIOHandler::poll (this=0xae5fb0e0, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1968
#4  0x80d45ee9 in os_aio_linux_handler (global_segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:2022
#5  0x80d4b077 in os_aio_handler (segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:5565
#6  0x8100177f in fil_aio_wait (segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4357
#7  0x80e6e688 in io_handler_thread (arg=0x824b962c <n+12>) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/srv/srv0start.cc:324
#8  0xb76e02b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb712016e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 4 (Thread 0xaddfab40 (LWP 7710)):
#0  0xb7752570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb77525e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d4571f in LinuxAIOHandler::collect (this=0xaddfa0e0) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1809
#3  0x80d45d85 in LinuxAIOHandler::poll (this=0xaddfa0e0, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1968
#4  0x80d45ee9 in os_aio_linux_handler (global_segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:2022
#5  0x80d4b077 in os_aio_handler (segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:5565
#6  0x8100177f in fil_aio_wait (segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4357
#7  0x80e6e688 in io_handler_thread (arg=0x824b9630 <n+16>) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/srv/srv0start.cc:324
#8  0xb76e02b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb712016e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 3 (Thread 0xad5f9b40 (LWP 7711)):
#0  0xb7752570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb77525e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d4571f in LinuxAIOHandler::collect (this=0xad5f90e0) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1809
#3  0x80d45d85 in LinuxAIOHandler::poll (this=0xad5f90e0, m1=0xad5f9150, m2=0xad5f9154, request=0xad5f9178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1968
#4  0x80d45ee9 in os_aio_linux_handler (global_segment=5, m1=0xad5f9150, m2=0xad5f9154, request=0xad5f9178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:2022
#5  0x80d4b077 in os_aio_handler (segment=5, m1=0xad5f9150, m2=0xad5f9154, request=0xad5f9178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:5565
#6  0x8100177f in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4357
#7  0x80e6e688 in io_handler_thread (arg=0x824b9634 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/srv/srv0start.cc:324
#8  0xb76e02b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb712016e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 2 (Thread 0xb2c33b40 (LWP 7698)):
#0  0xb7786c31 in __kernel_vsyscall ()
#1  0xb76e5e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x8116a3fe in safe_cond_timedwait (cond=0x824ce040 <COND_timer>, mp=0x824cdfc0 <LOCK_timer>, abstime=0xb2c33250, file=0x8160be68 "/home/buildbot/buildbot/build/mariadb-10.4.11/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.11/mysys/thr_mutex.c:546
#3  0x8116b3b0 in inline_mysql_cond_timedwait (that=0x824ce040 <COND_timer>, mutex=0x824cdfc0 <LOCK_timer>, abstime=0xb2c33250, src_file=0x8160beb8 "/home/buildbot/buildbot/build/mariadb-10.4.11/mysys/thr_timer.c", src_line=292) at /home/buildbot/buildbot/build/mariadb-10.4.11/include/mysql/psi/mysql_thread.h:1204
#4  0x8116bf8f in timer_handler (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.11/mysys/thr_timer.c:292
#5  0x80b4f81d in pfs_spawn_thread (arg=0x83d6e358) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/perfschema/pfs.cc:1862
#6  0xb76e02b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb712016e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 1 (Thread 0xaedfcb40 (LWP 7708)):
#0  0x81160e2e in my_read (Filedes=7, Buffer=0xaedfa90c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.11/mysys/my_read.c:63
#1  0x808ef979 in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/signal_handler.cc:66
#2  0x808f0161 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb7786c31 in __kernel_vsyscall ()
#5  0xb7064e89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb70663e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb705dd07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb705dd8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x8100471f in Check::validate (space=0x83f67970) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4569
#10 0x810021de in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4600
#11 0x80ff1f6a in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:216
#12 0x810016fe in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4355
#13 0x80e6e688 in io_handler_thread (arg=0x824b9628 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/srv/srv0start.cc:324
#14 0xb76e02b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb712016e in clone () from /lib/i386-linux-gnu/libc.so.6

Comment by Elena Stepanova [ 2019-12-10 ]

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/20411

10.4 aab6cefe8dc132ec6a16e6af5c616765

rpl.rpl_set_null_myisam 'mix'            w3 [ fail ]
        Test ended at 2019-12-04 05:53:48
 
CURRENT_TEST: rpl.rpl_set_null_myisam
 
 
Failed to start mysqld.2
mysqltest failed but provided no output
 
 
 - found 'core' (0/0)
 
Trying 'dbx' to get a backtrace
 
Trying 'gdb' to get a backtrace from coredump /mnt/buildbot/build/mariadb-10.4.11/mysql-test/var/3/log/rpl.rpl_set_null_myisam-mix/mysqld.2/data/core
Core generated by '/mnt/buildbot/build/mariadb-10.4.11/sql/mysqld'
Output from gdb follows. The first stack trace is from the failing thread.
The following stack traces are from all threads (so the failing one is
duplicated).
--------------------------
[New LWP 14033]
[New LWP 14013]
[New LWP 14028]
[New LWP 14029]
[New LWP 14030]
[New LWP 14031]
[New LWP 14032]
[New LWP 14034]
[New LWP 14026]
[New LWP 14024]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `/mnt/buildbot/build/mariadb-10.4.11/sql/mysqld --defaults-group-suffix=.2 --def'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x8113da62 in my_read (Filedes=7, Buffer=0xad6ae90c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.11/mysys/my_read.c:63
63	      int got_errno= my_errno= errno;
[Current thread is 1 (Thread 0xad6b0b40 (LWP 14033))]
#0  0x8113da62 in my_read (Filedes=7, Buffer=0xad6ae90c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.11/mysys/my_read.c:63
#1  0x808cb8b7 in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/signal_handler.cc:66
#2  0x808cc09f in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb77a3c31 in __kernel_vsyscall ()
#5  0xb7081e89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb70833e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb707ad07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb707ad8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x80fe0f03 in Check::validate (space=0x836b3150) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4569
#10 0x80fde9c2 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4600
#11 0x80fce74e in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:216
#12 0x80fddee2 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4355
#13 0x80e4a60a in io_handler_thread (arg=0x82496634 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/srv/srv0start.cc:324
#14 0xb76fd2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb713d16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 10 (Thread 0xb2c50b40 (LWP 14024)):
#0  0xb77a3c31 in __kernel_vsyscall ()
#1  0xb7702e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x81147032 in safe_cond_timedwait (cond=0x824ab040 <COND_timer>, mp=0x824aafc0 <LOCK_timer>, abstime=0xb2c50250, file=0x815e8f08 "/home/buildbot/buildbot/build/mariadb-10.4.11/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.11/mysys/thr_mutex.c:546
#3  0x81147fe4 in inline_mysql_cond_timedwait (that=0x824ab040 <COND_timer>, mutex=0x824aafc0 <LOCK_timer>, abstime=0xb2c50250, src_file=0x815e8f58 "/home/buildbot/buildbot/build/mariadb-10.4.11/mysys/thr_timer.c", src_line=292) at /home/buildbot/buildbot/build/mariadb-10.4.11/include/mysql/psi/mysql_thread.h:1204
#4  0x81148bc3 in timer_handler (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.11/mysys/thr_timer.c:292
#5  0x80b2b637 in pfs_spawn_thread (arg=0x834b9b68) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/perfschema/pfs.cc:1862
#6  0xb76fd2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb713d16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 9 (Thread 0xb1820b40 (LWP 14026)):
#0  0xb77a3c31 in __kernel_vsyscall ()
#1  0xb7702e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x81147032 in safe_cond_timedwait (cond=0x82462620 <COND_checkpoint>, mp=0x824625a0 <LOCK_checkpoint>, abstime=0xb182019c, file=0x81457538 "/home/buildbot/buildbot/build/mariadb-10.4.11/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.11/mysys/thr_mutex.c:546
#3  0x80b94295 in inline_mysql_cond_timedwait (that=0x82462620 <COND_checkpoint>, mutex=0x824625a0 <LOCK_checkpoint>, abstime=0xb182019c, src_file=0x81457588 "/home/buildbot/buildbot/build/mariadb-10.4.11/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/buildbot/build/mariadb-10.4.11/include/mysql/psi/mysql_thread.h:1204
#4  0x80b947fa in my_service_thread_sleep (control=0x81a823b8 <checkpoint_control>, sleep_time=29000000000) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/maria/ma_servicethread.c:115
#5  0x80b88038 in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/maria/ma_checkpoint.c:707
#6  0x80b2b637 in pfs_spawn_thread (arg=0x83536ac8) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/perfschema/pfs.cc:1862
#7  0xb76fd2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#8  0xb713d16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 8 (Thread 0xaceafb40 (LWP 14034)):
#0  0xb77a3c31 in __kernel_vsyscall ()
#1  0xb7702a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80d30376 in os_event::wait (this=0x83679760) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0event.cc:158
#3  0x80d2fdd7 in os_event::wait_low (this=0x83679760, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0event.cc:325
#4  0x80d3011d in os_event_wait_low (event=0x83679760, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0event.cc:502
#5  0x80f59ce2 in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/buf/buf0flu.cc:3047
#6  0xb76fd2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb713d16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 7 (Thread 0xadeb1b40 (LWP 14032)):
#0  0xb776f570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb776f5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d21759 in LinuxAIOHandler::collect (this=0xadeb10e0) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1809
#3  0x80d21dbf in LinuxAIOHandler::poll (this=0xadeb10e0, m1=0xadeb1150, m2=0xadeb1154, request=0xadeb1178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1968
#4  0x80d21f23 in os_aio_linux_handler (global_segment=4, m1=0xadeb1150, m2=0xadeb1154, request=0xadeb1178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:2022
#5  0x80d270b1 in os_aio_handler (segment=4, m1=0xadeb1150, m2=0xadeb1154, request=0xadeb1178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:5565
#6  0x80fddf63 in fil_aio_wait (segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4357
#7  0x80e4a60a in io_handler_thread (arg=0x82496630 <n+16>) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/srv/srv0start.cc:324
#8  0xb76fd2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb713d16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 6 (Thread 0xae6b2b40 (LWP 14031)):
#0  0xb776f570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb776f5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d21759 in LinuxAIOHandler::collect (this=0xae6b20e0) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1809
#3  0x80d21dbf in LinuxAIOHandler::poll (this=0xae6b20e0, m1=0xae6b2150, m2=0xae6b2154, request=0xae6b2178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1968
#4  0x80d21f23 in os_aio_linux_handler (global_segment=3, m1=0xae6b2150, m2=0xae6b2154, request=0xae6b2178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:2022
#5  0x80d270b1 in os_aio_handler (segment=3, m1=0xae6b2150, m2=0xae6b2154, request=0xae6b2178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:5565
#6  0x80fddf63 in fil_aio_wait (segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4357
#7  0x80e4a60a in io_handler_thread (arg=0x8249662c <n+12>) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/srv/srv0start.cc:324
#8  0xb76fd2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb713d16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 5 (Thread 0xaeeb3b40 (LWP 14030)):
#0  0xb776f570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb776f5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d21759 in LinuxAIOHandler::collect (this=0xaeeb30e0) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1809
#3  0x80d21dbf in LinuxAIOHandler::poll (this=0xaeeb30e0, m1=0xaeeb3150, m2=0xaeeb3154, request=0xaeeb3178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1968
#4  0x80d21f23 in os_aio_linux_handler (global_segment=2, m1=0xaeeb3150, m2=0xaeeb3154, request=0xaeeb3178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:2022
#5  0x80d270b1 in os_aio_handler (segment=2, m1=0xaeeb3150, m2=0xaeeb3154, request=0xaeeb3178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:5565
#6  0x80fddf63 in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4357
#7  0x80e4a60a in io_handler_thread (arg=0x82496628 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/srv/srv0start.cc:324
#8  0xb76fd2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb713d16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 4 (Thread 0xaf6b4b40 (LWP 14029)):
#0  0xb776f570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb776f5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d21759 in LinuxAIOHandler::collect (this=0xaf6b40e0) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1809
#3  0x80d21dbf in LinuxAIOHandler::poll (this=0xaf6b40e0, m1=0xaf6b4150, m2=0xaf6b4154, request=0xaf6b4178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1968
#4  0x80d21f23 in os_aio_linux_handler (global_segment=1, m1=0xaf6b4150, m2=0xaf6b4154, request=0xaf6b4178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:2022
#5  0x80d270b1 in os_aio_handler (segment=1, m1=0xaf6b4150, m2=0xaf6b4154, request=0xaf6b4178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:5565
#6  0x80fddf63 in fil_aio_wait (segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4357
#7  0x80e4a60a in io_handler_thread (arg=0x82496624 <n+4>) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/srv/srv0start.cc:324
#8  0xb76fd2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb713d16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 3 (Thread 0xafeb5b40 (LWP 14028)):
#0  0xb776f570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb776f5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d21759 in LinuxAIOHandler::collect (this=0xafeb50e0) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1809
#3  0x80d21dbf in LinuxAIOHandler::poll (this=0xafeb50e0, m1=0xafeb5150, m2=0xafeb5154, request=0xafeb5178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:1968
#4  0x80d21f23 in os_aio_linux_handler (global_segment=0, m1=0xafeb5150, m2=0xafeb5154, request=0xafeb5178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:2022
#5  0x80d270b1 in os_aio_handler (segment=0, m1=0xafeb5150, m2=0xafeb5154, request=0xafeb5178) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0file.cc:5565
#6  0x80fddf63 in fil_aio_wait (segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4357
#7  0x80e4a60a in io_handler_thread (arg=0x82496620 <n>) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/srv/srv0start.cc:324
#8  0xb76fd2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb713d16e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 2 (Thread 0xb6ec37c0 (LWP 14013)):
#0  0xb77a3c31 in __kernel_vsyscall ()
#1  0xb7702a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80d30376 in os_event::wait (this=0x835bd480) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0event.cc:158
#3  0x80d2fdd7 in os_event::wait_low (this=0x835bd480, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0event.cc:325
#4  0x80d3011d in os_event_wait_low (event=0x835bd480, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/os/os0event.cc:502
#5  0x80e53882 in sync_array_wait_event (arr=0x8354d608, cell=@0xbfc2e4c0: 0x8354d698) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/sync/sync0arr.cc:471
#6  0x80c59845 in TTASEventMutex<GenericPolicy>::enter (this=0x81a8d6f0 <fil_system+16>, max_spins=60, max_delay=4, filename=0x81594044 "/home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc", line=4588) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/include/ib0mutex.h:478
#7  0x80c578b2 in PolicyMutex<TTASEventMutex<GenericPolicy> >::enter (this=0x81a8d6f0 <fil_system+16>, n_spins=30, n_delay=4, name=0x81594044 "/home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc", line=4588) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/include/ib0mutex.h:595
#8  0x80fde971 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4588
#9  0x810048d8 in SysTablespace::open_or_create (this=0x824a38c0 <srv_sys_space>, is_temp=false, create_new_db=false, sum_new_sizes=0xbfc2e650, flush_lsn=0xbfc2e6a0) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fsp/fsp0sysspace.cc:924
#10 0x80e4eb06 in srv_start (create_new_db=false) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/srv/srv0start.cc:1583
#11 0x80c2ddd0 in innodb_init (p=0x83536ac8) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/handler/ha_innodb.cc:4258
#12 0x808cfe76 in ha_initialize_handlerton (plugin=0x834dc220) at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/handler.cc:557
#13 0x80598ae9 in plugin_initialize (tmp_root=0xbfc33868, plugin=0x834dc220, argc=0x81c34fb4 <remaining_argc>, argv=0x83377128, options_only=false) at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/sql_plugin.cc:1438
#14 0x8059977b in plugin_init (argc=0x81c34fb4 <remaining_argc>, argv=0x83377128, flags=0) at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/sql_plugin.cc:1720
#15 0x80446661 in init_server_components () at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/mysqld.cc:5199
#16 0x80447ba3 in mysqld_main (argc=142, argv=0x83377128) at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/mysqld.cc:5722
#17 0x8043b9db in main (argc=9, argv=0xbfc34554) at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/main.cc:25
 
Thread 1 (Thread 0xad6b0b40 (LWP 14033)):
#0  0x8113da62 in my_read (Filedes=7, Buffer=0xad6ae90c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.11/mysys/my_read.c:63
#1  0x808cb8b7 in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/signal_handler.cc:66
#2  0x808cc09f in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.11/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb77a3c31 in __kernel_vsyscall ()
#5  0xb7081e89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb70833e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb707ad07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb707ad8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x80fe0f03 in Check::validate (space=0x836b3150) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4569
#10 0x80fde9c2 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4600
#11 0x80fce74e in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:216
#12 0x80fddee2 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/fil/fil0fil.cc:4355
#13 0x80e4a60a in io_handler_thread (arg=0x82496634 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.11/storage/innobase/srv/srv0start.cc:324
#14 0xb76fd2b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb713d16e in clone () from /lib/i386-linux-gnu/libc.so.6

Comment by Elena Stepanova [ 2020-01-26 ]

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/20792

10.4 2d4b6571ecf15a4382702d26df9439bc

rpl.rpl_heartbeat                        w1 [ fail ]
        Test ended at 2020-01-14 02:25:32
 
CURRENT_TEST: rpl.rpl_heartbeat
 
 
Failed to start mysqld.2
mysqltest failed but provided no output
 
 
 - found 'core' (0/0)
 
Trying 'dbx' to get a backtrace
 
Trying 'gdb' to get a backtrace from coredump /mnt/buildbot/build/mariadb-10.4.12/mysql-test/var/1/log/rpl.rpl_heartbeat/mysqld.2/data/core
Core generated by '/mnt/buildbot/build/mariadb-10.4.12/sql/mysqld'
Output from gdb follows. The first stack trace is from the failing thread.
The following stack traces are from all threads (so the failing one is
duplicated).
--------------------------
[New LWP 18441]
[New LWP 18434]
[New LWP 18411]
[New LWP 18436]
[New LWP 18437]
[New LWP 18438]
[New LWP 18439]
[New LWP 18440]
[New LWP 18442]
[New LWP 18433]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `/mnt/buildbot/build/mariadb-10.4.12/sql/mysqld --defaults-group-suffix=.2 --def'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x811d0d5a in my_read (Filedes=7, Buffer=0xad5ae90c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.12/mysys/my_read.c:63
63	      int got_errno= my_errno= errno;
[Current thread is 1 (Thread 0xad5b0b40 (LWP 18441))]
#0  0x811d0d5a in my_read (Filedes=7, Buffer=0xad5ae90c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.12/mysys/my_read.c:63
#1  0x8095ace7 in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/signal_handler.cc:66
#2  0x8095b4cf in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb7707c31 in __kernel_vsyscall ()
#5  0xb6fe5e89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb6fe73e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb6fded07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb6fded8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x810744c9 in Check::validate (space=0x841bd178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4559
#10 0x81071ee3 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4590
#11 0x81061bcb in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:216
#12 0x8107138e in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4351
#13 0x80eddc46 in io_handler_thread (arg=0x825475b4 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0start.cc:324
#14 0xb76612b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb70a116e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 10 (Thread 0xb2bb4b40 (LWP 18433)):
#0  0xb7707c31 in __kernel_vsyscall ()
#1  0xb7666e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x811da32a in safe_cond_timedwait (cond=0x8255bfc0 <COND_timer>, mp=0x8255bf40 <LOCK_timer>, abstime=0xb2bb4250, file=0x81693dc8 "/home/buildbot/buildbot/build/mariadb-10.4.12/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.12/mysys/thr_mutex.c:546
#3  0x811db2dc in inline_mysql_cond_timedwait (that=0x8255bfc0 <COND_timer>, mutex=0x8255bf40 <LOCK_timer>, abstime=0xb2bb4250, src_file=0x81693e18 "/home/buildbot/buildbot/build/mariadb-10.4.12/mysys/thr_timer.c", src_line=292) at /home/buildbot/buildbot/build/mariadb-10.4.12/include/mysql/psi/mysql_thread.h:1204
#4  0x811dbebb in timer_handler (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.12/mysys/thr_timer.c:292
#5  0x80bba865 in pfs_spawn_thread (arg=0x83fb7610) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/perfschema/pfs.cc:1862
#6  0xb76612b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb70a116e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 9 (Thread 0xacdafb40 (LWP 18442)):
#0  0xb7707c31 in __kernel_vsyscall ()
#1  0xb7666a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80dc245e in os_event::wait (this=0x84183788) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:158
#3  0x80dc1ebf in os_event::wait_low (this=0x84183788, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:325
#4  0x80dc2205 in os_event_wait_low (event=0x84183788, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:502
#5  0x80fedbdd in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/buf/buf0flu.cc:3044
#6  0xb76612b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb70a116e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 8 (Thread 0xaddb1b40 (LWP 18440)):
#0  0xb76d3570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76d35e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80db25bd in LinuxAIOHandler::collect (this=0xaddb10e0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1813
#3  0x80db2cc9 in LinuxAIOHandler::poll (this=0xaddb10e0, m1=0xaddb1150, m2=0xaddb1154, request=0xaddb1178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1974
#4  0x80db2e2d in os_aio_linux_handler (global_segment=4, m1=0xaddb1150, m2=0xaddb1154, request=0xaddb1178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:2028
#5  0x80db83b9 in os_aio_handler (segment=4, m1=0xaddb1150, m2=0xaddb1154, request=0xaddb1178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:5598
#6  0x8107140f in fil_aio_wait (segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4353
#7  0x80eddc46 in io_handler_thread (arg=0x825475b0 <n+16>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0start.cc:324
#8  0xb76612b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb70a116e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 7 (Thread 0xae5b2b40 (LWP 18439)):
#0  0xb76d3570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76d35e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80db25bd in LinuxAIOHandler::collect (this=0xae5b20e0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1813
#3  0x80db2cc9 in LinuxAIOHandler::poll (this=0xae5b20e0, m1=0xae5b2150, m2=0xae5b2154, request=0xae5b2178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1974
#4  0x80db2e2d in os_aio_linux_handler (global_segment=3, m1=0xae5b2150, m2=0xae5b2154, request=0xae5b2178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:2028
#5  0x80db83b9 in os_aio_handler (segment=3, m1=0xae5b2150, m2=0xae5b2154, request=0xae5b2178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:5598
#6  0x8107140f in fil_aio_wait (segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4353
#7  0x80eddc46 in io_handler_thread (arg=0x825475ac <n+12>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0start.cc:324
#8  0xb76612b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb70a116e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 6 (Thread 0xaedb3b40 (LWP 18438)):
#0  0xb76d3570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76d35e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80db25bd in LinuxAIOHandler::collect (this=0xaedb30e0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1813
#3  0x80db2cc9 in LinuxAIOHandler::poll (this=0xaedb30e0, m1=0xaedb3150, m2=0xaedb3154, request=0xaedb3178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1974
#4  0x80db2e2d in os_aio_linux_handler (global_segment=2, m1=0xaedb3150, m2=0xaedb3154, request=0xaedb3178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:2028
#5  0x80db83b9 in os_aio_handler (segment=2, m1=0xaedb3150, m2=0xaedb3154, request=0xaedb3178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:5598
#6  0x8107140f in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4353
#7  0x80eddc46 in io_handler_thread (arg=0x825475a8 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0start.cc:324
#8  0xb76612b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb70a116e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 5 (Thread 0xaf5b4b40 (LWP 18437)):
#0  0xb76d3570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76d35e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80db25bd in LinuxAIOHandler::collect (this=0xaf5b40e0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1813
#3  0x80db2cc9 in LinuxAIOHandler::poll (this=0xaf5b40e0, m1=0xaf5b4150, m2=0xaf5b4154, request=0xaf5b4178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1974
#4  0x80db2e2d in os_aio_linux_handler (global_segment=1, m1=0xaf5b4150, m2=0xaf5b4154, request=0xaf5b4178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:2028
#5  0x80db83b9 in os_aio_handler (segment=1, m1=0xaf5b4150, m2=0xaf5b4154, request=0xaf5b4178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:5598
#6  0x8107140f in fil_aio_wait (segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4353
#7  0x80eddc46 in io_handler_thread (arg=0x825475a4 <n+4>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0start.cc:324
#8  0xb76612b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb70a116e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 4 (Thread 0xafdb5b40 (LWP 18436)):
#0  0xb76d3570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76d35e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80db25bd in LinuxAIOHandler::collect (this=0xafdb50e0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1813
#3  0x80db2cc9 in LinuxAIOHandler::poll (this=0xafdb50e0, m1=0xafdb5150, m2=0xafdb5154, request=0xafdb5178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1974
#4  0x80db2e2d in os_aio_linux_handler (global_segment=0, m1=0xafdb5150, m2=0xafdb5154, request=0xafdb5178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:2028
#5  0x80db83b9 in os_aio_handler (segment=0, m1=0xafdb5150, m2=0xafdb5154, request=0xafdb5178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:5598
#6  0x8107140f in fil_aio_wait (segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4353
#7  0x80eddc46 in io_handler_thread (arg=0x825475a0 <n>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0start.cc:324
#8  0xb76612b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb70a116e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 3 (Thread 0xb6e277c0 (LWP 18411)):
#0  0xb7707c31 in __kernel_vsyscall ()
#1  0xb7666a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80dc245e in os_event::wait (this=0x840c74a8) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:158
#3  0x80dc1ebf in os_event::wait_low (this=0x840c74a8, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:325
#4  0x80dc2205 in os_event_wait_low (event=0x840c74a8, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:502
#5  0x80ee6ec0 in sync_array_wait_event (arr=0x84057668, cell=@0xbfa6f830: 0x840576f8) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/sync/sync0arr.cc:471
#6  0x80ce8d75 in TTASEventMutex<GenericPolicy>::enter (this=0x81b3e690 <fil_system+16>, max_spins=60, max_delay=4, filename=0x816407c0 "/home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc", line=4578) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/include/ib0mutex.h:478
#7  0x80ce6e9c in PolicyMutex<TTASEventMutex<GenericPolicy> >::enter (this=0x81b3e690 <fil_system+16>, n_spins=30, n_delay=4, name=0x816407c0 "/home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc", line=4578) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/include/ib0mutex.h:595
#8  0x81071e92 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4578
#9  0x81098318 in SysTablespace::open_or_create (this=0x82554840 <srv_sys_space>, is_temp=false, create_new_db=false, sum_new_sizes=0xbfa6f9c0, flush_lsn=0xbfa6fa10) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fsp/fsp0sysspace.cc:924
#10 0x80ee2142 in srv_start (create_new_db=false) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0start.cc:1583
#11 0x80cbcf9e in innodb_init (p=0x84055bb8) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/handler/ha_innodb.cc:4247
#12 0x8095f31e in ha_initialize_handlerton (plugin=0x83fe6688) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/handler.cc:557
#13 0x80627461 in plugin_initialize (tmp_root=0xbfa74bc8, plugin=0x83fe6688, argc=0x81ce5f34 <remaining_argc>, argv=0x83e93330, options_only=false) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/sql_plugin.cc:1438
#14 0x806280f3 in plugin_init (argc=0x81ce5f34 <remaining_argc>, argv=0x83e93330, flags=0) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/sql_plugin.cc:1720
#15 0x804d4861 in init_server_components () at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/mysqld.cc:5199
#16 0x804d5da3 in mysqld_main (argc=140, argv=0x83e93330) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/mysqld.cc:5722
#17 0x804c9bdb in main (argc=7, argv=0xbfa758b4) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/main.cc:25
 
Thread 2 (Thread 0xb1743b40 (LWP 18434)):
#0  0xb7707c31 in __kernel_vsyscall ()
#1  0xb7666e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x811da32a in safe_cond_timedwait (cond=0x825135a0 <COND_checkpoint>, mp=0x82513520 <LOCK_checkpoint>, abstime=0xb174319c, file=0x81506138 "/home/buildbot/buildbot/build/mariadb-10.4.12/include/mysql/psi/mysql_thread.h", line=1204) at /home/buildbot/buildbot/build/mariadb-10.4.12/mysys/thr_mutex.c:546
#3  0x80c234c3 in inline_mysql_cond_timedwait (that=0x825135a0 <COND_checkpoint>, mutex=0x82513520 <LOCK_checkpoint>, abstime=0xb174319c, src_file=0x81506188 "/home/buildbot/buildbot/build/mariadb-10.4.12/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/buildbot/build/mariadb-10.4.12/include/mysql/psi/mysql_thread.h:1204
#4  0x80c23a28 in my_service_thread_sleep (control=0x81b333b8 <checkpoint_control>, sleep_time=29000000000) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/maria/ma_servicethread.c:115
#5  0x80c17266 in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/maria/ma_checkpoint.c:707
#6  0x80bba865 in pfs_spawn_thread (arg=0x84040b00) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/perfschema/pfs.cc:1862
#7  0xb76612b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#8  0xb70a116e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 1 (Thread 0xad5b0b40 (LWP 18441)):
#0  0x811d0d5a in my_read (Filedes=7, Buffer=0xad5ae90c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.12/mysys/my_read.c:63
#1  0x8095ace7 in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/signal_handler.cc:66
#2  0x8095b4cf in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb7707c31 in __kernel_vsyscall ()
#5  0xb6fe5e89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb6fe73e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb6fded07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb6fded8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x810744c9 in Check::validate (space=0x841bd178) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4559
#10 0x81071ee3 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4590
#11 0x81061bcb in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:216
#12 0x8107138e in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4351
#13 0x80eddc46 in io_handler_thread (arg=0x825475b4 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0start.cc:324
#14 0xb76612b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb70a116e in clone () from /lib/i386-linux-gnu/libc.so.6

Comment by Elena Stepanova [ 2020-05-09 ]

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/21684

10.4 517f659e6d5eeb7e01bf19ef8b782bc2

binlog_encryption.mysqlbinlog 'ncmpr'    w3 [ fail ]
        Test ended at 2020-03-17 11:55:01
 
CURRENT_TEST: binlog_encryption.mysqlbinlog
 
 
Failed to start mysqld.1
mysqltest failed but provided no output
The result from queries just before the failure was:
< snip >
RESET MASTER;
CREATE TABLE t1 (a INT);
INSERT INTO t1 VALUES (1),(2),(3);
REPLACE INTO t1 VALUES (4);
DROP TABLE t1;
FLUSH LOGS;
FOUND 1 /Ignorable event type 164.*/ in binlog_enc.sql
 
 
 
 - found 'core' (0/0)
 
Trying 'dbx' to get a backtrace
 
Trying 'gdb' to get a backtrace from coredump /mnt/buildbot/build/mariadb-10.4.13/mysql-test/var/3/log/binlog_encryption.mysqlbinlog-ncmpr/mysqld.1/data/core
Core generated by '/mnt/buildbot/build/mariadb-10.4.13/sql/mysqld'
Output from gdb follows. The first stack trace is from the failing thread.
The following stack traces are from all threads (so the failing one is
duplicated).
--------------------------
[New LWP 8905]
[New LWP 8886]
[New LWP 8892]
[New LWP 8900]
[New LWP 8901]
[New LWP 8902]
[New LWP 8903]
[New LWP 8904]
[New LWP 8906]
[New LWP 8891]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `/mnt/buildbot/build/mariadb-10.4.13/sql/mysqld --defaults-group-suffix=.1 --def'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x81175ea0 in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/my_read.c:63
63	      int got_errno= my_errno= errno;
[Current thread is 1 (Thread 0xad5f9b40 (LWP 8905))]
#0  0x81175ea0 in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/my_read.c:63
#1  0x808ffe01 in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/signal_handler.cc:66
#2  0x809005e9 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb777fc31 in __kernel_vsyscall ()
#5  0xb705de89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb705f3e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb7056d07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb7056d8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x81019e13 in Check::validate (space=0x82ee5ea8) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4559
#10 0x8101782d in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4590
#11 0x81007516 in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:216
#12 0x81016cd8 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4351
#13 0x80e83fd9 in io_handler_thread (arg=0x824ee774 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#14 0xb76d92b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb711916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 10 (Thread 0xb2c04b40 (LWP 8891)):
#0  0xb777fc31 in __kernel_vsyscall ()
#1  0xb76dee56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x8117f470 in safe_cond_timedwait (cond=0x82503180 <COND_timer>, mp=0x82503100 <LOCK_timer>, abstime=0xb2c04250, file=0x8163b968 "/home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/thr_mutex.c:546
#3  0x81180422 in inline_mysql_cond_timedwait (that=0x82503180 <COND_timer>, mutex=0x82503100 <LOCK_timer>, abstime=0xb2c04250, src_file=0x8163b9b8 "/home/buildbot/buildbot/build/mariadb-10.4.13/mysys/thr_timer.c", src_line=292) at /home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h:1211
#4  0x81181001 in timer_handler (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/thr_timer.c:292
#5  0x80b605d1 in pfs_spawn_thread (arg=0x82cdeb10) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/perfschema/pfs.cc:1869
#6  0xb76d92b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb711916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 9 (Thread 0xacdf8b40 (LWP 8906)):
#0  0xb777fc31 in __kernel_vsyscall ()
#1  0xb76dea8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80d6831a in os_event::wait (this=0x82eac4b8) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:158
#3  0x80d67d7b in os_event::wait_low (this=0x82eac4b8, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:325
#4  0x80d680c1 in os_event_wait_low (event=0x82eac4b8, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:502
#5  0x80f935eb in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/buf/buf0flu.cc:3044
#6  0xb76d92b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb711916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 8 (Thread 0xaddfab40 (LWP 8904)):
#0  0xb774b570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb774b5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d58479 in LinuxAIOHandler::collect (this=0xaddfa0e0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
#3  0x80d58b85 in LinuxAIOHandler::poll (this=0xaddfa0e0, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
#4  0x80d58ce9 in os_aio_linux_handler (global_segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
#5  0x80d5e275 in os_aio_handler (segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
#6  0x81016d59 in fil_aio_wait (segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4353
#7  0x80e83fd9 in io_handler_thread (arg=0x824ee770 <n+16>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#8  0xb76d92b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb711916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 7 (Thread 0xae5fbb40 (LWP 8903)):
#0  0xb774b570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb774b5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d58479 in LinuxAIOHandler::collect (this=0xae5fb0e0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
#3  0x80d58b85 in LinuxAIOHandler::poll (this=0xae5fb0e0, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
#4  0x80d58ce9 in os_aio_linux_handler (global_segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
#5  0x80d5e275 in os_aio_handler (segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
#6  0x81016d59 in fil_aio_wait (segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4353
#7  0x80e83fd9 in io_handler_thread (arg=0x824ee76c <n+12>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#8  0xb76d92b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb711916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 6 (Thread 0xaedfcb40 (LWP 8902)):
#0  0xb774b570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb774b5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d58479 in LinuxAIOHandler::collect (this=0xaedfc0e0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
#3  0x80d58b85 in LinuxAIOHandler::poll (this=0xaedfc0e0, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
#4  0x80d58ce9 in os_aio_linux_handler (global_segment=2, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
#5  0x80d5e275 in os_aio_handler (segment=2, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
#6  0x81016d59 in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4353
#7  0x80e83fd9 in io_handler_thread (arg=0x824ee768 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#8  0xb76d92b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb711916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 5 (Thread 0xaf5fdb40 (LWP 8901)):
#0  0xb774b570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb774b5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d58479 in LinuxAIOHandler::collect (this=0xaf5fd0e0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
#3  0x80d58b85 in LinuxAIOHandler::poll (this=0xaf5fd0e0, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
#4  0x80d58ce9 in os_aio_linux_handler (global_segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
#5  0x80d5e275 in os_aio_handler (segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
#6  0x81016d59 in fil_aio_wait (segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4353
#7  0x80e83fd9 in io_handler_thread (arg=0x824ee764 <n+4>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#8  0xb76d92b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb711916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 4 (Thread 0xafdfeb40 (LWP 8900)):
#0  0xb774b570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb774b5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80d58479 in LinuxAIOHandler::collect (this=0xafdfe0e0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
#3  0x80d58b85 in LinuxAIOHandler::poll (this=0xafdfe0e0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
#4  0x80d58ce9 in os_aio_linux_handler (global_segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
#5  0x80d5e275 in os_aio_handler (segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
#6  0x81016d59 in fil_aio_wait (segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4353
#7  0x80e83fd9 in io_handler_thread (arg=0x824ee760 <n>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#8  0xb76d92b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb711916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 3 (Thread 0xb17eab40 (LWP 8892)):
#0  0xb777fc31 in __kernel_vsyscall ()
#1  0xb76dee56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x8117f470 in safe_cond_timedwait (cond=0x824ba760 <COND_checkpoint>, mp=0x824ba6e0 <LOCK_checkpoint>, abstime=0xb17ea19c, file=0x814ae1d8 "/home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/thr_mutex.c:546
#3  0x80bc9515 in inline_mysql_cond_timedwait (that=0x824ba760 <COND_checkpoint>, mutex=0x824ba6e0 <LOCK_checkpoint>, abstime=0xb17ea19c, src_file=0x814ae228 "/home/buildbot/buildbot/build/mariadb-10.4.13/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h:1211
#4  0x80bc9a7a in my_service_thread_sleep (control=0x81ada418 <checkpoint_control>, sleep_time=29000000000) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/maria/ma_servicethread.c:115
#5  0x80bbd2b8 in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/maria/ma_checkpoint.c:707
#6  0x80b605d1 in pfs_spawn_thread (arg=0x82d69848) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/perfschema/pfs.cc:1869
#7  0xb76d92b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#8  0xb711916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 2 (Thread 0xb6e9f7c0 (LWP 8886)):
#0  0xb777fc31 in __kernel_vsyscall ()
#1  0xb76dea8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80d6831a in os_event::wait (this=0x82df01d8) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:158
#3  0x80d67d7b in os_event::wait_low (this=0x82df01d8, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:325
#4  0x80d680c1 in os_event_wait_low (event=0x82df01d8, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:502
#5  0x80e8d254 in sync_array_wait_event (arr=0x82d803f8, cell=@0xbf869a50: 0x82d80488) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/sync/sync0arr.cc:471
#6  0x80c8ecf9 in TTASEventMutex<GenericPolicy>::enter (this=0x81ae5690 <fil_system+16>, max_spins=60, max_delay=4, filename=0x815e8658 "/home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc", line=4578) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/include/ib0mutex.h:477
#7  0x80c8ce20 in PolicyMutex<TTASEventMutex<GenericPolicy> >::enter (this=0x81ae5690 <fil_system+16>, n_spins=30, n_delay=4, name=0x815e8658 "/home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc", line=4578) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/include/ib0mutex.h:594
#8  0x810177dc in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4578
#9  0x8103d2a4 in SysTablespace::open_or_create (this=0x824fba00 <srv_sys_space>, is_temp=false, create_new_db=false, sum_new_sizes=0xbf869be0, flush_lsn=0xbf869c30) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fsp/fsp0sysspace.cc:924
#10 0x80e884d5 in srv_start (create_new_db=false) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:1583
#11 0x80c62dee in innodb_init (p=0x82d7e900) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/handler/ha_innodb.cc:4247
#12 0x80904438 in ha_initialize_handlerton (plugin=0x82d0d980) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/handler.cc:557
#13 0x805caf1f in plugin_initialize (tmp_root=0xbf86ede8, plugin=0x82d0d980, argc=0x81c8cf34 <remaining_argc>, argv=0x82bba330, options_only=false) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/sql_plugin.cc:1438
#14 0x805cbbb1 in plugin_init (argc=0x81c8cf34 <remaining_argc>, argv=0x82bba330, flags=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/sql_plugin.cc:1720
#15 0x804781c3 in init_server_components () at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:5207
#16 0x80479705 in mysqld_main (argc=136, argv=0x82bba330) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:5730
#17 0x8046d43b in main (argc=8, argv=0xbf86fad4) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/main.cc:25
 
Thread 1 (Thread 0xad5f9b40 (LWP 8905)):
#0  0x81175ea0 in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/my_read.c:63
#1  0x808ffe01 in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/signal_handler.cc:66
#2  0x809005e9 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb777fc31 in __kernel_vsyscall ()
#5  0xb705de89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb705f3e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb7056d07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb7056d8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x81019e13 in Check::validate (space=0x82ee5ea8) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4559
#10 0x8101782d in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4590
#11 0x81007516 in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:216
#12 0x81016cd8 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4351
#13 0x80e83fd9 in io_handler_thread (arg=0x824ee774 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#14 0xb76d92b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb711916e in clone () from /lib/i386-linux-gnu/libc.so.6

Comment by Elena Stepanova [ 2020-05-09 ]

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/21985

10.4 476966b3fb05ce8c061a4b77a0ab0b72

main.partition_innodb_stmt 'innodb'      w2 [ fail ]
        Test ended at 2020-04-08 07:06:28
 
CURRENT_TEST: main.partition_innodb_stmt
 
 
Failed to start mysqld.1
mysqltest failed but provided no output
 
 
 - found 'core' (0/0)
 
Trying 'dbx' to get a backtrace
 
Trying 'gdb' to get a backtrace from coredump /mnt/buildbot/build/mariadb-10.4.13/mysql-test/var/2/log/main.partition_innodb_stmt-innodb/mysqld.1/data/core
Core generated by '/mnt/buildbot/build/mariadb-10.4.13/sql/mysqld'
Output from gdb follows. The first stack trace is from the failing thread.
The following stack traces are from all threads (so the failing one is
duplicated).
--------------------------
[New LWP 1923]
[New LWP 1903]
[New LWP 1916]
[New LWP 1924]
[New LWP 1922]
[New LWP 1921]
[New LWP 1920]
[New LWP 1919]
[New LWP 1918]
[New LWP 1915]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `/mnt/buildbot/build/mariadb-10.4.13/sql/mysqld --defaults-group-suffix=.1 --def'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x811d6bf3 in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/my_read.c:63
63	      int got_errno= my_errno= errno;
[Current thread is 1 (Thread 0xad5f9b40 (LWP 1923))]
#0  0x811d6bf3 in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/my_read.c:63
#1  0x8095c20b in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/signal_handler.cc:66
#2  0x8095c9f3 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb77cec31 in __kernel_vsyscall ()
#5  0xb70ace89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb70ae3e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb70a5d07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb70a5d8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x81079e1d in Check::validate (space=0x83d64828) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4555
#10 0x81077837 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4586
#11 0x81067512 in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:216
#12 0x81076ce2 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4347
#13 0x80ee3b63 in io_handler_thread (arg=0x82553774 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#14 0xb77282b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb716816e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 10 (Thread 0xb2c53b40 (LWP 1915)):
#0  0xb77cec31 in __kernel_vsyscall ()
#1  0xb772de56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x811e01e1 in safe_cond_timedwait (cond=0x82568180 <COND_timer>, mp=0x82568100 <LOCK_timer>, abstime=0xb2c53250, file=0x8169efe8 "/home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/thr_mutex.c:546
#3  0x811e1193 in inline_mysql_cond_timedwait (that=0x82568180 <COND_timer>, mutex=0x82568100 <LOCK_timer>, abstime=0xb2c53250, src_file=0x8169f038 "/home/buildbot/buildbot/build/mariadb-10.4.13/mysys/thr_timer.c", src_line=292) at /home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h:1211
#4  0x811e1d72 in timer_handler (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/thr_timer.c:292
#5  0x80bbda55 in pfs_spawn_thread (arg=0x83b5eb30) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/perfschema/pfs.cc:1869
#6  0xb77282b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb716816e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 9 (Thread 0xafdfeb40 (LWP 1918)):
#0  0xb779a570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb779a5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80db6f1d in LinuxAIOHandler::collect (this=0xafdfe0e0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
#3  0x80db7629 in LinuxAIOHandler::poll (this=0xafdfe0e0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
#4  0x80db778d in os_aio_linux_handler (global_segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
#5  0x80dbcd19 in os_aio_handler (segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
#6  0x81076d63 in fil_aio_wait (segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4349
#7  0x80ee3b63 in io_handler_thread (arg=0x82553760 <n>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#8  0xb77282b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb716816e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 8 (Thread 0xaf5fdb40 (LWP 1919)):
#0  0xb779a570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb779a5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80db6f1d in LinuxAIOHandler::collect (this=0xaf5fd0e0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
#3  0x80db7629 in LinuxAIOHandler::poll (this=0xaf5fd0e0, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
#4  0x80db778d in os_aio_linux_handler (global_segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
#5  0x80dbcd19 in os_aio_handler (segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
#6  0x81076d63 in fil_aio_wait (segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4349
#7  0x80ee3b63 in io_handler_thread (arg=0x82553764 <n+4>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#8  0xb77282b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb716816e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 7 (Thread 0xaedfcb40 (LWP 1920)):
#0  0xb779a570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb779a5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80db6f1d in LinuxAIOHandler::collect (this=0xaedfc0e0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
#3  0x80db7629 in LinuxAIOHandler::poll (this=0xaedfc0e0, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
#4  0x80db778d in os_aio_linux_handler (global_segment=2, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
#5  0x80dbcd19 in os_aio_handler (segment=2, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
#6  0x81076d63 in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4349
#7  0x80ee3b63 in io_handler_thread (arg=0x82553768 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#8  0xb77282b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb716816e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 6 (Thread 0xae5fbb40 (LWP 1921)):
#0  0xb779a570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb779a5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80db6f1d in LinuxAIOHandler::collect (this=0xae5fb0e0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
#3  0x80db7629 in LinuxAIOHandler::poll (this=0xae5fb0e0, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
#4  0x80db778d in os_aio_linux_handler (global_segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
#5  0x80dbcd19 in os_aio_handler (segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
#6  0x81076d63 in fil_aio_wait (segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4349
#7  0x80ee3b63 in io_handler_thread (arg=0x8255376c <n+12>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#8  0xb77282b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb716816e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 5 (Thread 0xaddfab40 (LWP 1922)):
#0  0xb779a570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb779a5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80db6f1d in LinuxAIOHandler::collect (this=0xaddfa0e0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
#3  0x80db7629 in LinuxAIOHandler::poll (this=0xaddfa0e0, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
#4  0x80db778d in os_aio_linux_handler (global_segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
#5  0x80dbcd19 in os_aio_handler (segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
#6  0x81076d63 in fil_aio_wait (segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4349
#7  0x80ee3b63 in io_handler_thread (arg=0x82553770 <n+16>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#8  0xb77282b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb716816e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 4 (Thread 0xacdf8b40 (LWP 1924)):
#0  0xb77cec31 in __kernel_vsyscall ()
#1  0xb772da8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80dc6dbe in os_event::wait (this=0x83d2ae38) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:158
#3  0x80dc681f in os_event::wait_low (this=0x83d2ae38, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:325
#4  0x80dc6b65 in os_event_wait_low (event=0x83d2ae38, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:502
#5  0x80ff3537 in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/buf/buf0flu.cc:3044
#6  0xb77282b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb716816e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 3 (Thread 0xb17ffb40 (LWP 1916)):
#0  0xb77cec31 in __kernel_vsyscall ()
#1  0xb772de56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x811e01e1 in safe_cond_timedwait (cond=0x8251f760 <COND_checkpoint>, mp=0x8251f6e0 <LOCK_checkpoint>, abstime=0xb17ff19c, file=0x815100f8 "/home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/thr_mutex.c:546
#3  0x80c26af6 in inline_mysql_cond_timedwait (that=0x8251f760 <COND_checkpoint>, mutex=0x8251f6e0 <LOCK_checkpoint>, abstime=0xb17ff19c, src_file=0x81510148 "/home/buildbot/buildbot/build/mariadb-10.4.13/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h:1211
#4  0x80c2705b in my_service_thread_sleep (control=0x81b3f418 <checkpoint_control>, sleep_time=29000000000) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/maria/ma_servicethread.c:115
#5  0x80c1a818 in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/maria/ma_checkpoint.c:707
#6  0x80bbda55 in pfs_spawn_thread (arg=0x83be8020) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/perfschema/pfs.cc:1869
#7  0xb77282b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#8  0xb716816e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 2 (Thread 0xb6eee7c0 (LWP 1903)):
#0  0xb77cec31 in __kernel_vsyscall ()
#1  0xb772da8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80dc6dbe in os_event::wait (this=0x83c6eb58) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:158
#3  0x80dc681f in os_event::wait_low (this=0x83c6eb58, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:325
#4  0x80dc6b65 in os_event_wait_low (event=0x83c6eb58, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:502
#5  0x80eecdde in sync_array_wait_event (arr=0x83bfed18, cell=@0xbfcaad20: 0x83bfeda8) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/sync/sync0arr.cc:471
#6  0x80ced3b7 in TTASEventMutex<GenericPolicy>::enter (this=0x81b4a690 <fil_system+16>, max_spins=60, max_delay=4, filename=0x8164bcd8 "/home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc", line=4574) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/include/ib0mutex.h:477
#7  0x80ceb4de in PolicyMutex<TTASEventMutex<GenericPolicy> >::enter (this=0x81b4a690 <fil_system+16>, n_spins=30, n_delay=4, name=0x8164bcd8 "/home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc", line=4574) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/include/ib0mutex.h:594
#8  0x810777e6 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4574
#9  0x8109d2ae in SysTablespace::open_or_create (this=0x82560a00 <srv_sys_space>, is_temp=false, create_new_db=false, sum_new_sizes=0xbfcaaeb0, flush_lsn=0xbfcaaf00) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fsp/fsp0sysspace.cc:924
#10 0x80ee805f in srv_start (create_new_db=false) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:1583
#11 0x80cc147c in innodb_init (p=0x83be8020) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/handler/ha_innodb.cc:4263
#12 0x8096089b in ha_initialize_handlerton (plugin=0x83b8d9f8) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/handler.cc:557
#13 0x80626b11 in plugin_initialize (tmp_root=0xbfcb00b8, plugin=0x83b8d9f8, argc=0x81cf1f34 <remaining_argc>, argv=0x83a3a578, options_only=false) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/sql_plugin.cc:1438
#14 0x806277a3 in plugin_init (argc=0x81cf1f34 <remaining_argc>, argv=0x83a3a578, flags=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/sql_plugin.cc:1720
#15 0x804d3ad3 in init_server_components () at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:5207
#16 0x804d5015 in mysqld_main (argc=150, argv=0x83a3a578) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:5730
#17 0x804c8d4b in main (argc=26, argv=0xbfcb0da4) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/main.cc:25
 
Thread 1 (Thread 0xad5f9b40 (LWP 1923)):
#0  0x811d6bf3 in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/my_read.c:63
#1  0x8095c20b in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/signal_handler.cc:66
#2  0x8095c9f3 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb77cec31 in __kernel_vsyscall ()
#5  0xb70ace89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb70ae3e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb70a5d07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb70a5d8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x81079e1d in Check::validate (space=0x83d64828) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4555
#10 0x81077837 in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4586
#11 0x81067512 in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:216
#12 0x81076ce2 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4347
#13 0x80ee3b63 in io_handler_thread (arg=0x82553774 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#14 0xb77282b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb716816e in clone () from /lib/i386-linux-gnu/libc.so.6

Comment by Elena Stepanova [ 2020-05-09 ]

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/22041

10.4 c1bdf62452139e9310d3234266802748

rpl.rpl_plugin_load 'stmt'               w3 [ fail ]
        Test ended at 2020-04-13 11:27:50
 
CURRENT_TEST: rpl.rpl_plugin_load
 
 
Failed to start mysqld.2
mysqltest failed but provided no output
The result from queries just before the failure was:
< snip >
connection slave;
Verify that example engine is not installed.
connection slave;
SELECT * FROM INFORMATION_SCHEMA.ENGINES WHERE ENGINE='EXAMPLE';
ENGINE	SUPPORT	COMMENT	TRANSACTIONS	XA	SAVEPOINTS
Uninstall example engine.
connection master;
Get binlog position before uninstall plugin.
UNINSTALL PLUGIN example;
Get binlog position after uninstall plugin.
Compute the difference of the  binlog positions.
Should be zero as uninstall plugin should not be replicated.
Delta
0
Verify that example engine is not installed.
SELECT * FROM INFORMATION_SCHEMA.ENGINES WHERE ENGINE='EXAMPLE';
ENGINE	SUPPORT	COMMENT	TRANSACTIONS	XA	SAVEPOINTS
connection slave;
include/rpl_end.inc
End of test
 
 
 
 - found 'core' (0/0)
 
Trying 'dbx' to get a backtrace
 
Trying 'gdb' to get a backtrace from coredump /mnt/buildbot/build/mariadb-10.4.13/mysql-test/var/3/log/rpl.rpl_plugin_load-stmt/mysqld.2/data/core
Core generated by '/mnt/buildbot/build/mariadb-10.4.13/sql/mysqld'
Output from gdb follows. The first stack trace is from the failing thread.
The following stack traces are from all threads (so the failing one is
duplicated).
--------------------------
[New LWP 16010]
[New LWP 16003]
[New LWP 15997]
[New LWP 16002]
[New LWP 16005]
[New LWP 16006]
[New LWP 16007]
[New LWP 16008]
[New LWP 16009]
[New LWP 16011]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `/mnt/buildbot/build/mariadb-10.4.13/sql/mysqld --defaults-group-suffix=.2 --def'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x811ebb99 in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/my_read.c:63
63	      int got_errno= my_errno= errno;
[Current thread is 1 (Thread 0xad5f9b40 (LWP 16010))]
#0  0x811ebb99 in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/my_read.c:63
#1  0x809711b1 in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/signal_handler.cc:66
#2  0x80971999 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb77b8c31 in __kernel_vsyscall ()
#5  0xb7096e89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb70983e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb708fd07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb708fd8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x8108edc3 in Check::validate (space=0x831116f0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4555
#10 0x8108c7dd in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4586
#11 0x8107c4b8 in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:216
#12 0x8108bc88 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4347
#13 0x80ef8b09 in io_handler_thread (arg=0x82568774 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#14 0xb77122b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb715216e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 10 (Thread 0xacdf8b40 (LWP 16011)):
#0  0xb77b8c31 in __kernel_vsyscall ()
#1  0xb7717a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80ddbd64 in os_event::wait (this=0x830d7d00) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:158
#3  0x80ddb7c5 in os_event::wait_low (this=0x830d7d00, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:325
#4  0x80ddbb0b in os_event_wait_low (event=0x830d7d00, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:502
#5  0x810084dd in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/buf/buf0flu.cc:3044
#6  0xb77122b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb715216e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 9 (Thread 0xaddfab40 (LWP 16009)):
#0  0xb7784570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb77845e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80dcbec3 in LinuxAIOHandler::collect (this=0xaddfa0e0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
#3  0x80dcc5cf in LinuxAIOHandler::poll (this=0xaddfa0e0, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
#4  0x80dcc733 in os_aio_linux_handler (global_segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
#5  0x80dd1cbf in os_aio_handler (segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
#6  0x8108bd09 in fil_aio_wait (segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4349
#7  0x80ef8b09 in io_handler_thread (arg=0x82568770 <n+16>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#8  0xb77122b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb715216e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 8 (Thread 0xae5fbb40 (LWP 16008)):
#0  0xb7784570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb77845e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80dcbec3 in LinuxAIOHandler::collect (this=0xae5fb0e0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
#3  0x80dcc5cf in LinuxAIOHandler::poll (this=0xae5fb0e0, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
#4  0x80dcc733 in os_aio_linux_handler (global_segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
#5  0x80dd1cbf in os_aio_handler (segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
#6  0x8108bd09 in fil_aio_wait (segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4349
#7  0x80ef8b09 in io_handler_thread (arg=0x8256876c <n+12>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#8  0xb77122b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb715216e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 7 (Thread 0xaedfcb40 (LWP 16007)):
#0  0xb7784570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb77845e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80dcbec3 in LinuxAIOHandler::collect (this=0xaedfc0e0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
#3  0x80dcc5cf in LinuxAIOHandler::poll (this=0xaedfc0e0, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
#4  0x80dcc733 in os_aio_linux_handler (global_segment=2, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
#5  0x80dd1cbf in os_aio_handler (segment=2, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
#6  0x8108bd09 in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4349
#7  0x80ef8b09 in io_handler_thread (arg=0x82568768 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#8  0xb77122b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb715216e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 6 (Thread 0xaf5fdb40 (LWP 16006)):
#0  0xb7784570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb77845e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80dcbec3 in LinuxAIOHandler::collect (this=0xaf5fd0e0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
#3  0x80dcc5cf in LinuxAIOHandler::poll (this=0xaf5fd0e0, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
#4  0x80dcc733 in os_aio_linux_handler (global_segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
#5  0x80dd1cbf in os_aio_handler (segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
#6  0x8108bd09 in fil_aio_wait (segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4349
#7  0x80ef8b09 in io_handler_thread (arg=0x82568764 <n+4>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#8  0xb77122b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb715216e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 5 (Thread 0xafdfeb40 (LWP 16005)):
#0  0xb7784570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb77845e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80dcbec3 in LinuxAIOHandler::collect (this=0xafdfe0e0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
#3  0x80dcc5cf in LinuxAIOHandler::poll (this=0xafdfe0e0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
#4  0x80dcc733 in os_aio_linux_handler (global_segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
#5  0x80dd1cbf in os_aio_handler (segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
#6  0x8108bd09 in fil_aio_wait (segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4349
#7  0x80ef8b09 in io_handler_thread (arg=0x82568760 <n>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#8  0xb77122b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb715216e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 4 (Thread 0xb2c3db40 (LWP 16002)):
#0  0xb77b8c31 in __kernel_vsyscall ()
#1  0xb7717e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x811f5187 in safe_cond_timedwait (cond=0x8257d180 <COND_timer>, mp=0x8257d100 <LOCK_timer>, abstime=0xb2c3d250, file=0x816b3fe8 "/home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/thr_mutex.c:546
#3  0x811f6139 in inline_mysql_cond_timedwait (that=0x8257d180 <COND_timer>, mutex=0x8257d100 <LOCK_timer>, abstime=0xb2c3d250, src_file=0x816b4038 "/home/buildbot/buildbot/build/mariadb-10.4.13/mysys/thr_timer.c", src_line=292) at /home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h:1211
#4  0x811f6d18 in timer_handler (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/thr_timer.c:292
#5  0x80bd29fb in pfs_spawn_thread (arg=0x82f0bb08) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/perfschema/pfs.cc:1869
#6  0xb77122b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb715216e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 3 (Thread 0xb6ed87c0 (LWP 15997)):
#0  0xb77b8c31 in __kernel_vsyscall ()
#1  0xb7717a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80ddbd64 in os_event::wait (this=0x8301ba20) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:158
#3  0x80ddb7c5 in os_event::wait_low (this=0x8301ba20, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:325
#4  0x80ddbb0b in os_event_wait_low (event=0x8301ba20, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:502
#5  0x80f01d84 in sync_array_wait_event (arr=0x82fabc18, cell=@0xbfee54d0: 0x82fabca8) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/sync/sync0arr.cc:471
#6  0x80d0235d in TTASEventMutex<GenericPolicy>::enter (this=0x81b5f690 <fil_system+16>, max_spins=60, max_delay=4, filename=0x81660cd8 "/home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc", line=4574) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/include/ib0mutex.h:477
#7  0x80d00484 in PolicyMutex<TTASEventMutex<GenericPolicy> >::enter (this=0x81b5f690 <fil_system+16>, n_spins=30, n_delay=4, name=0x81660cd8 "/home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc", line=4574) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/include/ib0mutex.h:594
#8  0x8108c78c in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4574
#9  0x810b2254 in SysTablespace::open_or_create (this=0x82575a00 <srv_sys_space>, is_temp=false, create_new_db=false, sum_new_sizes=0xbfee5660, flush_lsn=0xbfee56b0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fsp/fsp0sysspace.cc:924
#10 0x80efd005 in srv_start (create_new_db=false) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:1583
#11 0x80cd6422 in innodb_init (p=0x82faa168) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/handler/ha_innodb.cc:4263
#12 0x80975841 in ha_initialize_handlerton (plugin=0x82f3aaa8) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/handler.cc:557
#13 0x8063bb11 in plugin_initialize (tmp_root=0xbfeea868, plugin=0x82f3aaa8, argc=0x81d06f34 <remaining_argc>, argv=0x82de7330, options_only=false) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/sql_plugin.cc:1438
#14 0x8063c7a3 in plugin_init (argc=0x81d06f34 <remaining_argc>, argv=0x82de7330, flags=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/sql_plugin.cc:1720
#15 0x804e8ad3 in init_server_components () at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:5207
#16 0x804ea015 in mysqld_main (argc=142, argv=0x82de7330) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:5730
#17 0x804ddd4b in main (argc=9, argv=0xbfeeb554) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/main.cc:25
 
Thread 2 (Thread 0xb17ffb40 (LWP 16003)):
#0  0xb77b8c31 in __kernel_vsyscall ()
#1  0xb7717e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x811f5187 in safe_cond_timedwait (cond=0x82534760 <COND_checkpoint>, mp=0x825346e0 <LOCK_checkpoint>, abstime=0xb17ff19c, file=0x815250f8 "/home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/thr_mutex.c:546
#3  0x80c3ba9c in inline_mysql_cond_timedwait (that=0x82534760 <COND_checkpoint>, mutex=0x825346e0 <LOCK_checkpoint>, abstime=0xb17ff19c, src_file=0x81525148 "/home/buildbot/buildbot/build/mariadb-10.4.13/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h:1211
#4  0x80c3c001 in my_service_thread_sleep (control=0x81b54418 <checkpoint_control>, sleep_time=29000000000) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/maria/ma_servicethread.c:115
#5  0x80c2f7be in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/maria/ma_checkpoint.c:707
#6  0x80bd29fb in pfs_spawn_thread (arg=0x82f95020) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/perfschema/pfs.cc:1869
#7  0xb77122b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#8  0xb715216e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 1 (Thread 0xad5f9b40 (LWP 16010)):
#0  0x811ebb99 in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/my_read.c:63
#1  0x809711b1 in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/signal_handler.cc:66
#2  0x80971999 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb77b8c31 in __kernel_vsyscall ()
#5  0xb7096e89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb70983e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb708fd07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb708fd8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x8108edc3 in Check::validate (space=0x831116f0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4555
#10 0x8108c7dd in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4586
#11 0x8107c4b8 in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:216
#12 0x8108bc88 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4347
#13 0x80ef8b09 in io_handler_thread (arg=0x82568774 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
#14 0xb77122b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb715216e in clone () from /lib/i386-linux-gnu/libc.so.6

Comment by Elena Stepanova [ 2020-06-19 ]

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/22687

rpl.rpl_mdev-11092 'innodb,row'          w2 [ fail ]
        Test ended at 2020-05-20 11:41:53
 
CURRENT_TEST: rpl.rpl_mdev-11092
 
 
Failed to start mysqld.2
mysqltest failed but provided no output
 
 
 - found 'core' (0/0)
 
Trying 'dbx' to get a backtrace
 
Trying 'gdb' to get a backtrace from coredump /mnt/buildbot/build/mariadb-10.4.14/mysql-test/var/2/log/rpl.rpl_mdev-11092-innodb,row/mysqld.2/data/core
Core generated by '/mnt/buildbot/build/mariadb-10.4.14/sql/mysqld'
Output from gdb follows. The first stack trace is from the failing thread.
The following stack traces are from all threads (so the failing one is
duplicated).
--------------------------
[New LWP 2904]
[New LWP 2873]
[New LWP 2892]
[New LWP 2893]
[New LWP 2899]
[New LWP 2900]
[New LWP 2901]
[New LWP 2902]
[New LWP 2903]
[New LWP 2905]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `/mnt/buildbot/build/mariadb-10.4.14/sql/mysqld --defaults-group-suffix=.2 --def'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x8110b6fd in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.14/mysys/my_read.c:63
63	      int got_errno= my_errno= errno;
[Current thread is 1 (Thread 0xad5f9b40 (LWP 2904))]
#0  0x8110b6fd in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.14/mysys/my_read.c:63
#1  0x80895f77 in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.14/sql/signal_handler.cc:66
#2  0x8089675f in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.14/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb76ffc31 in __kernel_vsyscall ()
#5  0xb6fdde89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb6fdf3e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb6fd6d07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb6fd6d8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x80faf4e1 in Check::validate (space=0x841c8c28) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fil/fil0fil.cc:4556
#10 0x80facefb in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fil/fil0fil.cc:4587
#11 0x80f9cbda in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fil/fil0fil.cc:216
#12 0x80fac3a6 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fil/fil0fil.cc:4348
#13 0x80e19f5d in io_handler_thread (arg=0x824898f4 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/srv/srv0start.cc:324
#14 0xb76592b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb709916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 10 (Thread 0xacdf8b40 (LWP 2905)):
#0  0xb76ffc31 in __kernel_vsyscall ()
#1  0xb765ea8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80cffc28 in os_event::wait (this=0x84180b30) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0event.cc:158
#3  0x80cff689 in os_event::wait_low (this=0x84180b30, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0event.cc:325
#4  0x80cff9cf in os_event_wait_low (event=0x84180b30, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0event.cc:502
#5  0x80f28487 in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/buf/buf0flu.cc:3044
#6  0xb76592b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb709916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 9 (Thread 0xaddfab40 (LWP 2903)):
#0  0xb76cb570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76cb5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80cefd87 in LinuxAIOHandler::collect (this=0xaddfa0e0) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:1819
#3  0x80cf0493 in LinuxAIOHandler::poll (this=0xaddfa0e0, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:1980
#4  0x80cf05f7 in os_aio_linux_handler (global_segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:2034
#5  0x80cf5b83 in os_aio_handler (segment=4, m1=0xaddfa150, m2=0xaddfa154, request=0xaddfa178) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:5604
#6  0x80fac427 in fil_aio_wait (segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fil/fil0fil.cc:4350
#7  0x80e19f5d in io_handler_thread (arg=0x824898f0 <n+16>) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/srv/srv0start.cc:324
#8  0xb76592b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb709916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 8 (Thread 0xae5fbb40 (LWP 2902)):
#0  0xb76cb570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76cb5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80cefd87 in LinuxAIOHandler::collect (this=0xae5fb0e0) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:1819
#3  0x80cf0493 in LinuxAIOHandler::poll (this=0xae5fb0e0, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:1980
#4  0x80cf05f7 in os_aio_linux_handler (global_segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:2034
#5  0x80cf5b83 in os_aio_handler (segment=3, m1=0xae5fb150, m2=0xae5fb154, request=0xae5fb178) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:5604
#6  0x80fac427 in fil_aio_wait (segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fil/fil0fil.cc:4350
#7  0x80e19f5d in io_handler_thread (arg=0x824898ec <n+12>) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/srv/srv0start.cc:324
#8  0xb76592b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb709916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 7 (Thread 0xaedfcb40 (LWP 2901)):
#0  0xb76cb570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76cb5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80cefd87 in LinuxAIOHandler::collect (this=0xaedfc0e0) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:1819
#3  0x80cf0493 in LinuxAIOHandler::poll (this=0xaedfc0e0, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:1980
#4  0x80cf05f7 in os_aio_linux_handler (global_segment=2, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:2034
#5  0x80cf5b83 in os_aio_handler (segment=2, m1=0xaedfc150, m2=0xaedfc154, request=0xaedfc178) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:5604
#6  0x80fac427 in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fil/fil0fil.cc:4350
#7  0x80e19f5d in io_handler_thread (arg=0x824898e8 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/srv/srv0start.cc:324
#8  0xb76592b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb709916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 6 (Thread 0xaf5fdb40 (LWP 2900)):
#0  0xb76cb570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76cb5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80cefd87 in LinuxAIOHandler::collect (this=0xaf5fd0e0) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:1819
#3  0x80cf0493 in LinuxAIOHandler::poll (this=0xaf5fd0e0, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:1980
#4  0x80cf05f7 in os_aio_linux_handler (global_segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:2034
#5  0x80cf5b83 in os_aio_handler (segment=1, m1=0xaf5fd150, m2=0xaf5fd154, request=0xaf5fd178) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:5604
#6  0x80fac427 in fil_aio_wait (segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fil/fil0fil.cc:4350
#7  0x80e19f5d in io_handler_thread (arg=0x824898e4 <n+4>) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/srv/srv0start.cc:324
#8  0xb76592b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb709916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 5 (Thread 0xafdfeb40 (LWP 2899)):
#0  0xb76cb570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
#1  0xb76cb5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
#2  0x80cefd87 in LinuxAIOHandler::collect (this=0xafdfe0e0) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:1819
#3  0x80cf0493 in LinuxAIOHandler::poll (this=0xafdfe0e0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:1980
#4  0x80cf05f7 in os_aio_linux_handler (global_segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:2034
#5  0x80cf5b83 in os_aio_handler (segment=0, m1=0xafdfe150, m2=0xafdfe154, request=0xafdfe178) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0file.cc:5604
#6  0x80fac427 in fil_aio_wait (segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fil/fil0fil.cc:4350
#7  0x80e19f5d in io_handler_thread (arg=0x824898e0 <n>) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/srv/srv0start.cc:324
#8  0xb76592b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#9  0xb709916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 4 (Thread 0xb170eb40 (LWP 2893)):
#0  0xb76ffc31 in __kernel_vsyscall ()
#1  0xb765ee56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x81114d9f in safe_cond_timedwait (cond=0x824558e0 <COND_checkpoint>, mp=0x82455860 <LOCK_checkpoint>, abstime=0xb170e19c, file=0x81444478 "/home/buildbot/buildbot/build/mariadb-10.4.14/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.14/mysys/thr_mutex.c:546
#3  0x80b5e75c in inline_mysql_cond_timedwait (that=0x824558e0 <COND_checkpoint>, mutex=0x82455860 <LOCK_checkpoint>, abstime=0xb170e19c, src_file=0x814444c8 "/home/buildbot/buildbot/build/mariadb-10.4.14/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/buildbot/build/mariadb-10.4.14/include/mysql/psi/mysql_thread.h:1211
#4  0x80b5ecc1 in my_service_thread_sleep (control=0x81a76418 <checkpoint_control>, sleep_time=29000000000) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/maria/ma_servicethread.c:115
#5  0x80b5247e in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/maria/ma_checkpoint.c:707
#6  0x80af568b in pfs_spawn_thread (arg=0x8403dfd8) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/perfschema/pfs.cc:1869
#7  0xb76592b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#8  0xb709916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 3 (Thread 0xb2b84b40 (LWP 2892)):
#0  0xb76ffc31 in __kernel_vsyscall ()
#1  0xb765ee56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x81114d9f in safe_cond_timedwait (cond=0x8249e340 <COND_timer>, mp=0x8249e2c0 <LOCK_timer>, abstime=0xb2b84250, file=0x815d3914 "/home/buildbot/buildbot/build/mariadb-10.4.14/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.14/mysys/thr_mutex.c:546
#3  0x81115d51 in inline_mysql_cond_timedwait (that=0x8249e340 <COND_timer>, mutex=0x8249e2c0 <LOCK_timer>, abstime=0xb2b84250, src_file=0x815d3964 "/home/buildbot/buildbot/build/mariadb-10.4.14/mysys/thr_timer.c", src_line=292) at /home/buildbot/buildbot/build/mariadb-10.4.14/include/mysql/psi/mysql_thread.h:1211
#4  0x81116930 in timer_handler (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.14/mysys/thr_timer.c:292
#5  0x80af568b in pfs_spawn_thread (arg=0x83fb4b08) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/perfschema/pfs.cc:1869
#6  0xb76592b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#7  0xb709916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Thread 2 (Thread 0xb6e1f7c0 (LWP 2873)):
#0  0xb76ffc31 in __kernel_vsyscall ()
#1  0xb765ea8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x80cffc28 in os_event::wait (this=0x840c4868) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0event.cc:158
#3  0x80cff689 in os_event::wait_low (this=0x840c4868, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0event.cc:325
#4  0x80cff9cf in os_event_wait_low (event=0x840c4868, reset_sig_count=2) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/os/os0event.cc:502
#5  0x80e23256 in sync_array_wait_event (arr=0x84054ac8, cell=@0xbfc4aa20: 0x84054b58) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/sync/sync0arr.cc:471
#6  0x80c24519 in TTASEventMutex<GenericPolicy>::enter (this=0x81a80810 <fil_system+16>, max_spins=60, max_delay=4, filename=0x815807f0 "/home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fil/fil0fil.cc", line=4575) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/include/ib0mutex.h:477
#7  0x80c2267e in PolicyMutex<TTASEventMutex<GenericPolicy> >::enter (this=0x81a80810 <fil_system+16>, n_spins=30, n_delay=4, name=0x815807f0 "/home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fil/fil0fil.cc", line=4575) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/include/ib0mutex.h:594
#8  0x80faceaa in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fil/fil0fil.cc:4575
#9  0x80fd27ca in SysTablespace::open_or_create (this=0x82496bc0 <srv_sys_space>, is_temp=false, create_new_db=false, sum_new_sizes=0xbfc4abb0, flush_lsn=0xbfc4ac00) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fsp/fsp0sysspace.cc:924
#10 0x80e1e440 in srv_start (create_new_db=false) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/srv/srv0start.cc:1585
#11 0x80bf8e38 in innodb_init (p=0x8403dfd8) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/handler/ha_innodb.cc:4210
#12 0x8089a215 in ha_initialize_handlerton (plugin=0x83fe3aa8) at /home/buildbot/buildbot/build/mariadb-10.4.14/sql/handler.cc:557
#13 0x805684fb in plugin_initialize (tmp_root=0xbfc4fdb8, plugin=0x83fe3aa8, argc=0x81c280b4 <remaining_argc>, argv=0x83e90330, options_only=false) at /home/buildbot/buildbot/build/mariadb-10.4.14/sql/sql_plugin.cc:1438
#14 0x8056918d in plugin_init (argc=0x81c280b4 <remaining_argc>, argv=0x83e90330, flags=0) at /home/buildbot/buildbot/build/mariadb-10.4.14/sql/sql_plugin.cc:1720
#15 0x804195e2 in init_server_components () at /home/buildbot/buildbot/build/mariadb-10.4.14/sql/mysqld.cc:5218
#16 0x8041ab0c in mysqld_main (argc=161, argv=0x83e90330) at /home/buildbot/buildbot/build/mariadb-10.4.14/sql/mysqld.cc:5747
#17 0x8040eabb in main (argc=28, argv=0xbfc50524) at /home/buildbot/buildbot/build/mariadb-10.4.14/sql/main.cc:25
 
Thread 1 (Thread 0xad5f9b40 (LWP 2904)):
#0  0x8110b6fd in my_read (Filedes=7, Buffer=0xad5f790c "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=4096, MyFlags=0) at /home/buildbot/buildbot/build/mariadb-10.4.14/mysys/my_read.c:63
#1  0x80895f77 in output_core_info () at /home/buildbot/buildbot/build/mariadb-10.4.14/sql/signal_handler.cc:66
#2  0x8089675f in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.14/sql/signal_handler.cc:339
#3  <signal handler called>
#4  0xb76ffc31 in __kernel_vsyscall ()
#5  0xb6fdde89 in raise () from /lib/i386-linux-gnu/libc.so.6
#6  0xb6fdf3e7 in abort () from /lib/i386-linux-gnu/libc.so.6
#7  0xb6fd6d07 in ?? () from /lib/i386-linux-gnu/libc.so.6
#8  0xb6fd6d8b in __assert_fail () from /lib/i386-linux-gnu/libc.so.6
#9  0x80faf4e1 in Check::validate (space=0x841c8c28) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fil/fil0fil.cc:4556
#10 0x80facefb in fil_validate () at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fil/fil0fil.cc:4587
#11 0x80f9cbda in fil_validate_skip () at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fil/fil0fil.cc:216
#12 0x80fac3a6 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/fil/fil0fil.cc:4348
#13 0x80e19f5d in io_handler_thread (arg=0x824898f4 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.14/storage/innobase/srv/srv0start.cc:324
#14 0xb76592b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#15 0xb709916e in clone () from /lib/i386-linux-gnu/libc.so.6
 
Compressed file /mnt/buildbot/build/mariadb-10.4.14/mysql-test/var/2/log/rpl.rpl_mdev-11092-innodb,row/mysqld.2/data/core
 - skipping '/mnt/buildbot/build/mariadb-10.4.14/mysql-test/var/2/log/rpl.rpl_mdev-11092-innodb,row/'

Comment by Eugene Kosov (Inactive) [ 2020-06-27 ]

Failing assertion is:

		ut_ad(space->id != TRX_SYS_SPACE
		      || space == fil_system.sys_space);

Here is a sequence of events during the server start:

1. fil_system.sys_space = nullptr during a global variable initialization

2. TRX_SYS_SPACE fil_space_t is created:

Thread 1 hit Breakpoint 3, fil_space_create (name=0x536073 "innodb_system", id=0, flags=0, purpose=FIL_TYPE_TABLESPACE, crypt_data=0x0, mode=FIL_ENCRYPTION_DEFAULT) at fil0fil.cc:1224
1224		space->id = id;
(rr) bt
#0  fil_space_create (name=0x536073 "innodb_system", id=0, flags=0, purpose=FIL_TYPE_TABLESPACE, crypt_data=0x0, mode=FIL_ENCRYPTION_DEFAULT) at fil0fil.cc:1224
#1  0x0000000001417eea in SysTablespace::open_or_create (this=0x1e0fee8 <srv_sys_space>, is_temp=false, create_new_db=false, sum_new_sizes=0x7ffef9099be8, flush_lsn=0x7ffef909dbc0) at fsp0sysspace.cc:916
#2  0x0000000001725145 in srv_start (create_new_db=false) at srv0start.cc:1584
#3  0x00000000012acbab in innodb_init (p=0x324e648) at ha_innodb.cc:4211
#4  0x0000000000b5421b in ha_initialize_handlerton (plugin=0x31893b0) at handler.cc:557
#5  0x0000000000f0556e in plugin_initialize (tmp_root=0x7ffef909e220, plugin=0x31893b0, argc=0x1dd93d8 <remaining_argc>, argv=0x3059be8, options_only=false) at sql_plugin.cc:1438
#6  0x0000000000f0501a in plugin_init (argc=0x1dd93d8 <remaining_argc>, argv=0x3059be8, flags=0) at sql_plugin.cc:1720
#7  0x0000000000b1b168 in init_server_components () at mysqld.cc:5217
#8  0x0000000000b175cc in mysqld_main (argc=147, argv=0x3059be8) at mysqld.cc:5746
#9  0x0000000000b136d2 in main (argc=23, argv=0x7ffef909e9a8) at main.cc:25
(rr) p fil_system.sys_space
$1 = (fil_space_t *) 0x0

At this point it becomes visible to Check::validate()! And the topic assertion will fail, because fil_system.sys_space is still a nullptr.

3. fil_system.sys_space is initialized with a just created fil_space_t*.

Thread 1 hit Hardware watchpoint 4: fil_system.sys_space
 
Old value = (fil_space_t *) 0x0
New value = (fil_space_t *) 0x347f660
0x0000000001417ef8 in SysTablespace::open_or_create (this=0x1e0fee8 <srv_sys_space>, is_temp=false, create_new_db=false, sum_new_sizes=0x7ffef9099be8, flush_lsn=0x7ffef909dbc0) at fsp0sysspace.cc:916
916				space = fil_system.sys_space = fil_space_create(

So, the topic assertion should be relaxed to allow fil_system.sys_space to be nullptr.

Comment by Eugene Kosov (Inactive) [ 2020-06-27 ]

There is not such an assertion before 10.3, so the fix should go into 10.3:
https://github.com/MariaDB/server/blob/97f7d4a9b4da77cb79699a0ea873e4a0e628e8a3/storage/innobase/fil/fil0fil.cc#L5300
https://github.com/MariaDB/server/blob/697273554f0952160baa10606c02be1f6cf3c6c5/storage/innobase/fil/fil0fil.cc#L4638

Generated at Thu Feb 08 08:57:42 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.