Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Won't Fix
-
10.2(EOL), 10.3(EOL), 10.4(EOL)
Description
http://buildbot.askmonty.org/buildbot/builders/kvm-deb-jessie-x86/builds/3235/steps/mtr/logs/stdio
innodb.innodb_zip_innochecksum3 '16k,innodb' w4 [ fail ]
|
Test ended at 2017-11-14 17:37:32
|
|
CURRENT_TEST: innodb.innodb_zip_innochecksum3
|
Error: Unable to lock file:: /dev/shm/var/4/mysqld.1/data//test/tab1.ibd
|
fcntl: Resource temporarily unavailable
|
mysqltest: At line 109: exec of '/usr/bin/innochecksum --write=CRC32 /dev/shm/var/4/mysqld.1/data//test/tab1.ibd' failed, error: 256, status: 1, errno: 32
|
Output from before failure:
|
[1(b)]: Rewrite into new checksum=crc32 for all *.ibd file and ibdata1
|
|
|
|
The result from queries just before the failure was:
|
< snip >
|
VALUES (@col_1,@col_2,@col_3,@col_4,@cl_5,@col_6,1);
|
SELECT * FROM tab2 ORDER BY col_7;
|
# stop the server
|
[1(a)]: Rewrite into new checksum=InnoDB for all *.ibd file and ibdata1
|
: start the server with innodb_checksum_algorithm=strict_innodb
|
INSERT INTO tab1 (pk, linestring_key, linestring_nokey)
|
VALUES (2, ST_GeomFromText('LINESTRING(10 10,20 20,30 30)'), ST_GeomFromText('LINESTRING(10 10,20 20,30 30)'));
|
SET @col_1 = repeat('a', 5);
|
SET @col_2 = repeat('b', 20);
|
SET @col_3 = repeat('c', 100);
|
SET @col_4 = repeat('d', 100);
|
SET @col_5 = repeat('e', 100);
|
SET @col_6 = repeat('f', 100);
|
INSERT INTO tab2(col_1,col_2,col_3,col_4,col_5,col_6,col_7)
|
VALUES (@col_1,@col_2,@col_3,@col_4,@cl_5,@col_6,6);
|
SELECT pk,ST_AsText(linestring_key),ST_AsText(linestring_nokey)
|
FROM tab1 ORDER BY pk;
|
SELECT * FROM tab2 ORDER BY col_7;
|
# stop the server
|
[1(b)]: Rewrite into new checksum=crc32 for all *.ibd file and ibdata1
|
|
More results from queries before failure can be found in /dev/shm/var/4/log/innodb_zip_innochecksum3.log
|
Attachments
Issue Links
- is duplicated by
-
MDEV-20158 innodb_zip.innochecksum failed in buildbot, server didn't shut down
-
- Closed
-
- relates to
-
MDEV-14080 InnoDB shutdown sometimes hangs
-
- Closed
-
-
MDEV-22577 innodb_fast_shutdown=0 fails to report purge progress
-
- Closed
-
-
MDEV-26264 server wont start after clean shutdown
-
- Open
-
-
MDEV-27125 galera_new_cluster hangs with galera_recovery
-
- Open
-
-
MDEV-32033 all queries get stuck until a timeout occurs on all of them
-
- Open
-
-
MDEV-26327 Assertion `!"purge was not shut down"' in logs_empty_and_mark_files_at_shutdown during bootstrap
-
- Open
-
Activity
We got a recent occurrence on a 10.4-based branch:
innodb.innodb-get-fk 'innodb' w3 [ fail ]
|
Test ended at 2021-03-04 18:03:53
|
|
CURRENT_TEST: innodb.innodb-get-fk
|
mysqltest: At line 41: query 'ALTER TABLE `repro`.`crew_role_assigned` COMMENT = 'innodb_read_only'' failed with wrong errno 1286: 'Unknown storage engine 'InnoDB'', instead of 1036...
|
…
|
Thread 19 (Thread 0xae7fdb40 (LWP 25014)):
|
#0 0xb63f54f4 in ?? () from /lib/i386-linux-gnu/libaio.so.1
|
#1 0xb63f5548 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
|
#2 0xb6cd0282 in LinuxAIOHandler::collect (this=this@entry=0xae7fd1b0) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:1819
|
#3 0xb6cd648a in LinuxAIOHandler::poll (this=this@entry=0xae7fd1b0, m1=m1@entry=0xae7fd240, m2=m2@entry=0xae7fd244, request=request@entry=0xae7fd250) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:1980
|
#4 0xb6cda4ab in os_aio_linux_handler (request=0xae7fd250, m2=0xae7fd244, m1=0xae7fd240, global_segment=1) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:2034
|
#5 os_aio_handler (segment=segment@entry=1, m1=m1@entry=0xae7fd240, m2=m2@entry=0xae7fd244, request=request@entry=0xae7fd250) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:5607
|
#6 0xb6e65ec0 in fil_aio_wait (segment=segment@entry=1) at /home/buildbot/buildbot/build/storage/innobase/fil/fil0fil.cc:4315
|
#7 0xb6d6d120 in io_handler_thread (arg=0xb793d944 <n+4>) at /home/buildbot/buildbot/build/storage/innobase/srv/srv0start.cc:324
|
#8 0xb63aef72 in start_thread (arg=0xae7fdb40) at pthread_create.c:312
|
#9 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 18 (Thread 0xa6bf1b40 (LWP 25030)):
|
#0 0xb63fe424 in __kernel_vsyscall ()
|
#1 0xb63b2d0b in pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
|
#2 0xb6cdb50f in wait (this=<optimized out>) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:158
|
#3 wait_low (reset_sig_count=<optimized out>, this=0xb9dcfab8) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:325
|
#4 os_event_wait_low (event=0xb9dcfab8, reset_sig_count=0) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:502
|
#5 0xb6d6af30 in srv_resume_thread (sig_count=0, timeout_usec=0, wait=true, slot=0xb787e9c4 <srv_sys+196>) at /home/buildbot/buildbot/build/storage/innobase/srv/srv0srv.cc:907
|
#6 srv_worker_thread (arg=0x0) at /home/buildbot/buildbot/build/storage/innobase/srv/srv0srv.cc:2514
|
#7 0xb63aef72 in start_thread (arg=0xa6bf1b40) at pthread_create.c:312
|
#8 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 17 (Thread 0xa4ffdb40 (LWP 25033)):
|
#0 0xb63fe424 in __kernel_vsyscall ()
|
#1 0xb63b2d0b in pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
|
#2 0xb6cdb50f in wait (this=<optimized out>) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:158
|
#3 wait_low (reset_sig_count=<optimized out>, this=0xb9dcfc70) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:325
|
#4 os_event_wait_low (event=0xb9dcfc70, reset_sig_count=0) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:502
|
#5 0xb6dff370 in buf_resize_thread () at /home/buildbot/buildbot/build/storage/innobase/buf/buf0buf.cc:3142
|
#6 0xb63aef72 in start_thread (arg=0xa4ffdb40) at pthread_create.c:312
|
#7 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 16 (Thread 0xb09cab40 (LWP 25012)):
|
#0 0xb63fe424 in __kernel_vsyscall ()
|
#1 0xb63b30ed in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:245
|
#2 0xb6f19280 in inline_mysql_cond_timedwait (src_file=0xb735af04 "/home/buildbot/buildbot/build/storage/maria/ma_servicethread.c", src_line=116, abstime=0xb09ca224, mutex=0xb814bf34 <LOCK_checkpoint>, that=0xb814bf00 <COND_checkpoint>) at /home/buildbot/buildbot/build/include/mysql/psi/mysql_thread.h:1211
|
#3 my_service_thread_sleep (control=control@entry=0xb7880384 <checkpoint_control>, sleep_time=30000000000) at /home/buildbot/buildbot/build/storage/maria/ma_servicethread.c:115
|
#4 0xb6f0e84d in ma_checkpoint_background (arg=arg@entry=0x1e) at /home/buildbot/buildbot/build/storage/maria/ma_checkpoint.c:707
|
#5 0xb7012fcd in pfs_spawn_thread (arg=0xb9da4f08) at /home/buildbot/buildbot/build/storage/perfschema/pfs.cc:1869
|
#6 0xb63aef72 in start_thread (arg=0xb09cab40) at pthread_create.c:312
|
#7 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 15 (Thread 0xac7f9b40 (LWP 25018)):
|
#0 0xb63f54f4 in ?? () from /lib/i386-linux-gnu/libaio.so.1
|
#1 0xb63f5548 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
|
#2 0xb6cd0282 in LinuxAIOHandler::collect (this=this@entry=0xac7f91b0) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:1819
|
#3 0xb6cd648a in LinuxAIOHandler::poll (this=this@entry=0xac7f91b0, m1=m1@entry=0xac7f9240, m2=m2@entry=0xac7f9244, request=request@entry=0xac7f9250) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:1980
|
#4 0xb6cda4ab in os_aio_linux_handler (request=0xac7f9250, m2=0xac7f9244, m1=0xac7f9240, global_segment=5) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:2034
|
#5 os_aio_handler (segment=segment@entry=5, m1=m1@entry=0xac7f9240, m2=m2@entry=0xac7f9244, request=request@entry=0xac7f9250) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:5607
|
#6 0xb6e65ec0 in fil_aio_wait (segment=segment@entry=5) at /home/buildbot/buildbot/build/storage/innobase/fil/fil0fil.cc:4315
|
#7 0xb6d6d120 in io_handler_thread (arg=0xb793d954 <n+20>) at /home/buildbot/buildbot/build/storage/innobase/srv/srv0start.cc:324
|
#8 0xb63aef72 in start_thread (arg=0xac7f9b40) at pthread_create.c:312
|
#9 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 14 (Thread 0xaabf9b40 (LWP 25021)):
|
#0 0xb63fe424 in __kernel_vsyscall ()
|
#1 0xb63b30ed in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:245
|
#2 0xb6cdafe2 in timed_wait (abstime=0xaabf9194, this=0xb9edad70) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:275
|
#3 os_event::wait_time_low (this=0xb9edad70, time_in_usec=1000000, reset_sig_count=<optimized out>) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:385
|
#4 0xb6cdb499 in os_event_wait_time_low (event=event@entry=0xb9edad70, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:485
|
#5 0xb6cae4ce in lock_wait_timeout_thread () at /home/buildbot/buildbot/build/storage/innobase/lock/lock0wait.cc:522
|
#6 0xb63aef72 in start_thread (arg=0xaabf9b40) at pthread_create.c:312
|
#7 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 13 (Thread 0xabff8b40 (LWP 25019)):
|
#0 0xb63fe424 in __kernel_vsyscall ()
|
#1 0xb63b30ed in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:245
|
#2 0xb6cdafe2 in timed_wait (abstime=0xabff80f4, this=0xb9dcfc18) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:275
|
#3 os_event::wait_time_low (this=0xb9dcfc18, time_in_usec=1000000, reset_sig_count=<optimized out>) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:385
|
#4 0xb6cdb499 in os_event_wait_time_low (event=0xb9dcfc18, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=61) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:485
|
#5 0xb6e0db92 in pc_sleep_if_needed (cur_time=2057248, sig_count=61, next_loop_time=2058248) at /home/buildbot/buildbot/build/storage/innobase/buf/buf0flu.cc:2638
|
#6 buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/storage/innobase/buf/buf0flu.cc:3097
|
#7 0xb63aef72 in start_thread (arg=0xabff8b40) at pthread_create.c:312
|
#8 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 12 (Thread 0xaa3f8b40 (LWP 25022)):
|
#0 0xb63fe424 in __kernel_vsyscall ()
|
#1 0xb63b30ed in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:245
|
#2 0xb6cdafe2 in timed_wait (abstime=0xaa3f8164, this=0xb9dcfb10) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:275
|
#3 os_event::wait_time_low (this=0xb9dcfb10, time_in_usec=1000000, reset_sig_count=<optimized out>) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:385
|
#4 0xb6cdb499 in os_event_wait_time_low (event=0xb9dcfb10, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:485
|
#5 0xb6d697b2 in srv_error_monitor_thread () at /home/buildbot/buildbot/build/storage/innobase/srv/srv0srv.cc:1851
|
#6 0xb63aef72 in start_thread (arg=0xaa3f8b40) at pthread_create.c:312
|
#7 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 11 (Thread 0xab3fab40 (LWP 25026)):
|
#0 0xb63fe424 in __kernel_vsyscall ()
|
#1 0xb63b65b6 in nanosleep () at ../sysdeps/unix/syscall-template.S:81
|
#2 0xb6cdb858 in os_thread_sleep (tm=tm@entry=1000000) at /home/buildbot/buildbot/build/storage/innobase/os/os0thread.cc:231
|
#3 0xb6d6a487 in srv_master_sleep () at /home/buildbot/buildbot/build/storage/innobase/srv/srv0srv.cc:2338
|
#4 srv_master_thread (arg=0x0) at /home/buildbot/buildbot/build/storage/innobase/srv/srv0srv.cc:2378
|
#5 0xb63aef72 in start_thread (arg=0xab3fab40) at pthread_create.c:312
|
#6 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 10 (Thread 0xa5fffb40 (LWP 25031)):
|
#0 0xb63fe424 in __kernel_vsyscall ()
|
#1 0xb63b2d0b in pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
|
#2 0xb6cdb50f in wait (this=<optimized out>) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:158
|
#3 wait_low (reset_sig_count=<optimized out>, this=0xb9dcfbc0) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:325
|
#4 os_event_wait_low (event=0xb9dcfbc0, reset_sig_count=0) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:502
|
#5 0xb6e0735e in buf_dump_thread () at /home/buildbot/buildbot/build/storage/innobase/buf/buf0dump.cc:832
|
#6 0xb63aef72 in start_thread (arg=0xa5fffb40) at pthread_create.c:312
|
#7 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 9 (Thread 0xb01c4b40 (LWP 25041)):
|
#0 0xb63fe424 in __kernel_vsyscall ()
|
#1 0xb60e87ba in do_sigwaitinfo (set=0xfffffffc, set@entry=0xb01c41e4, info=info@entry=0xb01c4264) at ../nptl/sysdeps/unix/sysv/linux/../../../../../sysdeps/unix/sysv/linux/sigwaitinfo.c:54
|
#2 0xb60e8859 in __GI___sigwaitinfo (set=0xb01c41e4, info=0xb01c4264) at ../nptl/sysdeps/unix/sysv/linux/../../../../../sysdeps/unix/sysv/linux/sigwaitinfo.c:81
|
#3 0xb6773330 in my_sigwait (code=<synthetic pointer>, sig=0xb01c41e0, set=0xb01c41e4) at /home/buildbot/buildbot/build/include/my_pthread.h:196
|
#4 signal_hand (arg=arg@entry=0x0) at /home/buildbot/buildbot/build/sql/mysqld.cc:3224
|
#5 0xb7012fcd in pfs_spawn_thread (arg=0xb9f58d50) at /home/buildbot/buildbot/build/storage/perfschema/pfs.cc:1869
|
#6 0xb63aef72 in start_thread (arg=0xb01c4b40) at pthread_create.c:312
|
#7 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 8 (Thread 0xb017ab40 (LWP 25042)):
|
#0 0xb63fe424 in __kernel_vsyscall ()
|
#1 0xb63b2d0b in pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:187
|
#2 0xb683dcb5 in inline_mysql_cond_wait (that=0xb78e81a0 <COND_manager>, mutex=0xb78e81d4 <LOCK_manager>, src_file=0xb71d488c "/home/buildbot/buildbot/build/sql/sql_manager.cc", src_line=102) at /home/buildbot/buildbot/build/include/mysql/psi/mysql_thread.h:1174
|
#3 handle_manager (arg=arg@entry=0x0) at /home/buildbot/buildbot/build/sql/sql_manager.cc:102
|
#4 0xb7012fcd in pfs_spawn_thread (arg=0xb9fcec70) at /home/buildbot/buildbot/build/storage/perfschema/pfs.cc:1869
|
#5 0xb63aef72 in start_thread (arg=0xb017ab40) at pthread_create.c:312
|
#6 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 7 (Thread 0xadffcb40 (LWP 25015)):
|
#0 0xb63f54f4 in ?? () from /lib/i386-linux-gnu/libaio.so.1
|
#1 0xb63f5548 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
|
#2 0xb6cd0282 in LinuxAIOHandler::collect (this=this@entry=0xadffc1b0) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:1819
|
#3 0xb6cd648a in LinuxAIOHandler::poll (this=this@entry=0xadffc1b0, m1=m1@entry=0xadffc240, m2=m2@entry=0xadffc244, request=request@entry=0xadffc250) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:1980
|
#4 0xb6cda4ab in os_aio_linux_handler (request=0xadffc250, m2=0xadffc244, m1=0xadffc240, global_segment=2) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:2034
|
#5 os_aio_handler (segment=segment@entry=2, m1=m1@entry=0xadffc240, m2=m2@entry=0xadffc244, request=request@entry=0xadffc250) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:5607
|
#6 0xb6e65ec0 in fil_aio_wait (segment=segment@entry=2) at /home/buildbot/buildbot/build/storage/innobase/fil/fil0fil.cc:4315
|
#7 0xb6d6d120 in io_handler_thread (arg=0xb793d948 <n+8>) at /home/buildbot/buildbot/build/storage/innobase/srv/srv0start.cc:324
|
#8 0xb63aef72 in start_thread (arg=0xadffcb40) at pthread_create.c:312
|
#9 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 6 (Thread 0xa9bf7b40 (LWP 25023)):
|
#0 0xb63fe424 in __kernel_vsyscall ()
|
#1 0xb63b30ed in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:245
|
#2 0xb6cdafe2 in timed_wait (abstime=0xa9bf71e4, this=0xb9dcfb68) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:275
|
#3 os_event::wait_time_low (this=0xb9dcfb68, time_in_usec=5000000, reset_sig_count=<optimized out>) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:385
|
#4 0xb6cdb499 in os_event_wait_time_low (event=0xb9dcfb68, time_in_usec=time_in_usec@entry=5000000, reset_sig_count=1) at /home/buildbot/buildbot/build/storage/innobase/os/os0event.cc:485
|
#5 0xb6d6911c in srv_monitor_thread () at /home/buildbot/buildbot/build/storage/innobase/srv/srv0srv.cc:1687
|
#6 0xb63aef72 in start_thread (arg=0xa9bf7b40) at pthread_create.c:312
|
#7 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 5 (Thread 0xad7fbb40 (LWP 25016)):
|
#0 0xb63f54f4 in ?? () from /lib/i386-linux-gnu/libaio.so.1
|
#1 0xb63f5548 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
|
#2 0xb6cd0282 in LinuxAIOHandler::collect (this=this@entry=0xad7fb1b0) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:1819
|
#3 0xb6cd648a in LinuxAIOHandler::poll (this=this@entry=0xad7fb1b0, m1=m1@entry=0xad7fb240, m2=m2@entry=0xad7fb244, request=request@entry=0xad7fb250) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:1980
|
#4 0xb6cda4ab in os_aio_linux_handler (request=0xad7fb250, m2=0xad7fb244, m1=0xad7fb240, global_segment=3) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:2034
|
#5 os_aio_handler (segment=segment@entry=3, m1=m1@entry=0xad7fb240, m2=m2@entry=0xad7fb244, request=request@entry=0xad7fb250) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:5607
|
#6 0xb6e65ec0 in fil_aio_wait (segment=segment@entry=3) at /home/buildbot/buildbot/build/storage/innobase/fil/fil0fil.cc:4315
|
#7 0xb6d6d120 in io_handler_thread (arg=0xb793d94c <n+12>) at /home/buildbot/buildbot/build/storage/innobase/srv/srv0start.cc:324
|
#8 0xb63aef72 in start_thread (arg=0xad7fbb40) at pthread_create.c:312
|
#9 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 4 (Thread 0xacffab40 (LWP 25017)):
|
#0 0xb63f54f4 in ?? () from /lib/i386-linux-gnu/libaio.so.1
|
#1 0xb63f5548 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
|
#2 0xb6cd0282 in LinuxAIOHandler::collect (this=this@entry=0xacffa1b0) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:1819
|
#3 0xb6cd648a in LinuxAIOHandler::poll (this=this@entry=0xacffa1b0, m1=m1@entry=0xacffa240, m2=m2@entry=0xacffa244, request=request@entry=0xacffa250) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:1980
|
#4 0xb6cda4ab in os_aio_linux_handler (request=0xacffa250, m2=0xacffa244, m1=0xacffa240, global_segment=4) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:2034
|
#5 os_aio_handler (segment=segment@entry=4, m1=m1@entry=0xacffa240, m2=m2@entry=0xacffa244, request=request@entry=0xacffa250) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:5607
|
#6 0xb6e65ec0 in fil_aio_wait (segment=segment@entry=4) at /home/buildbot/buildbot/build/storage/innobase/fil/fil0fil.cc:4315
|
#7 0xb6d6d120 in io_handler_thread (arg=0xb793d950 <n+16>) at /home/buildbot/buildbot/build/storage/innobase/srv/srv0start.cc:324
|
#8 0xb63aef72 in start_thread (arg=0xacffab40) at pthread_create.c:312
|
#9 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 3 (Thread 0xaeffeb40 (LWP 25013)):
|
#0 0xb63f54f4 in ?? () from /lib/i386-linux-gnu/libaio.so.1
|
#1 0xb63f5548 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
|
#2 0xb6cd0282 in LinuxAIOHandler::collect (this=this@entry=0xaeffe1b0) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:1819
|
#3 0xb6cd648a in LinuxAIOHandler::poll (this=this@entry=0xaeffe1b0, m1=m1@entry=0xaeffe240, m2=m2@entry=0xaeffe244, request=request@entry=0xaeffe250) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:1980
|
#4 0xb6cda4ab in os_aio_linux_handler (request=0xaeffe250, m2=0xaeffe244, m1=0xaeffe240, global_segment=0) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:2034
|
#5 os_aio_handler (segment=segment@entry=0, m1=m1@entry=0xaeffe240, m2=m2@entry=0xaeffe244, request=request@entry=0xaeffe250) at /home/buildbot/buildbot/build/storage/innobase/os/os0file.cc:5607
|
#6 0xb6e65ec0 in fil_aio_wait (segment=segment@entry=0) at /home/buildbot/buildbot/build/storage/innobase/fil/fil0fil.cc:4315
|
#7 0xb6d6d120 in io_handler_thread (arg=0xb793d940 <n>) at /home/buildbot/buildbot/build/storage/innobase/srv/srv0start.cc:324
|
#8 0xb63aef72 in start_thread (arg=0xaeffeb40) at pthread_create.c:312
|
#9 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 2 (Thread 0xb1dffb40 (LWP 25011)):
|
#0 0xb63fe424 in __kernel_vsyscall ()
|
#1 0xb63b30ed in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:245
|
#2 0xb7104bc2 in inline_mysql_cond_timedwait (that=0xb815d480 <COND_timer>, mutex=0xb815d4b4 <LOCK_timer>, src_file=0xb737edb0 "/home/buildbot/buildbot/build/mysys/thr_timer.c", src_line=292, abstime=0xb1dff2d4) at /home/buildbot/buildbot/build/include/mysql/psi/mysql_thread.h:1211
|
#3 timer_handler (arg=arg@entry=0x0) at /home/buildbot/buildbot/build/mysys/thr_timer.c:292
|
#4 0xb7012fcd in pfs_spawn_thread (arg=0xb9d254e8) at /home/buildbot/buildbot/build/storage/perfschema/pfs.cc:1869
|
#5 0xb63aef72 in start_thread (arg=0xb1dffb40) at pthread_create.c:312
|
#6 0xb61a743e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
|
|
Thread 1 (Thread 0xb609a940 (LWP 25010)):
|
#0 0xb63fe424 in __kernel_vsyscall ()
|
#1 0xb63b3fb3 in __pthread_kill (threadid=3054086464, signo=signo@entry=6) at ../nptl/sysdeps/unix/sysv/linux/pthread_kill.c:61
|
#2 0xb71001d7 in my_write_core (sig=6) at /home/buildbot/buildbot/build/mysys/stacktrace.c:386
|
#3 0xb6a93dc6 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/sql/signal_handler.cc:344
|
#4 <signal handler called>
|
#5 0xb63fe424 in __kernel_vsyscall ()
|
#6 0xb619f661 in select () at ../sysdeps/unix/syscall-template.S:81
|
#7 0xb70fda8a in my_sleep (m_seconds=1000) at /home/buildbot/buildbot/build/mysys/my_sleep.c:29
|
#8 0xb677dd0c in close_connections () at /home/buildbot/buildbot/build/sql/mysqld.cc:1775
|
#9 mysqld_main (argc=147, argv=0xbf9b0dd4) at /home/buildbot/buildbot/build/sql/mysqld.cc:5924
|
#10 0xb67431a7 in main (argc=23, argv=0xbf9b0dd4) at /home/buildbot/buildbot/build/sql/main.cc:25
|
The reason for failing to start up the server (as well as for the stack trace output) was that the server had been forcibly killed due to a shutdown timeout:
CURRENT_TEST: innodb.innodb-get-fk
|
2021-03-04 18:02:49 0 [Note] /usr/local/mariadb-10.4.19-linux-i686/bin/mysqld (initiated by: root[root] @ localhost []): Normal shutdown
|
2021-03-04 18:02:49 0 [Note] Event Scheduler: Purging the queue. 0 events
|
2021-03-04 18:02:49 0 [Note] InnoDB: FTS optimize thread exiting.
|
210304 18:03:49 [ERROR] mysqld got signal 6 ;
|
…
|
2021-03-04 18:03:52 0 [Warning] InnoDB: innodb_read_only prevents crash recovery
|
2021-03-04 18:03:52 0 [ERROR] InnoDB: Plugin initialization aborted with error Read only transaction
|
2021-03-04 18:03:52 0 [Note] InnoDB: Starting shutdown...
|
2021-03-04 18:03:52 0 [ERROR] Plugin 'InnoDB' init function returned error.
|
2021-03-04 18:03:52 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
|
Among the stack traces, we can observe that a single srv_worker_thread is left, and no purge coordinator. So, it looks like this issue again.
10.4 9024498e88f01f235f56db1dc18389baaef5b0b2 |
binlog_encryption.rpl_binlog_errors 'innodb,stmt' w44 [ fail ] Found warnings/errors in server log file!
|
Test ended at 2021-09-22 18:19:09
|
line
|
Attempting backtrace. You can use the following information to find out
|
In mysqld.2.err we have the following:
CURRENT_TEST: binlog_encryption.rpl_binlog_errors
|
…
|
2021-09-22 18:18:01 0 [Note] /dev/shm/10.4/sql/mysqld (initiated by: root[root] @ localhost [127.0.0.1]): Normal shutdown
|
2021-09-22 18:18:01 0 [Note] Event Scheduler: Purging the queue. 0 events
|
2021-09-22 18:18:01 0 [Note] InnoDB: FTS optimize thread exiting.
|
210922 18:19:02 [ERROR] mysqld got signal 6 ;
|
Note the 61-second delay between the 2 last messages. That is, shutdown_server in the test environment delivered the SIGABRT.
We have Thread 10 waiting for srv_shutdown_state <= SRV_SHUTDOWN_INITIATED (it is SRV_SHUTDOWN_INITIATED. The purge coordinator has exited, but a worker is left:
Thread 19 (Thread 0x7f3c137fe640 (LWP 1670932)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x5556c26e6398) at ../sysdeps/nptl/futex-internal.h:186
|
#1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x5556c26e6348, cond=0x5556c26e6370) at pthread_cond_wait.c:508
|
#2 __pthread_cond_wait (cond=0x5556c26e6370, mutex=0x5556c26e6348) at pthread_cond_wait.c:638
|
#3 0x00005556c0216c9b in os_event::wait (this=0x5556c26e6330) at /mariadb/10.4/storage/innobase/os/os0event.cc:158
|
#4 os_event::wait_low (this=0x5556c26e6330, reset_sig_count=1, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:325
|
#5 0x00005556c0217335 in os_event_wait_low (event=0x5556c26e6398, reset_sig_count=128, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:502
|
#6 0x00005556c03f8447 in buf_resize_thread () at /mariadb/10.4/storage/innobase/buf/buf0buf.cc:3142
|
#7 0x00007f3c61a8deae in start_thread (arg=0x7f3c137fe640) at pthread_create.c:463
|
#8 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 18 (Thread 0x7f3c28ff9640 (LWP 1670930)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x5556c26e6298) at ../sysdeps/nptl/futex-internal.h:186
|
#1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x5556c26e6248, cond=0x5556c26e6270) at pthread_cond_wait.c:508
|
#2 __pthread_cond_wait (cond=0x5556c26e6270, mutex=0x5556c26e6248) at pthread_cond_wait.c:638
|
#3 0x00005556c0216c9b in os_event::wait (this=0x5556c26e6230) at /mariadb/10.4/storage/innobase/os/os0event.cc:158
|
#4 os_event::wait_low (this=0x5556c26e6230, reset_sig_count=1, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:325
|
#5 0x00005556c0217335 in os_event_wait_low (event=0x5556c26e6298, reset_sig_count=128, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:502
|
#6 0x00005556c041a280 in buf_dump_thread () at /mariadb/10.4/storage/innobase/buf/buf0dump.cc:832
|
#7 0x00007f3c61a8deae in start_thread (arg=0x7f3c28ff9640) at pthread_create.c:463
|
#8 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 17 (Thread 0x7f3c5c053640 (LWP 1670940)):
|
#0 0x00007f3c616cfbd2 in __GI___sigtimedwait (set=0x7f3c5c052d08, info=0x7f3c5c052c88, timeout=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:36
|
#1 0x00005556bfc180eb in my_sigwait (set=0x7f3c5c052d08, sig=0x7f3c5c052c70, code=<optimized out>) at /mariadb/10.4/include/my_pthread.h:196
|
#2 signal_hand (arg=<optimized out>) at /mariadb/10.4/sql/mysqld.cc:3219
|
#3 0x00005556c062a5a6 in pfs_spawn_thread (arg=0x5556c29f2478) at /mariadb/10.4/storage/perfschema/pfs.cc:1869
|
#4 0x00007f3c61a8deae in start_thread (arg=0x7f3c5c053640) at pthread_create.c:463
|
#5 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 16 (Thread 0x7f3c44ff9640 (LWP 1670903)):
|
#0 futex_abstimed_wait_cancelable (private=0, abstime=0x7f3c44ff8d60, clockid=1157598000, expected=0, futex_word=0x5556c26e6218) at ../sysdeps/nptl/futex-internal.h:323
|
#1 __pthread_cond_wait_common (abstime=0x7f3c44ff8d60, clockid=1157598000, mutex=0x5556c26e61c8, cond=0x5556c26e61f0) at pthread_cond_wait.c:520
|
#2 __pthread_cond_timedwait (cond=0x5556c26e61f0, mutex=0x5556c26e61c8, abstime=0x7f3c44ff8d60) at pthread_cond_wait.c:656
|
#3 0x00005556c0216ad9 in os_event::timed_wait (this=this@entry=0x5556c26e61b0, abstime=abstime@entry=0x7f3c44ff8d60) at /mariadb/10.4/storage/innobase/os/os0event.cc:275
|
#4 0x00005556c0216e6b in os_event::wait_time_low (this=0x5556c26e61b0, time_in_usec=<optimized out>, time_in_usec@entry=5000000, reset_sig_count=1) at /mariadb/10.4/storage/innobase/os/os0event.cc:385
|
#5 0x00005556c0217325 in os_event_wait_time_low (event=0x5556c26e6218, time_in_usec=393, time_in_usec@entry=5000000, reset_sig_count=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:485
|
#6 0x00005556c0327710 in srv_monitor_thread () at /mariadb/10.4/storage/innobase/srv/srv0srv.cc:1684
|
#7 0x00007f3c61a8deae in start_thread (arg=0x7f3c44ff9640) at pthread_create.c:463
|
#8 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 15 (Thread 0x7f3c457fa640 (LWP 1670902)):
|
#0 futex_abstimed_wait_cancelable (private=0, abstime=0x7f3c457f9b80, clockid=1165990224, expected=0, futex_word=0x5556c26e6198) at ../sysdeps/nptl/futex-internal.h:323
|
#1 __pthread_cond_wait_common (abstime=0x7f3c457f9b80, clockid=1165990224, mutex=0x5556c26e6148, cond=0x5556c26e6170) at pthread_cond_wait.c:520
|
#2 __pthread_cond_timedwait (cond=0x5556c26e6170, mutex=0x5556c26e6148, abstime=0x7f3c457f9b80) at pthread_cond_wait.c:656
|
#3 0x00005556c0216ad9 in os_event::timed_wait (this=this@entry=0x5556c26e6130, abstime=abstime@entry=0x7f3c457f9b80) at /mariadb/10.4/storage/innobase/os/os0event.cc:275
|
#4 0x00005556c0216e6b in os_event::wait_time_low (this=0x5556c26e6130, time_in_usec=<optimized out>, time_in_usec@entry=1000000, reset_sig_count=1) at /mariadb/10.4/storage/innobase/os/os0event.cc:385
|
#5 0x00005556c0217325 in os_event_wait_time_low (event=0x5556c26e6198, time_in_usec=393, time_in_usec@entry=1000000, reset_sig_count=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:485
|
#6 0x00005556c0327af2 in srv_error_monitor_thread () at /mariadb/10.4/storage/innobase/srv/srv0srv.cc:1848
|
#7 0x00007f3c61a8deae in start_thread (arg=0x7f3c457fa640) at pthread_create.c:463
|
#8 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 14 (Thread 0x7f3c45ffb640 (LWP 1670901)):
|
#0 futex_abstimed_wait_cancelable (private=0, abstime=0x7f3c45fface0, clockid=1174383280, expected=0, futex_word=0x5556c291abe8) at ../sysdeps/nptl/futex-internal.h:323
|
#1 __pthread_cond_wait_common (abstime=0x7f3c45fface0, clockid=1174383280, mutex=0x5556c291ab98, cond=0x5556c291abc0) at pthread_cond_wait.c:520
|
#2 __pthread_cond_timedwait (cond=0x5556c291abc0, mutex=0x5556c291ab98, abstime=0x7f3c45fface0) at pthread_cond_wait.c:656
|
#3 0x00005556c0216ad9 in os_event::timed_wait (this=this@entry=0x5556c291ab80, abstime=abstime@entry=0x7f3c45fface0) at /mariadb/10.4/storage/innobase/os/os0event.cc:275
|
#4 0x00005556c0216e6b in os_event::wait_time_low (this=this@entry=0x5556c291ab80, time_in_usec=<optimized out>, time_in_usec@entry=1000000, reset_sig_count=reset_sig_count@entry=1) at /mariadb/10.4/storage/innobase/os/os0event.cc:385
|
#5 0x00005556c0217325 in os_event_wait_time_low (event=0x5556c291abe8, event@entry=0x5556c291ab80, time_in_usec=393, time_in_usec@entry=1000000, reset_sig_count=0, reset_sig_count@entry=1) at /mariadb/10.4/storage/innobase/os/os0event.cc:485
|
#6 0x00005556c01e3a49 in lock_wait_timeout_thread () at /mariadb/10.4/storage/innobase/lock/lock0wait.cc:527
|
#7 0x00007f3c61a8deae in start_thread (arg=0x7f3c45ffb640) at pthread_create.c:463
|
#8 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 13 (Thread 0x7f3c477fe640 (LWP 1670894)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x5556c276835c) at ../sysdeps/nptl/futex-internal.h:186
|
#1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x5556c2768308, cond=0x5556c2768330) at pthread_cond_wait.c:508
|
#2 __pthread_cond_wait (cond=0x5556c2768330, mutex=0x5556c2768308) at pthread_cond_wait.c:638
|
#3 0x00005556c0216c9b in os_event::wait (this=0x5556c27682f0) at /mariadb/10.4/storage/innobase/os/os0event.cc:158
|
#4 os_event::wait_low (this=this@entry=0x5556c27682f0, reset_sig_count=2, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:325
|
#5 0x00005556c0217335 in os_event_wait_low (event=0x5556c276835c, event@entry=0x5556c27682f0, reset_sig_count=128, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:502
|
#6 0x00005556c020f70a in os_aio_simulated_handler (global_segment=5, m1=<optimized out>, m2=<optimized out>, type=0x7f3c477fdd50) at /mariadb/10.4/storage/innobase/os/os0file.cc:6942
|
#7 os_aio_handler (segment=segment@entry=5, m1=m1@entry=0x7f3c477fdba0, m2=m2@entry=0x7f3c477fdbb0, request=request@entry=0x7f3c477fdd50) at /mariadb/10.4/storage/innobase/os/os0file.cc:5378
|
#8 0x00005556c049f956 in fil_aio_wait (segment=segment@entry=5) at /mariadb/10.4/storage/innobase/fil/fil0fil.cc:4315
|
#9 0x00005556c032b4d8 in io_handler_thread (arg=<optimized out>) at /mariadb/10.4/storage/innobase/srv/srv0start.cc:324
|
#10 0x00007f3c61a8deae in start_thread (arg=0x7f3c477fe640) at pthread_create.c:463
|
#11 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 12 (Thread 0x7f3c56659640 (LWP 1670941)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x5556c0ff2928 <COND_manager+40>) at ../sysdeps/nptl/futex-internal.h:186
|
#1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x5556c0ff2878 <LOCK_manager+40>, cond=0x5556c0ff2900 <COND_manager>) at pthread_cond_wait.c:508
|
#2 __pthread_cond_wait (cond=cond@entry=0x5556c0ff2900 <COND_manager>, mutex=mutex@entry=0x5556c0ff2878 <LOCK_manager+40>) at pthread_cond_wait.c:638
|
#3 0x00005556c068b3f7 in safe_cond_wait (cond=0x5556c0ff2928 <COND_manager+40>, mp=0x5556c0ff2850 <LOCK_manager>, file=0x5556c0705e8c "/mariadb/10.4/include/mysql/psi/mysql_thread.h", line=1174) at /mariadb/10.4/mysys/thr_mutex.c:492
|
#4 0x00005556bfcf7f28 in inline_mysql_cond_wait (that=0x5556c0ff2900 <COND_manager>, src_line=102, mutex=<optimized out>, src_file=<optimized out>) at /mariadb/10.4/include/mysql/psi/mysql_thread.h:1174
|
#5 handle_manager (arg=<optimized out>) at /mariadb/10.4/sql/sql_manager.cc:102
|
#6 0x00005556c062a5a6 in pfs_spawn_thread (arg=0x5556c29f1008) at /mariadb/10.4/storage/perfschema/pfs.cc:1869
|
#7 0x00007f3c61a8deae in start_thread (arg=0x7f3c56659640) at pthread_create.c:463
|
#8 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 11 (Thread 0x7f3c297fa640 (LWP 1670918)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x5556c26e611c) at ../sysdeps/nptl/futex-internal.h:186
|
#1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x5556c26e60c8, cond=0x5556c26e60f0) at pthread_cond_wait.c:508
|
#2 __pthread_cond_wait (cond=0x5556c26e60f0, mutex=0x5556c26e60c8) at pthread_cond_wait.c:638
|
#3 0x00005556c0216c9b in os_event::wait (this=0x5556c26e60b0) at /mariadb/10.4/storage/innobase/os/os0event.cc:158
|
#4 os_event::wait_low (this=0x5556c26e60b0, reset_sig_count=8, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:325
|
#5 0x00005556c0217335 in os_event_wait_low (event=0x5556c26e611c, reset_sig_count=128, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:502
|
#6 0x00005556c0329710 in srv_resume_thread (slot=slot@entry=0x5556c0f696b0 <srv_sys+1328>, sig_count=sig_count@entry=0, wait=true, timeout_usec=timeout_usec@entry=0) at /mariadb/10.4/storage/innobase/srv/srv0srv.cc:904
|
#7 0x00005556c0329ad5 in srv_worker_thread (arg=<optimized out>) at /mariadb/10.4/storage/innobase/srv/srv0srv.cc:2518
|
#8 0x00007f3c61a8deae in start_thread (arg=0x7f3c297fa640) at pthread_create.c:463
|
#9 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 10 (Thread 0x7f3c2b7fe640 (LWP 1670914)):
|
#0 0x00007f3c61757b21 in __GI___clock_nanosleep (clock_id=clock_id@entry=0, flags=flags@entry=0, req=0x7f3c2b7fdba8, rem=0x0) at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:46
|
#1 0x00007f3c6175cec3 in __GI___nanosleep (requested_time=<optimized out>, remaining=<optimized out>) at nanosleep.c:27
|
#2 0x00005556c02176b1 in os_thread_sleep (tm=tm@entry=1000000) at /mariadb/10.4/storage/innobase/os/os0thread.cc:231
|
#3 0x00005556c0328409 in srv_master_sleep () at /mariadb/10.4/storage/innobase/srv/srv0srv.cc:2335
|
#4 srv_master_thread (arg=<optimized out>) at /mariadb/10.4/storage/innobase/srv/srv0srv.cc:2375
|
#5 0x00007f3c61a8deae in start_thread (arg=0x7f3c2b7fe640) at pthread_create.c:463
|
#6 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 9 (Thread 0x7f3c55c58640 (LWP 1670890)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x5556c276815c) at ../sysdeps/nptl/futex-internal.h:186
|
#1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x5556c2768108, cond=0x5556c2768130) at pthread_cond_wait.c:508
|
#2 __pthread_cond_wait (cond=0x5556c2768130, mutex=0x5556c2768108) at pthread_cond_wait.c:638
|
#3 0x00005556c0216c9b in os_event::wait (this=0x5556c27680f0) at /mariadb/10.4/storage/innobase/os/os0event.cc:158
|
#4 os_event::wait_low (this=this@entry=0x5556c27680f0, reset_sig_count=2, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:325
|
#5 0x00005556c0217335 in os_event_wait_low (event=0x5556c276815c, event@entry=0x5556c27680f0, reset_sig_count=128, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:502
|
#6 0x00005556c020f70a in os_aio_simulated_handler (global_segment=1, m1=<optimized out>, m2=<optimized out>, type=0x7f3c55c57d50) at /mariadb/10.4/storage/innobase/os/os0file.cc:6942
|
#7 os_aio_handler (segment=segment@entry=1, m1=m1@entry=0x7f3c55c57ba0, m2=m2@entry=0x7f3c55c57bb0, request=request@entry=0x7f3c55c57d50) at /mariadb/10.4/storage/innobase/os/os0file.cc:5378
|
#8 0x00005556c049f956 in fil_aio_wait (segment=segment@entry=1) at /mariadb/10.4/storage/innobase/fil/fil0fil.cc:4315
|
#9 0x00005556c032b4d8 in io_handler_thread (arg=<optimized out>) at /mariadb/10.4/storage/innobase/srv/srv0start.cc:324
|
#10 0x00007f3c61a8deae in start_thread (arg=0x7f3c55c58640) at pthread_create.c:463
|
#11 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 8 (Thread 0x7f3c5785a640 (LWP 1669810)):
|
#0 futex_abstimed_wait_cancelable (private=0, abstime=0x7f3c57859cd0, clockid=1468374016, expected=0, futex_word=0x5556c1879898 <COND_checkpoint+40>) at ../sysdeps/nptl/futex-internal.h:323
|
#1 __pthread_cond_wait_common (abstime=0x7f3c57859cd0, clockid=1468374016, mutex=0x5556c18797e0 <LOCK_checkpoint+40>, cond=0x5556c1879870 <COND_checkpoint>) at pthread_cond_wait.c:520
|
#2 __pthread_cond_timedwait (cond=cond@entry=0x5556c1879870 <COND_checkpoint>, mutex=mutex@entry=0x5556c18797e0 <LOCK_checkpoint+40>, abstime=abstime@entry=0x7f3c57859cd0) at pthread_cond_wait.c:656
|
#3 0x00005556c068b62f in safe_cond_timedwait (cond=cond@entry=0x5556c1879870 <COND_checkpoint>, mp=mp@entry=0x5556c18797b8 <LOCK_checkpoint>, abstime=abstime@entry=0x7f3c57859cd0, file=0x5556c0705e8c "/mariadb/10.4/include/mysql/psi/mysql_thread.h", line=line@entry=1211) at /mariadb/10.4/mysys/thr_mutex.c:546
|
#4 0x00005556c0564a8f in inline_mysql_cond_timedwait (that=0x5556c1879870 <COND_checkpoint>, mutex=0x5556c18797b8 <LOCK_checkpoint>, abstime=0x7f3c57859cd0, src_line=116, src_file=<optimized out>) at /mariadb/10.4/include/mysql/psi/mysql_thread.h:1211
|
#5 my_service_thread_sleep (control=control@entry=0x5556c0f6e590 <checkpoint_control>, sleep_time=<optimized out>) at /mariadb/10.4/storage/maria/ma_servicethread.c:115
|
#6 0x00005556c055c671 in ma_checkpoint_background (arg=<optimized out>, arg@entry=0x1e) at /mariadb/10.4/storage/maria/ma_checkpoint.c:707
|
#7 0x00005556c062a5a6 in pfs_spawn_thread (arg=0x5556c269df68) at /mariadb/10.4/storage/perfschema/pfs.cc:1869
|
#8 0x00007f3c61a8deae in start_thread (arg=0x7f3c5785a640) at pthread_create.c:463
|
#9 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 7 (Thread 0x7f3c54c56640 (LWP 1670892)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x5556c2768258) at ../sysdeps/nptl/futex-internal.h:186
|
#1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x5556c2768208, cond=0x5556c2768230) at pthread_cond_wait.c:508
|
#2 __pthread_cond_wait (cond=0x5556c2768230, mutex=0x5556c2768208) at pthread_cond_wait.c:638
|
#3 0x00005556c0216c9b in os_event::wait (this=0x5556c27681f0) at /mariadb/10.4/storage/innobase/os/os0event.cc:158
|
#4 os_event::wait_low (this=this@entry=0x5556c27681f0, reset_sig_count=1, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:325
|
#5 0x00005556c0217335 in os_event_wait_low (event=0x5556c2768258, event@entry=0x5556c27681f0, reset_sig_count=128, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:502
|
#6 0x00005556c020f70a in os_aio_simulated_handler (global_segment=3, m1=<optimized out>, m2=<optimized out>, type=0x7f3c54c55d50) at /mariadb/10.4/storage/innobase/os/os0file.cc:6942
|
#7 os_aio_handler (segment=segment@entry=3, m1=m1@entry=0x7f3c54c55ba0, m2=m2@entry=0x7f3c54c55bb0, request=request@entry=0x7f3c54c55d50) at /mariadb/10.4/storage/innobase/os/os0file.cc:5378
|
#8 0x00005556c049f956 in fil_aio_wait (segment=segment@entry=3) at /mariadb/10.4/storage/innobase/fil/fil0fil.cc:4315
|
#9 0x00005556c032b4d8 in io_handler_thread (arg=<optimized out>) at /mariadb/10.4/storage/innobase/srv/srv0start.cc:324
|
#10 0x00007f3c61a8deae in start_thread (arg=0x7f3c54c56640) at pthread_create.c:463
|
#11 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 6 (Thread 0x7f3c55457640 (LWP 1670891)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x5556c27681d8) at ../sysdeps/nptl/futex-internal.h:186
|
#1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x5556c2768188, cond=0x5556c27681b0) at pthread_cond_wait.c:508
|
#2 __pthread_cond_wait (cond=0x5556c27681b0, mutex=0x5556c2768188) at pthread_cond_wait.c:638
|
#3 0x00005556c0216c9b in os_event::wait (this=0x5556c2768170) at /mariadb/10.4/storage/innobase/os/os0event.cc:158
|
#4 os_event::wait_low (this=this@entry=0x5556c2768170, reset_sig_count=1, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:325
|
#5 0x00005556c0217335 in os_event_wait_low (event=0x5556c27681d8, event@entry=0x5556c2768170, reset_sig_count=128, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:502
|
#6 0x00005556c020f70a in os_aio_simulated_handler (global_segment=2, m1=<optimized out>, m2=<optimized out>, type=0x7f3c55456d50) at /mariadb/10.4/storage/innobase/os/os0file.cc:6942
|
#7 os_aio_handler (segment=segment@entry=2, m1=m1@entry=0x7f3c55456ba0, m2=m2@entry=0x7f3c55456bb0, request=request@entry=0x7f3c55456d50) at /mariadb/10.4/storage/innobase/os/os0file.cc:5378
|
#8 0x00005556c049f956 in fil_aio_wait (segment=segment@entry=2) at /mariadb/10.4/storage/innobase/fil/fil0fil.cc:4315
|
#9 0x00005556c032b4d8 in io_handler_thread (arg=<optimized out>) at /mariadb/10.4/storage/innobase/srv/srv0start.cc:324
|
#10 0x00007f3c61a8deae in start_thread (arg=0x7f3c55457640) at pthread_create.c:463
|
#11 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 5 (Thread 0x7f3c56459640 (LWP 1670889)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x5556c27680d8) at ../sysdeps/nptl/futex-internal.h:186
|
#1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x5556c2768088, cond=0x5556c27680b0) at pthread_cond_wait.c:508
|
#2 __pthread_cond_wait (cond=0x5556c27680b0, mutex=0x5556c2768088) at pthread_cond_wait.c:638
|
#3 0x00005556c0216c9b in os_event::wait (this=0x5556c2768070) at /mariadb/10.4/storage/innobase/os/os0event.cc:158
|
#4 os_event::wait_low (this=this@entry=0x5556c2768070, reset_sig_count=1, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:325
|
#5 0x00005556c0217335 in os_event_wait_low (event=0x5556c27680d8, event@entry=0x5556c2768070, reset_sig_count=128, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:502
|
#6 0x00005556c020f70a in os_aio_simulated_handler (global_segment=0, m1=<optimized out>, m2=<optimized out>, type=0x7f3c56458d50) at /mariadb/10.4/storage/innobase/os/os0file.cc:6942
|
#7 os_aio_handler (segment=segment@entry=0, m1=m1@entry=0x7f3c56458ba0, m2=m2@entry=0x7f3c56458bb0, request=request@entry=0x7f3c56458d50) at /mariadb/10.4/storage/innobase/os/os0file.cc:5378
|
#8 0x00005556c049f956 in fil_aio_wait (segment=segment@entry=0) at /mariadb/10.4/storage/innobase/fil/fil0fil.cc:4315
|
#9 0x00005556c032b4d8 in io_handler_thread (arg=<optimized out>) at /mariadb/10.4/storage/innobase/srv/srv0start.cc:324
|
#10 0x00007f3c61a8deae in start_thread (arg=0x7f3c56459640) at pthread_create.c:463
|
#11 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 4 (Thread 0x7f3c5d078640 (LWP 1669804)):
|
#0 futex_abstimed_wait_cancelable (private=0, abstime=0x7f3c5d077d40, clockid=1560771696, expected=0, futex_word=0x5556c188c8b8 <COND_timer+40>) at ../sysdeps/nptl/futex-internal.h:323
|
#1 __pthread_cond_wait_common (abstime=0x7f3c5d077d40, clockid=1560771696, mutex=0x5556c188c808 <LOCK_timer+40>, cond=0x5556c188c890 <COND_timer>) at pthread_cond_wait.c:520
|
#2 __pthread_cond_timedwait (cond=cond@entry=0x5556c188c890 <COND_timer>, mutex=mutex@entry=0x5556c188c808 <LOCK_timer+40>, abstime=abstime@entry=0x7f3c5d077d40) at pthread_cond_wait.c:656
|
#3 0x00005556c068b62f in safe_cond_timedwait (cond=0x5556c188c890 <COND_timer>, mp=0x5556c188c7e0 <LOCK_timer>, abstime=abstime@entry=0x7f3c5d077d40, file=0x5556c0705e8c "/mariadb/10.4/include/mysql/psi/mysql_thread.h", line=line@entry=1211) at /mariadb/10.4/mysys/thr_mutex.c:546
|
#4 0x00005556c068bfca in inline_mysql_cond_timedwait (abstime=0x7f3c5d077d40, src_line=292, that=<optimized out>, mutex=<optimized out>, src_file=<optimized out>) at /mariadb/10.4/include/mysql/psi/mysql_thread.h:1211
|
#5 timer_handler (arg=arg@entry=0x0) at /mariadb/10.4/mysys/thr_timer.c:292
|
#6 0x00005556c062a5a6 in pfs_spawn_thread (arg=0x5556c25e07b8) at /mariadb/10.4/storage/perfschema/pfs.cc:1869
|
#7 0x00007f3c61a8deae in start_thread (arg=0x7f3c5d078640) at pthread_create.c:463
|
#8 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 3 (Thread 0x7f3c46ffd640 (LWP 1670895)):
|
#0 futex_abstimed_wait_cancelable (private=0, abstime=0x7f3c46ffcb20, clockid=1191168240, expected=0, futex_word=0x5556c26e631c) at ../sysdeps/nptl/futex-internal.h:323
|
#1 __pthread_cond_wait_common (abstime=0x7f3c46ffcb20, clockid=1191168240, mutex=0x5556c26e62c8, cond=0x5556c26e62f0) at pthread_cond_wait.c:520
|
#2 __pthread_cond_timedwait (cond=0x5556c26e62f0, mutex=0x5556c26e62c8, abstime=0x7f3c46ffcb20) at pthread_cond_wait.c:656
|
#3 0x00005556c0216ad9 in os_event::timed_wait (this=this@entry=0x5556c26e62b0, abstime=abstime@entry=0x7f3c46ffcb20) at /mariadb/10.4/storage/innobase/os/os0event.cc:275
|
#4 0x00005556c0216e6b in os_event::wait_time_low (this=0x5556c26e62b0, time_in_usec=<optimized out>, reset_sig_count=reset_sig_count@entry=63) at /mariadb/10.4/storage/innobase/os/os0event.cc:385
|
#5 0x00005556c0217325 in os_event_wait_time_low (event=0x5556c26e631c, time_in_usec=393, reset_sig_count=0, reset_sig_count@entry=63) at /mariadb/10.4/storage/innobase/os/os0event.cc:485
|
#6 0x00005556c0424b21 in pc_sleep_if_needed (next_loop_time=40622825, sig_count=63, cur_time=40621825) at /mariadb/10.4/storage/innobase/buf/buf0flu.cc:2637
|
#7 buf_flush_page_cleaner_coordinator () at /mariadb/10.4/storage/innobase/buf/buf0flu.cc:3096
|
#8 0x00007f3c61a8deae in start_thread (arg=0x7f3c46ffd640) at pthread_create.c:463
|
#9 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 2 (Thread 0x7f3c47fff640 (LWP 1670893)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x5556c27682dc) at ../sysdeps/nptl/futex-internal.h:186
|
#1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x5556c2768288, cond=0x5556c27682b0) at pthread_cond_wait.c:508
|
#2 __pthread_cond_wait (cond=0x5556c27682b0, mutex=0x5556c2768288) at pthread_cond_wait.c:638
|
#3 0x00005556c0216c9b in os_event::wait (this=0x5556c2768270) at /mariadb/10.4/storage/innobase/os/os0event.cc:158
|
#4 os_event::wait_low (this=this@entry=0x5556c2768270, reset_sig_count=2, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:325
|
#5 0x00005556c0217335 in os_event_wait_low (event=0x5556c27682dc, event@entry=0x5556c2768270, reset_sig_count=128, reset_sig_count@entry=0) at /mariadb/10.4/storage/innobase/os/os0event.cc:502
|
#6 0x00005556c020f70a in os_aio_simulated_handler (global_segment=4, m1=<optimized out>, m2=<optimized out>, type=0x7f3c47ffed50) at /mariadb/10.4/storage/innobase/os/os0file.cc:6942
|
#7 os_aio_handler (segment=segment@entry=4, m1=m1@entry=0x7f3c47ffeba0, m2=m2@entry=0x7f3c47ffebb0, request=request@entry=0x7f3c47ffed50) at /mariadb/10.4/storage/innobase/os/os0file.cc:5378
|
#8 0x00005556c049f956 in fil_aio_wait (segment=segment@entry=4) at /mariadb/10.4/storage/innobase/fil/fil0fil.cc:4315
|
#9 0x00005556c032b4d8 in io_handler_thread (arg=<optimized out>) at /mariadb/10.4/storage/innobase/srv/srv0start.cc:324
|
#10 0x00007f3c61a8deae in start_thread (arg=0x7f3c47fff640) at pthread_create.c:463
|
#11 0x00007f3c6178fa5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
Thread 1 (Thread 0x7f3c614427c0 (LWP 1669790)):
|
#0 __pthread_kill (threadid=<optimized out>, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:54
|
#1 0x00005556bff58bf7 in handle_fatal_signal (sig=6) at /mariadb/10.4/sql/signal_handler.cc:356
|
#2 <signal handler called>
|
#3 0x00007f3c61787543 in __GI___select (nfds=nfds@entry=0, readfds=readfds@entry=0x0, writefds=writefds@entry=0x0, exceptfds=exceptfds@entry=0x0, timeout=timeout@entry=0x7ffd0aa889a8) at ../sysdeps/unix/sysv/linux/select.c:41
|
#4 0x00005556c0683920 in my_sleep (m_seconds=<optimized out>) at /mariadb/10.4/mysys/my_sleep.c:29
|
#5 0x00005556bfc1ef7a in close_connections () at /mariadb/10.4/sql/mysqld.cc:1769
|
#6 0x00005556bfc19636 in mysqld_main (argc=<optimized out>, argv=<optimized out>) at /mariadb/10.4/sql/mysqld.cc:5922
|
#7 0x00007f3c616b9e4a in __libc_start_main (main=0x5556bfc151f0 <main(int, char**)>, argc=25, argv=0x7ffd0aa88c28, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffd0aa88c18) at ../csu/libc-start.c:314
|
#8 0x00005556bfc1512a in _start () at /mariadb/10.4/sql/my_decimal.h:143
|
I just reproduced this on 10.3 for the test innodb.doublewrite,32k. There was no srv_purge_coordinator_thread, but one srv_worker_thread was waiting:
#4 ___pthread_cond_wait (cond=0x55980841ea50, mutex=0x55980841ea28) at ./nptl/pthread_cond_wait.c:618
|
#5 0x0000559806288eef in os_event::wait (this=this@entry=0x55980841ea10) at /mariadb/10.3-MDEV-25004/storage/innobase/os/os0event.cc:158
|
#6 0x0000559806288b45 in os_event::wait_low (this=0x55980841ea10, reset_sig_count=169, reset_sig_count@entry=0) at /mariadb/10.3-MDEV-25004/storage/innobase/os/os0event.cc:325
|
#7 0x0000559806288c13 in os_event_wait_low (event=<optimized out>, reset_sig_count=reset_sig_count@entry=0) at /mariadb/10.3-MDEV-25004/storage/innobase/os/os0event.cc:502
|
#8 0x000055980635baca in srv_resume_thread (slot=slot@entry=0x55980669e870 <srv_sys+1264>, sig_count=sig_count@entry=0, wait=wait@entry=true, timeout_usec=timeout_usec@entry=0) at /mariadb/10.3-MDEV-25004/storage/innobase/srv/srv0srv.cc:909
|
#9 0x000055980635c30f in srv_worker_thread (arg=<optimized out>) at /mariadb/10.3-MDEV-25004/storage/innobase/srv/srv0srv.cc:2505
|
#10 0x00007fb1393c8fd4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
|
#11 0x00007fb13944966c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
|
A number of io_handler_thread and many other InnoDB threads were idle. srv_shutdown_state was SRV_SHUTDOWN_INITIATED.
I suspect that this may have been fixed by MDEV-16264 in 10.5 already (closed on 2019-11-15, 2 months after I had set the 10.5 version on this ticket).
There was one more occurrence on a CI system:
10.4 6c842f4ef71584e4b66698ba5fe354e90ef3a930 |
rpl.rpl_upgrade_master_info 'row' w7 [ fail ] Found warnings/errors in server log file!
|
Test ended at 2024-02-16 15:30:39
|
line
|
Attempting backtrace. You can use the following information to find out
|
^ Found warnings in /home/buildbot/amd64-ubuntu-2204-debug-ps/build/mysql-test/var/7/log/mysqld.2.err
|
Among the stack traces, we see one srv_worker_thread but no srv_purge_coordinator_thread:
10.4 6c842f4ef71584e4b66698ba5fe354e90ef3a930 |
Thread 14 (Thread 0x7f2c997fa640 (LWP 126856)):
|
#0 0x00007f2cbd5d0117 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
|
No symbol table info available.
|
#1 0x00007f2cbd5d2a41 in pthread_cond_wait () from /lib/x86_64-linux-gnu/libc.so.6
|
No symbol table info available.
|
#2 0x00005651bb149ee4 in os_event::wait (this=0x5651bd0b98b0) at /home/buildbot/amd64-ubuntu-2204-debug-ps/build/storage/innobase/os/os0event.cc:158
|
ret = 0
|
#3 0x00005651bb1499af in os_event::wait_low (this=0x5651bd0b98b0, reset_sig_count=6) at /home/buildbot/amd64-ubuntu-2204-debug-ps/build/storage/innobase/os/os0event.cc:325
|
No locals.
|
#4 0x00005651bb149ca7 in os_event_wait_low (event=0x5651bd0b98b0, reset_sig_count=0) at /home/buildbot/amd64-ubuntu-2204-debug-ps/build/storage/innobase/os/os0event.cc:502
|
No locals.
|
#5 0x00005651bb276ba3 in srv_resume_thread (slot=0x5651bc06d670 <srv_sys+1328>, sig_count=0, wait=true, timeout_usec=0) at /home/buildbot/amd64-ubuntu-2204-debug-ps/build/storage/innobase/srv/srv0srv.cc:897
|
timeout = false
|
__PRETTY_FUNCTION__ = "bool srv_resume_thread(srv_slot_t*, int64_t, bool, ulint)"
|
#6 0x00005651bb27b246 in srv_worker_thread (arg=0x0) at /home/buildbot/amd64-ubuntu-2204-debug-ps/build/storage/innobase/srv/srv0srv.cc:2484
|
slot = 0x5651bc06d670 <srv_sys+1328>
|
__PRETTY_FUNCTION__ = "void* srv_worker_thread(void*)"
|
thd = 0x7f2c7c000da0
|
#7 0x00007f2cbd5d3ac3 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
|
No symbol table info available.
|
#8 0x00007f2cbd664a04 in clone () from /lib/x86_64-linux-gnu/libc.so.6
|
No symbol table info available.
|
I don’t think it is feasible to fix this; 10.4 is so close to its end of life, and this bug always was very hard to reproduce.
I just got this locally on 10.4:
10.4 55e48b77226fec7619986fc1206930bcf63439fe
rpl.rpl_heartbeat_basic 'mix' w46 [ fail ] Found warnings/errors in server log file!
Test ended at 2020-09-22 15:00:06
line
Attempting backtrace. You can use the following information to find out
Only one srv_worker_thread exists:
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x55a37df97fd8) at ../sysdeps/nptl/futex-internal.h:183
#1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x55a37df97f88, cond=0x55a37df97fb0) at pthread_cond_wait.c:508
#2 __pthread_cond_wait (cond=0x55a37df97fb0, mutex=0x55a37df97f88) at pthread_cond_wait.c:638
#3 0x000055a37bc0ef39 in os_event::wait (this=0x55a37df97f70) at /mariadb/10.4/storage/innobase/os/os0event.cc:158
#4 os_event::wait_low (this=0x55a37df97f70, reset_sig_count=7) at /mariadb/10.4/storage/innobase/os/os0event.cc:325
#5 0x000055a37bc0f595 in os_event_wait_low (event=0x55a37df97fd8, reset_sig_count=128) at /mariadb/10.4/storage/innobase/os/os0event.cc:502
#6 0x000055a37bd26450 in srv_resume_thread (slot=0x55a37c966fb0 <srv_sys+560>, sig_count=0, wait=true, timeout_usec=0) at /mariadb/10.4/storage/innobase/srv/srv0srv.cc:910
#7 0x000055a37bd26756 in srv_worker_thread (arg=<optimized out>) at /mariadb/10.4/storage/innobase/srv/srv0srv.cc:2517
#8 0x00007fa8909f0ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477
#9 0x00007fa89004ceaf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
The purge coordinator thread is nowhere to be seen. srv_shutdown_state has not advanced from SRV_SHUTDOWN_INITIATED.