[MDEV-22098] connection attempts sometimes hang on waiting FLUSH TABLES Created: 2020-03-31  Updated: 2020-08-25  Resolved: 2020-05-01

Status: Closed
Project: MariaDB Server
Component/s: OTHER
Affects Version/s: 10.2.29, 10.1.44, 10.3.18, 10.3.22
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Hartmut Holzgraefe Assignee: Unassigned
Resolution: Incomplete Votes: 0
Labels: need_feedback


 Description   

When FLUSH TABLES has to wait for another query to finish, I see the weird effect that two out of three client connect attempts hang until FLUSH has finished, while one out of three attempts succeed.

How to reproduce:

  • first session

USE test;
 
CREATE TABLE t1(id int primary key);
 
INSERT INTO t1 VALUES(1);
 
SELECT id, SLEEP(3600) FROM t1;

  • second session

FLUSH TABLES;

  • third terminal just trying to connect, as root, with

mysql -A

The weird effect is that two attempts in a row hang and need to be terminated with CTRL-C, then one attempt succeeds, then the next two hang again, then one succeeds, and so on ...

 Comments   
Comment by Elena Stepanova [ 2020-04-01 ]

On some reason I can't reproduce it. The description is clear enough, I don't think I could be doing it wrongly, so maybe it somehow depends on the environment and/or build.

hholzgra,
Could you get stack traces from the running (hanging) client and the server while the client is hanging? Maybe it will give us some idea.
Also, could you please specify which system and build you are using and paste the config, if it has something non-default?

Comment by Hartmut Holzgraefe [ 2020-04-01 ]

Client side:

root@ubuntu-bionic:~# gdb --batch --eval-command="thread apply all bt" /usr/bin/mysql 4959
 
[Thread debugging using libthread_db enabled]
 
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
 
0x00007f163e494bc4 in __GI___poll (fds=0x7ffde36102c8, nfds=1, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
 
29	../sysdeps/unix/sysv/linux/poll.c: No such file or directory.
 
 
 
Thread 1 (Thread 0x7f1640136b80 (LWP 4959)):
 
#0  0x00007f163e494bc4 in __GI___poll (fds=0x7ffde36102c8, nfds=1, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
 
#1  0x000055e8217c05ca in pvio_socket_wait_io_or_timeout ()
 
#2  0x000055e8217c094b in pvio_socket_read ()
 
#3  0x000055e8217ab2bf in ma_pvio_read ()
 
#4  0x000055e8217ab4e6 in ma_pvio_cache_read ()
 
#5  0x000055e8217c3956 in ?? ()
 
#6  0x000055e8217c479c in ma_net_read ()
 
#7  0x000055e8217a6a32 in ma_net_safe_read ()
 
#8  0x000055e8217a7f8a in mthd_my_real_connect ()
 
#9  0x000055e8217a4916 in mysql_real_connect ()
 
#10 0x000055e8217948a4 in ?? ()
 
#11 0x000055e821798b33 in ?? ()
 
#12 0x000055e821793a78 in main ()

Comment by Hartmut Holzgraefe [ 2020-04-01 ]

server side:

root@ubuntu-bionic:~# gdb --batch --eval-command="thread apply all bt" /usr/sbin/mysqld $(pgrep -xn mysqld)
 
[New LWP 4532]
 
[New LWP 4533]
 
[New LWP 4535]
 
[New LWP 4536]
 
[New LWP 4537]
 
[New LWP 4538]
 
[New LWP 4539]
 
[New LWP 4540]
 
[New LWP 4541]
 
[New LWP 4542]
 
[New LWP 4543]
 
[New LWP 4544]
 
[New LWP 4545]
 
[New LWP 4547]
 
[New LWP 4548]
 
[New LWP 4549]
 
[New LWP 4551]
 
[New LWP 4552]
 
[New LWP 4553]
 
[New LWP 4554]
 
[New LWP 4555]
 
[New LWP 4556]
 
[New LWP 4557]
 
[New LWP 4558]
 
[New LWP 4559]
 
[New LWP 4560]
 
[New LWP 4561]
 
[New LWP 4562]
 
[New LWP 4563]
 
[New LWP 4564]
 
[New LWP 4565]
 
[New LWP 4566]
 
[New LWP 4567]
 
[New LWP 4568]
 
[New LWP 4583]
 
[New LWP 4592]
 
[New LWP 4593]
 
[New LWP 4929]
 
[New LWP 4951]
 
[Thread debugging using libthread_db enabled]
 
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
 
0x00007f576c28bbf9 in __GI___poll (fds=0x7ffff244e270, nfds=3, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
 
29	../sysdeps/unix/sysv/linux/poll.c: No such file or directory.
 
 
 
Thread 40 (Thread 0x7f57509dc700 (LWP 4951)):
 
#0  0x00007f576c298bb7 in epoll_wait (epfd=23, events=0x7f57509d8d60, maxevents=1024, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
 
#1  0x000055ba0eb9e91b in io_poll_wait(int, epoll_event*, int, int) ()
 
#2  0x000055ba0eb9f344 in get_event(worker_thread_t*, thread_group_t*, timespec*) ()
 
#3  0x000055ba0eb9f700 in ?? ()
 
#4  0x00007f576cc966db in start_thread (arg=0x7f57509dc700) at pthread_create.c:463
 
#5  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 39 (Thread 0x7f55683a1700 (LWP 4929)):
 
#0  0x00007f576c298bb7 in epoll_wait (epfd=22, events=0x7f556839dd60, maxevents=1024, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
 
#1  0x000055ba0eb9e91b in io_poll_wait(int, epoll_event*, int, int) ()
 
#2  0x000055ba0eb9f344 in get_event(worker_thread_t*, thread_group_t*, timespec*) ()
 
#3  0x000055ba0eb9f700 in ?? ()
 
#4  0x00007f576cc966db in start_thread (arg=0x7f55683a1700) at pthread_create.c:463
 
#5  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 38 (Thread 0x7f5750071700 (LWP 4593)):
 
#0  0x00007f576c298bb7 in epoll_wait (epfd=24, events=0x7f575006dd60, maxevents=1024, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
 
#1  0x000055ba0eb9e91b in io_poll_wait(int, epoll_event*, int, int) ()
 
#2  0x000055ba0eb9f344 in get_event(worker_thread_t*, thread_group_t*, timespec*) ()
 
#3  0x000055ba0eb9f700 in ?? ()
 
#4  0x00007f576cc966db in start_thread (arg=0x7f5750071700) at pthread_create.c:463
 
#5  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 37 (Thread 0x7f57508fb700 (LWP 4592)):
 
#0  0x00007f576cc9cf85 in futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7f57508f8c10, expected=0, futex_word=0x7f5504000e60) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
 
#1  __pthread_cond_wait_common (abstime=0x7f57508f8c10, mutex=0x7f5504000e08, cond=0x7f5504000e38) at pthread_cond_wait.c:539
 
#2  __pthread_cond_timedwait (cond=0x7f5504000e38, mutex=0x7f5504000e08, abstime=0x7f57508f8c10) at pthread_cond_wait.c:667
 
#3  0x000055ba0eace182 in MDL_wait::timed_wait(MDL_context_owner*, timespec*, bool, PSI_stage_info_v1 const*) ()
 
#4  0x000055ba0ea9a2e4 in TABLE_SHARE::wait_for_old_version(THD*, timespec*, unsigned int) ()
 
#5  0x000055ba0e98f253 in close_cached_tables(THD*, TABLE_LIST*, bool, unsigned long) ()
 
#6  0x000055ba0eae1de9 in reload_acl_and_cache(THD*, unsigned long long, TABLE_LIST*, int*) ()
 
#7  0x000055ba0e9ea41b in mysql_execute_command(THD*) ()
 
#8  0x000055ba0e9f154a in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) ()
 
#9  0x000055ba0e9f3ec7 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) ()
 
#10 0x000055ba0e9f4c88 in do_command(THD*) ()
 
#11 0x000055ba0eb05df0 in tp_callback(TP_connection*) ()
 
#12 0x000055ba0eb9f6b0 in ?? ()
 
#13 0x00007f576cc966db in start_thread (arg=0x7f57508fb700) at pthread_create.c:463
 
#14 0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 36 (Thread 0x7f5750991700 (LWP 4583)):
 
#0  0x00007f576cc9cf85 in futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7f575098e0f0, expected=0, futex_word=0x7f575098e1d8) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
 
#1  __pthread_cond_wait_common (abstime=0x7f575098e0f0, mutex=0x55ba0faae2c0 <LOCK_item_func_sleep>, cond=0x7f575098e1b0) at pthread_cond_wait.c:539
 
#2  __pthread_cond_timedwait (cond=0x7f575098e1b0, mutex=0x55ba0faae2c0 <LOCK_item_func_sleep>, abstime=0x7f575098e0f0) at pthread_cond_wait.c:667
 
#3  0x000055ba0ec302de in Interruptible_wait::wait(st_mysql_cond*, st_mysql_mutex*) ()
 
#4  0x000055ba0ec304d2 in Item_func_sleep::val_int() ()
 
#5  0x000055ba0eb24b19 in Type_handler::Item_send_long(Item*, Protocol*, st_value*) const ()
 
#6  0x000055ba0e942de8 in Protocol::send_result_set_row(List<Item>*) ()
 
#7  0x000055ba0e9a7983 in select_send::send_data(List<Item>&) ()
 
#8  0x000055ba0ea31c4f in ?? ()
 
#9  0x000055ba0ea17088 in ?? ()
 
#10 0x000055ba0ea22600 in sub_select(JOIN*, st_join_table*, bool) ()
 
#11 0x000055ba0ea425c6 in JOIN::exec_inner() ()
 
#12 0x000055ba0ea42913 in JOIN::exec() ()
 
#13 0x000055ba0ea40f8b in mysql_select(THD*, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) ()
 
#14 0x000055ba0ea418fd in handle_select(THD*, LEX*, select_result*, unsigned long) ()
 
#15 0x000055ba0e9e24f1 in ?? ()
 
#16 0x000055ba0e9ef210 in mysql_execute_command(THD*) ()
 
#17 0x000055ba0e9f154a in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) ()
 
#18 0x000055ba0e9f3ec7 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) ()
 
#19 0x000055ba0e9f4c88 in do_command(THD*) ()
 
#20 0x000055ba0eb05df0 in tp_callback(TP_connection*) ()
 
#21 0x000055ba0eb9f6b0 in ?? ()
 
#22 0x00007f576cc966db in start_thread (arg=0x7f5750991700) at pthread_create.c:463
 
#23 0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 35 (Thread 0x7f5750a27700 (LWP 4568)):
 
#0  0x00007f576cc9c9f3 in futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55ba0fa85568 <COND_slave_background+40>) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
 
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x55ba0fa82a00 <LOCK_slave_background>, cond=0x55ba0fa85540 <COND_slave_background>) at pthread_cond_wait.c:502
 
#2  __pthread_cond_wait (cond=0x55ba0fa85540 <COND_slave_background>, mutex=0x55ba0fa82a00 <LOCK_slave_background>) at pthread_cond_wait.c:655
 
#3  0x000055ba0e94fb67 in handle_slave_background ()
 
#4  0x00007f576cc966db in start_thread (arg=0x7f5750a27700) at pthread_create.c:463
 
#5  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 34 (Thread 0x7f5750a72700 (LWP 4567)):
 
#0  0x00007f576c1b726c in __GI___sigtimedwait (set=<optimized out>, set@entry=0x7f5750a71da0, info=info@entry=0x7f5750a71c70, timeout=timeout@entry=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:42
 
#1  0x00007f576cca145c in __sigwait (set=0x7f5750a71da0, sig=0x7f5750a71d48) at ../sysdeps/unix/sysv/linux/sigwait.c:28
 
#2  0x000055ba0e9301cb in signal_hand ()
 
#3  0x00007f576cc966db in start_thread (arg=0x7f5750a72700) at pthread_create.c:463
 
#4  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 33 (Thread 0x7f555ccbb700 (LWP 4566)):
 
#0  0x00007f576cc9cf85 in futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7f555ccbadf0, expected=0, futex_word=0x55ba0faac438) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
 
#1  __pthread_cond_wait_common (abstime=0x7f555ccbadf0, mutex=0x55ba0faac3e0, cond=0x55ba0faac410) at pthread_cond_wait.c:539
 
#2  __pthread_cond_timedwait (cond=0x55ba0faac410, mutex=0x55ba0faac3e0, abstime=0x7f555ccbadf0) at pthread_cond_wait.c:667
 
#3  0x000055ba0eb9e1fe in ?? ()
 
#4  0x00007f576cc966db in start_thread (arg=0x7f555ccbb700) at pthread_create.c:463
 
#5  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 32 (Thread 0x7f576406f700 (LWP 4565)):
 
#0  0x00007f576cc9c9f3 in futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55ba0faaf550 <mysql_bin_log+3248>) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
 
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x55ba0faaf4f8 <mysql_bin_log+3160>, cond=0x55ba0faaf528 <mysql_bin_log+3208>) at pthread_cond_wait.c:502
 
#2  __pthread_cond_wait (cond=0x55ba0faaf528 <mysql_bin_log+3208>, mutex=0x55ba0faaf4f8 <mysql_bin_log+3160>) at pthread_cond_wait.c:655
 
#3  0x000055ba0ec9eec3 in binlog_background_thread ()
 
#4  0x00007f576cc966db in start_thread (arg=0x7f576406f700) at pthread_create.c:463
 
#5  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 31 (Thread 0x7f5568ba2700 (LWP 4564)):
 
#0  0x00007f576cc9c9f3 in futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x7f5568ba1e28) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
 
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x7f5568ba1dd0, cond=0x7f5568ba1e00) at pthread_cond_wait.c:502
 
#2  __pthread_cond_wait (cond=0x7f5568ba1e00, mutex=0x7f5568ba1dd0) at pthread_cond_wait.c:655
 
#3  0x000055ba0ed4713b in ?? ()
 
#4  0x00007f576cc966db in start_thread (arg=0x7f5568ba2700) at pthread_create.c:463
 
#5  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 30 (Thread 0x7f55693a3700 (LWP 4563)):
 
#0  0x00007f576cc9c9f3 in futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55ba120680f0) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
 
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x55ba120680a0, cond=0x55ba120680c8) at pthread_cond_wait.c:502
 
#2  __pthread_cond_wait (cond=0x55ba120680c8, mutex=0x55ba120680a0) at pthread_cond_wait.c:655
 
#3  0x000055ba0edc4010 in ?? ()
 
#4  0x000055ba0eee3612 in ?? ()
 
#5  0x00007f576cc966db in start_thread (arg=0x7f55693a3700) at pthread_create.c:463
 
#6  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 29 (Thread 0x7f5569ba4700 (LWP 4562)):
 
#0  0x00007f576cca0c60 in __GI___nanosleep (requested_time=0x7f5569ba3750, remaining=0x0) at ../sysdeps/unix/sysv/linux/nanosleep.c:28
 
#1  0x000055ba0edc4280 in ?? ()
 
#2  0x000055ba0eed4b02 in ?? ()
 
#3  0x00007f576cc966db in start_thread (arg=0x7f5569ba4700) at pthread_create.c:463
 
#4  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 28 (Thread 0x7f556a3a5700 (LWP 4561)):
 
#0  0x00007f576cc9c9f3 in futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55ba12068010) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
 
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x55ba12067fc0, cond=0x55ba12067fe8) at pthread_cond_wait.c:502
 
#2  __pthread_cond_wait (cond=0x55ba12067fe8, mutex=0x55ba12067fc0) at pthread_cond_wait.c:655
 
#3  0x000055ba0edc4010 in ?? ()
 
#4  0x000055ba0eeeaf48 in ?? ()
 
#5  0x00007f576cc966db in start_thread (arg=0x7f556a3a5700) at pthread_create.c:463
 
#6  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 27 (Thread 0x7f556aba6700 (LWP 4560)):
 
#0  0x00007f576cc9c9f3 in futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55ba12067ec4) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
 
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x55ba12067e70, cond=0x55ba12067e98) at pthread_cond_wait.c:502
 
#2  __pthread_cond_wait (cond=0x55ba12067e98, mutex=0x55ba12067e70) at pthread_cond_wait.c:655
 
#3  0x000055ba0edc4010 in ?? ()
 
#4  0x000055ba0ee53699 in ?? ()
 
#5  0x00007f576cc966db in start_thread (arg=0x7f556aba6700) at pthread_create.c:463
 
#6  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 26 (Thread 0x7f556b3a7700 (LWP 4559)):
 
#0  0x00007f576cc9c9f3 in futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55ba12067e50) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
 
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x55ba12067e00, cond=0x55ba12067e28) at pthread_cond_wait.c:502
 
#2  __pthread_cond_wait (cond=0x55ba12067e28, mutex=0x55ba12067e00) at pthread_cond_wait.c:655
 
#3  0x000055ba0edc4010 in ?? ()
 
#4  0x000055ba0ee53699 in ?? ()
 
#5  0x00007f576cc966db in start_thread (arg=0x7f556b3a7700) at pthread_create.c:463
 
#6  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 25 (Thread 0x7f556bba8700 (LWP 4558)):
 
#0  0x00007f576cc9c9f3 in futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55ba12067de0) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
 
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x55ba12067d90, cond=0x55ba12067db8) at pthread_cond_wait.c:502
 
#2  __pthread_cond_wait (cond=0x55ba12067db8, mutex=0x55ba12067d90) at pthread_cond_wait.c:655
 
#3  0x000055ba0edc4010 in ?? ()
 
#4  0x000055ba0ee53699 in ?? ()
 
#5  0x00007f576cc966db in start_thread (arg=0x7f556bba8700) at pthread_create.c:463
 
#6  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 24 (Thread 0x7f556c3a9700 (LWP 4557)):
 
#0  0x00007f576cc9c9f3 in futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55ba12067d70) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
 
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x55ba12067d20, cond=0x55ba12067d48) at pthread_cond_wait.c:502
 
#2  __pthread_cond_wait (cond=0x55ba12067d48, mutex=0x55ba12067d20) at pthread_cond_wait.c:655
 
#3  0x000055ba0edc4010 in ?? ()
 
#4  0x000055ba0ee54cbd in ?? ()
 
#5  0x00007f576cc966db in start_thread (arg=0x7f556c3a9700) at pthread_create.c:463
 
#6  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 23 (Thread 0x7f556f3af700 (LWP 4556)):
 
#0  0x00007f576cca0c60 in __GI___nanosleep (requested_time=0x7f556f3aed60, remaining=0x0) at ../sysdeps/unix/sysv/linux/nanosleep.c:28
 
#1  0x000055ba0edc4280 in ?? ()
 
#2  0x000055ba0ee527e8 in ?? ()
 
#3  0x00007f576cc966db in start_thread (arg=0x7f556f3af700) at pthread_create.c:463
 
#4  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 22 (Thread 0x7f556cbaa700 (LWP 4555)):
 
#0  0x00007f576cc9cf85 in futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7f556cba9b10, expected=0, futex_word=0x55ba1c43d590) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
 
#1  __pthread_cond_wait_common (abstime=0x7f556cba9b10, mutex=0x55ba1c43d540, cond=0x55ba1c43d568) at pthread_cond_wait.c:539
 
#2  __pthread_cond_timedwait (cond=0x55ba1c43d568, mutex=0x55ba1c43d540, abstime=0x7f556cba9b10) at pthread_cond_wait.c:667
 
#3  0x000055ba0edc38b2 in ?? ()
 
#4  0x000055ba0edc3f01 in ?? ()
 
#5  0x000055ba0eea0540 in ?? ()
 
#6  0x000055ba0ef82102 in ?? ()
 
#7  0x00007f576cc966db in start_thread (arg=0x7f556cbaa700) at pthread_create.c:463
 
#8  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 21 (Thread 0x7f556d3ab700 (LWP 4554)):
 
#0  0x00007f576cc9cf85 in futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7f556d3aace0, expected=0, futex_word=0x55ba1c415a54) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
 
#1  __pthread_cond_wait_common (abstime=0x7f556d3aace0, mutex=0x55ba1c415a00, cond=0x55ba1c415a28) at pthread_cond_wait.c:539
 
#2  __pthread_cond_timedwait (cond=0x55ba1c415a28, mutex=0x55ba1c415a00, abstime=0x7f556d3aace0) at pthread_cond_wait.c:667
 
#3  0x000055ba0edc38b2 in ?? ()
 
#4  0x000055ba0edc3f01 in ?? ()
 
#5  0x000055ba0ef39e4b in ?? ()
 
#6  0x00007f576cc966db in start_thread (arg=0x7f556d3ab700) at pthread_create.c:463
 
#7  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 20 (Thread 0x7f556dbac700 (LWP 4553)):
 
#0  0x00007f576cc9cf85 in futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7f556dbabd70, expected=0, futex_word=0x55ba12067fa0) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
 
#1  __pthread_cond_wait_common (abstime=0x7f556dbabd70, mutex=0x55ba12067f50, cond=0x55ba12067f78) at pthread_cond_wait.c:539
 
#2  __pthread_cond_timedwait (cond=0x55ba12067f78, mutex=0x55ba12067f50, abstime=0x7f556dbabd70) at pthread_cond_wait.c:667
 
#3  0x000055ba0edc38b2 in ?? ()
 
#4  0x000055ba0edc3f01 in ?? ()
 
#5  0x000055ba0ee56408 in ?? ()
 
#6  0x00007f576cc966db in start_thread (arg=0x7f556dbac700) at pthread_create.c:463
 
#7  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 19 (Thread 0x7f556e3ad700 (LWP 4552)):
 
#0  0x00007f576cc9cf85 in futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7f556e3acc40, expected=0, futex_word=0x55ba12067f30) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
 
#1  __pthread_cond_wait_common (abstime=0x7f556e3acc40, mutex=0x55ba12067ee0, cond=0x55ba12067f08) at pthread_cond_wait.c:539
 
#2  __pthread_cond_timedwait (cond=0x55ba12067f08, mutex=0x55ba12067ee0, abstime=0x7f556e3acc40) at pthread_cond_wait.c:667
 
#3  0x000055ba0edc38b2 in ?? ()
 
#4  0x000055ba0edc3f01 in ?? ()
 
#5  0x000055ba0ee5179f in ?? ()
 
#6  0x00007f576cc966db in start_thread (arg=0x7f556e3ad700) at pthread_create.c:463
 
#7  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 18 (Thread 0x7f556ebae700 (LWP 4551)):
 
#0  0x00007f576cc9cf85 in futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7f556ebadce0, expected=0, futex_word=0x55ba1c3f3ae0) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
 
#1  __pthread_cond_wait_common (abstime=0x7f556ebadce0, mutex=0x55ba1c3f3a90, cond=0x55ba1c3f3ab8) at pthread_cond_wait.c:539
 
#2  __pthread_cond_timedwait (cond=0x55ba1c3f3ab8, mutex=0x55ba1c3f3a90, abstime=0x7f556ebadce0) at pthread_cond_wait.c:667
 
#3  0x000055ba0edc38b2 in ?? ()
 
#4  0x000055ba0edc3f01 in ?? ()
 
#5  0x000055ba0ed9d509 in ?? ()
 
#6  0x00007f576cc966db in start_thread (arg=0x7f556ebae700) at pthread_create.c:463
 
#7  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 17 (Thread 0x7f555e4f1700 (LWP 4549)):
 
#0  0x00007f576cc9c9f3 in futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55ba1c3f4bb4) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
 
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x55ba1c3f4b60, cond=0x55ba1c3f4b88) at pthread_cond_wait.c:502
 
#2  __pthread_cond_wait (cond=0x55ba1c3f4b88, mutex=0x55ba1c3f4b60) at pthread_cond_wait.c:655
 
#3  0x000055ba0edc4010 in ?? ()
 
#4  0x000055ba0eef3983 in ?? ()
 
#5  0x00007f576cc966db in start_thread (arg=0x7f555e4f1700) at pthread_create.c:463
 
#6  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 16 (Thread 0x7f555ecf2700 (LWP 4548)):
 
#0  0x00007f576cc9c9f3 in futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55ba1c3f4bb4) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
 
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x55ba1c3f4b60, cond=0x55ba1c3f4b88) at pthread_cond_wait.c:502
 
#2  __pthread_cond_wait (cond=0x55ba1c3f4b88, mutex=0x55ba1c3f4b60) at pthread_cond_wait.c:655
 
#3  0x000055ba0edc4010 in ?? ()
 
#4  0x000055ba0eef3983 in ?? ()
 
#5  0x00007f576cc966db in start_thread (arg=0x7f555ecf2700) at pthread_create.c:463
 
#6  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 15 (Thread 0x7f555f4f3700 (LWP 4547)):
 
#0  0x00007f576cc9c9f3 in futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55ba1c3f4bb4) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
 
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x55ba1c3f4b60, cond=0x55ba1c3f4b88) at pthread_cond_wait.c:502
 
#2  __pthread_cond_wait (cond=0x55ba1c3f4b88, mutex=0x55ba1c3f4b60) at pthread_cond_wait.c:655
 
#3  0x000055ba0edc4010 in ?? ()
 
#4  0x000055ba0eef3983 in ?? ()
 
#5  0x00007f576cc966db in start_thread (arg=0x7f555f4f3700) at pthread_create.c:463
 
#6  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 14 (Thread 0x7f55604f5700 (LWP 4545)):
 
#0  0x00007f576cc9cf85 in futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7f55604f4b70, expected=0, futex_word=0x55ba12068084) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
 
#1  __pthread_cond_wait_common (abstime=0x7f55604f4b70, mutex=0x55ba12068030, cond=0x55ba12068058) at pthread_cond_wait.c:539
 
#2  __pthread_cond_timedwait (cond=0x55ba12068058, mutex=0x55ba12068030, abstime=0x7f55604f4b70) at pthread_cond_wait.c:667
 
#3  0x000055ba0edc38b2 in ?? ()
 
#4  0x000055ba0edc3f01 in ?? ()
 
#5  0x000055ba0eef5784 in ?? ()
 
#6  0x00007f576cc966db in start_thread (arg=0x7f55604f5700) at pthread_create.c:463
 
#7  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 13 (Thread 0x7f5560cf6700 (LWP 4544)):
 
#0  0x00007f576e15563a in ?? () from target:/lib/x86_64-linux-gnu/libaio.so.1
 
#1  0x000055ba0edbd871 in ?? ()
 
#2  0x000055ba0edbde16 in ?? ()
 
#3  0x000055ba0edbf49b in ?? ()
 
#4  0x000055ba0ef471b8 in ?? ()
 
#5  0x000055ba0ee57210 in ?? ()
 
#6  0x00007f576cc966db in start_thread (arg=0x7f5560cf6700) at pthread_create.c:463
 
#7  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 12 (Thread 0x7f55614f7700 (LWP 4543)):
 
#0  0x00007f576e15563a in ?? () from target:/lib/x86_64-linux-gnu/libaio.so.1
 
#1  0x000055ba0edbd871 in ?? ()
 
#2  0x000055ba0edbde16 in ?? ()
 
#3  0x000055ba0edbf49b in ?? ()
 
#4  0x000055ba0ef471b8 in ?? ()
 
#5  0x000055ba0ee57210 in ?? ()
 
#6  0x00007f576cc966db in start_thread (arg=0x7f55614f7700) at pthread_create.c:463
 
#7  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 11 (Thread 0x7f5561cf8700 (LWP 4542)):
 
#0  0x00007f576e15563a in ?? () from target:/lib/x86_64-linux-gnu/libaio.so.1
 
#1  0x000055ba0edbd871 in ?? ()
 
#2  0x000055ba0edbde16 in ?? ()
 
#3  0x000055ba0edbf49b in ?? ()
 
#4  0x000055ba0ef471b8 in ?? ()
 
#5  0x000055ba0ee57210 in ?? ()
 
#6  0x00007f576cc966db in start_thread (arg=0x7f5561cf8700) at pthread_create.c:463
 
#7  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 10 (Thread 0x7f55624f9700 (LWP 4541)):
 
#0  0x00007f576e15563a in ?? () from target:/lib/x86_64-linux-gnu/libaio.so.1
 
#1  0x000055ba0edbd871 in ?? ()
 
#2  0x000055ba0edbde16 in ?? ()
 
#3  0x000055ba0edbf49b in ?? ()
 
#4  0x000055ba0ef471b8 in ?? ()
 
#5  0x000055ba0ee57210 in ?? ()
 
#6  0x00007f576cc966db in start_thread (arg=0x7f55624f9700) at pthread_create.c:463
 
#7  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 9 (Thread 0x7f5562cfa700 (LWP 4540)):
 
#0  0x00007f576e15563a in ?? () from target:/lib/x86_64-linux-gnu/libaio.so.1
 
#1  0x000055ba0edbd871 in ?? ()
 
#2  0x000055ba0edbde16 in ?? ()
 
#3  0x000055ba0edbf49b in ?? ()
 
#4  0x000055ba0ef471b8 in ?? ()
 
#5  0x000055ba0ee57210 in ?? ()
 
#6  0x00007f576cc966db in start_thread (arg=0x7f5562cfa700) at pthread_create.c:463
 
#7  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 8 (Thread 0x7f55634fb700 (LWP 4539)):
 
#0  0x00007f576e15563a in ?? () from target:/lib/x86_64-linux-gnu/libaio.so.1
 
#1  0x000055ba0edbd871 in ?? ()
 
#2  0x000055ba0edbde16 in ?? ()
 
#3  0x000055ba0edbf49b in ?? ()
 
#4  0x000055ba0ef471b8 in ?? ()
 
#5  0x000055ba0ee57210 in ?? ()
 
#6  0x00007f576cc966db in start_thread (arg=0x7f55634fb700) at pthread_create.c:463
 
#7  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 7 (Thread 0x7f5563cfc700 (LWP 4538)):
 
#0  0x00007f576e15563a in ?? () from target:/lib/x86_64-linux-gnu/libaio.so.1
 
#1  0x000055ba0edbd871 in ?? ()
 
#2  0x000055ba0edbde16 in ?? ()
 
#3  0x000055ba0edbf49b in ?? ()
 
#4  0x000055ba0ef471b8 in ?? ()
 
#5  0x000055ba0ee57210 in ?? ()
 
#6  0x00007f576cc966db in start_thread (arg=0x7f5563cfc700) at pthread_create.c:463
 
#7  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 6 (Thread 0x7f55644fd700 (LWP 4537)):
 
#0  0x00007f576e15563a in ?? () from target:/lib/x86_64-linux-gnu/libaio.so.1
 
#1  0x000055ba0edbd871 in ?? ()
 
#2  0x000055ba0edbde16 in ?? ()
 
#3  0x000055ba0edbf49b in ?? ()
 
#4  0x000055ba0ef471b8 in ?? ()
 
#5  0x000055ba0ee57210 in ?? ()
 
#6  0x00007f576cc966db in start_thread (arg=0x7f55644fd700) at pthread_create.c:463
 
#7  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 5 (Thread 0x7f5564cfe700 (LWP 4536)):
 
#0  0x00007f576e15563a in ?? () from target:/lib/x86_64-linux-gnu/libaio.so.1
 
#1  0x000055ba0edbd871 in ?? ()
 
#2  0x000055ba0edbde16 in ?? ()
 
#3  0x000055ba0edbf49b in ?? ()
 
#4  0x000055ba0ef471b8 in ?? ()
 
#5  0x000055ba0ee57210 in ?? ()
 
#6  0x00007f576cc966db in start_thread (arg=0x7f5564cfe700) at pthread_create.c:463
 
#7  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 4 (Thread 0x7f5750872700 (LWP 4535)):
 
#0  0x00007f576e15563a in ?? () from target:/lib/x86_64-linux-gnu/libaio.so.1
 
#1  0x000055ba0edbd871 in ?? ()
 
#2  0x000055ba0edbde16 in ?? ()
 
#3  0x000055ba0edbf49b in ?? ()
 
#4  0x000055ba0ef471b8 in ?? ()
 
#5  0x000055ba0ee57210 in ?? ()
 
#6  0x00007f576cc966db in start_thread (arg=0x7f5750872700) at pthread_create.c:463
 
#7  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 3 (Thread 0x7f576489a700 (LWP 4533)):
 
#0  0x00007f576cc9cf85 in futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7f5764899d60, expected=0, futex_word=0x55ba10305388) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
 
#1  __pthread_cond_wait_common (abstime=0x7f5764899d60, mutex=0x55ba103053a0, cond=0x55ba10305360) at pthread_cond_wait.c:539
 
#2  __pthread_cond_timedwait (cond=0x55ba10305360, mutex=0x55ba103053a0, abstime=0x7f5764899d60) at pthread_cond_wait.c:667
 
#3  0x000055ba0effcef9 in ?? ()
 
#4  0x000055ba0eff1110 in ?? ()
 
#5  0x00007f576cc966db in start_thread (arg=0x7f576489a700) at pthread_create.c:463
 
#6  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 2 (Thread 0x7f5766dc2700 (LWP 4532)):
 
#0  0x00007f576cc9cf85 in futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7f5766dc1df0, expected=0, futex_word=0x55ba10314da8) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
 
#1  __pthread_cond_wait_common (abstime=0x7f5766dc1df0, mutex=0x55ba10314dc0, cond=0x55ba10314d80) at pthread_cond_wait.c:539
 
#2  __pthread_cond_timedwait (cond=0x55ba10314d80, mutex=0x55ba10314dc0, abstime=0x7f5766dc1df0) at pthread_cond_wait.c:667
 
#3  0x000055ba0f137cf8 in ?? ()
 
#4  0x00007f576cc966db in start_thread (arg=0x7f5766dc2700) at pthread_create.c:463
 
#5  0x00007f576c29888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
Thread 1 (Thread 0x7f576e78b540 (LWP 4531)):
 
#0  0x00007f576c28bbf9 in __GI___poll (fds=0x7ffff244e270, nfds=3, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
 
#1  0x000055ba0e9398b7 in handle_connections_sockets() ()
 
#2  0x000055ba0e93b050 in mysqld_main(int, char**) ()
 
#3  0x00007f576c198b97 in __libc_start_main (main=0x55ba0e919af0 <main>, argc=1, argv=0x7ffff244e618, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffff244e608) at ../csu/libc-start.c:310
 
#4  0x000055ba0e92d4ca in _start ()

Comment by Hartmut Holzgraefe [ 2020-04-01 ]

I am running an ubuntu/bionic64 virtualbox VM controlled by Vagrant to reproduce this.

I can only reproduce it when starting the mysql client manually from bash prompt, when I try to fully automate the test connections never fails.

Valerii could also reproduce this yesterday, I don't know his exact setup though ...

Comment by Elena Stepanova [ 2020-04-01 ]

Is it an Ubuntu package installation, or a binary tarball, or a source build?
Does the client belong to the same package, or can it be that it's pre-installed separately in the VM? Are there any other MySQL/MariaDB pieces pre-installed system-wide? (In modern Ubuntus/Debians there usually are).
Are all connections connecting via socket or tcp?

I'm not sure if any of this is important, for now just trying to find out what the difference might be.

Comment by Hartmut Holzgraefe [ 2020-04-01 ]

In my case latest binaries from our Ubuntu repository. Valerii used self compiled 10.3 as far as I remember.

I install with 

curl -sS https://downloads.mariadb.com/MariaDB/mariadb_repo_setup | sudo bash -s -- --mariadb-server-version=mariadb-10.3
 
apt-get install -y mariadb-server

I use simple socket connections only.

Comment by Hartmut Holzgraefe [ 2020-04-01 ]

Ok, now this is somewhat embarrassing, the problem is indeed somewhere else completely.

My vagrant test environment had a configuration artifact from a previous test regarding some pool-of-threads:

thread_handling=pool-of-threads
 
thread_pool_size=3
 
thread_pool_max_threads=5
 
thread_pool_oversubscribe=0

I'm still not sure why this only fails for two connect attempts out of three, but it's actually the thread pool configuration, and not the FLUSH, that seems to be causing this.

Need to discuss with Valerii tomorrow what his exact setup was ...

Generated at Thu Feb 08 09:12:10 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.