|
Parallel slave seems to arrive at dealock. Show slave status displays all but two threads
are idle, and two remaining a Worker and the driver thread were waiting for each other.
[
| 147983 |
system user |
|
NULL |
Connect |
641 |
Waiting for master to send event |
NULL |
0.000 |
| 148044 |
system user |
|
NULL |
Connect |
568 |
Waiting for work from SQL thread |
NULL |
0.000 |
| 148045 |
system user |
|
NULL |
Connect |
568 |
Waiting for prior transaction to start commit before starting next transaction |
NULL |
0.000 |
| 148046 |
system user |
|
NULL |
Connect |
500 |
Waiting for work from SQL thread |
NULL |
0.000 |
| 148047 |
system user |
|
NULL |
Connect |
568 |
Waiting for work from SQL thread |
NULL |
0.000 |
| 148048 |
system user |
|
NULL |
Connect |
502 |
Waiting for room in worker thread event queue |
NULL |
Furthermore a stack trace details how they are hanging.
A suspicion is that the thread 4 missed a notification to proceed and release
room that the SQL thread was concerned with.
Thread 4 (Thread 0x7f02b1061700 (LWP 28448)):
#0 0x00007f0436725945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x000055e99ea656e0 in inline_mysql_cond_wait (src_file=0x55e99f042d00 "/home/buildbot/buildbot/build/mariadb-10.1.19/sql/rpl_parallel.cc", src_line=348, mutex=0x7ef825f32c08, that=0x7ef7cb8cf088) at /home/buildbot/buildbot/build/mariadb-10.1.19/include/mysql/psi/mysql_thread.h:1165
#2 do_gco_wait (old_stage=0x7f02b1060ca0, did_enter_cond=<synthetic pointer>, gco=0x7ef7cb8cf088, rgi=0x7ef78778a900) at /home/buildbot/buildbot/build/mariadb-10.1.19/sql/rpl_parallel.cc:348
#3 handle_rpl_parallel_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.1.19/sql/rpl_parallel.cc:1141
#4 0x00007f0436721e25 in start_thread () from /lib64/libpthread.so.0
#5 0x00007f0434b4034d in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x7f013fbae700 (LWP 28449)):
#0 0x00007f0436725945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x000055e99ea64f73 in inline_mysql_cond_wait (src_file=0x55e99f042d00 "/home/buildbot/buildbot/build/mariadb-10.1.19/sql/rpl_parallel.cc", src_line=1052, mutex=<optimized out>, that=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.1.19/include/mysql/psi/mysql_thread.h:1165
#2 handle_rpl_parallel_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.1.19/sql/rpl_parallel.cc:1052
#3 0x00007f0436721e25 in start_thread () from /lib64/libpthread.so.0
#4 0x00007f0434b4034d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x7f02b10a2700 (LWP 28450)):
#0 0x00007f0436725945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x000055e99ea65cb3 in inline_mysql_cond_wait (src_file=0x55e99f042d00 "/home/buildbot/buildbot/build/mariadb-10.1.19/sql/rpl_parallel.cc", src_line=2018, mutex=0x7ef7dda5cd40, that=0x7ef7dda5cda8) at /home/buildbot/buildbot/build/mariadb-10.1.19/include/mysql/psi/mysql_thread.h:1165
#2 rpl_parallel_entry::choose_thread (this=this@entry=0x7ef825f32c08, rgi=rgi@entry=0x7ef788e98600, did_enter_cond=did_enter_cond@entry=0x7f02b10a12c0, old_stage=old_stage@entry=0x7f02b10a12d0, reuse=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.1.19/sql/rpl_parallel.cc:2018
#3 0x000055e99ea66c48 in rpl_parallel::do_event (this=this@entry=0x7f0431be24b8, serial_rgi=serial_rgi@entry=0x7ef788e98600, ev=ev@entry=0x7ef78d5d4148, event_size=event_size@entry=3366) at /home/buildbot/buildbot/build/mariadb-10.1.19/sql/rpl_parallel.cc:2552
#4 0x000055e99e8d7731 in exec_relay_log_event (serial_rgi=0x7ef788e98600, rli=0x7f0431bdf9a0, thd=0x7ef781f1f008) at /home/buildbot/buildbot/build/mariadb-10.1.19/sql/slave.cc:3813
#5 handle_slave_sql (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.1.19/sql/slave.cc:4937
#6 0x00007f0436721e25 in start_thread () from /lib64/libpthread.so.0
#7 0x00007f0434b4034d in clone () from /lib64/libc.so.6
|