[MDEV-8890] Server gets locked after killing ANALYZE INSERT/REPLACE DELAYED Created: 2015-10-02  Updated: 2015-10-11

Status: Confirmed
Project: MariaDB Server
Component/s: Admin statements, Data Manipulation - Insert
Affects Version/s: 10.1
Fix Version/s: 10.1

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Sergei Petrunia
Resolution: Unresolved Votes: 0
Labels: deadlock, hang

Attachments: HTML File threads1     HTML File threads2    

 Description   

Note: see also MDEV-8889, they might be somehow related.

The test runs ANALYZE REPLACE DELAYED and kills some of these queries in a rapid fashion. Soon after the start the flow gets stuck, and the processlist looks like this:

+----+---------+-----------------+-------+----------------+------+------------------------------+-------------------------------------------------------------------------+----------+
 
| Id | User    | Host            | db    | Command        | Time | State                        | Info                                                                    | Progress |
 
+----+---------+-----------------+-------+----------------+------+------------------------------+-------------------------------------------------------------------------+----------+
 
|  2 | rqg     | localhost:33215 | mysql | Sleep          |   53 |                              | NULL                                                                    |    0.000 |
 
|  8 | rqg     | localhost:33221 | test  | Sleep          |    0 |                              | NULL                                                                    |    0.000 |
 
|  9 | rqg     | localhost:33222 | test  | Killed         |   32 | waiting for handler insert   | ANALYZE REPLACE DELAYED INTO `BB` ( `col_varchar_key` ) VALUES ( NULL ) |    0.000 |
 
| 10 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
 
| 11 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
 
| 12 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
 
| 13 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
 
| 14 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
 
| 15 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
 
| 16 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for table level lock |                                                                         |    0.000 |
 
| 17 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
 
| 18 | rqg     | localhost:33223 | test  | Killed         |   31 | waiting for handler insert   | ANALYZE REPLACE DELAYED INTO `BB` ( `col_time_nokey` ) VALUES ( NULL )  |    0.000 |
 
| 19 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
 
| 20 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
 
| 21 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
 
| 22 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
 
| 23 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for table level lock |                                                                         |    0.000 |
 
| 24 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
 
| 25 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for table level lock |                                                                         |    0.000 |
 
| 26 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for table level lock |                                                                         |    0.000 |
 
| 27 | root    | localhost:38193 | test  | Query          |    0 | init                         | show processlist                                                        |    0.000 |
 
+----+---------+-----------------+-------+----------------+------+------------------------------+-------------------------------------------------------------------------+----------+

If lock_wait_timeout is set to a reasonable value, eventually the timeout is reached, and the hanging ANALYZE queries die, along with some of DELAYED connections; but some DELAYED stay:

MariaDB [test]> show processlist;
 
+----+---------+-----------------+------+----------------+------+--------------------+------------------+----------+
 
| Id | User    | Host            | db   | Command        | Time | State              | Info             | Progress |
 
+----+---------+-----------------+------+----------------+------+--------------------+------------------+----------+
 
| 10 | DELAYED | localhost       | test | Killed         | NULL | Waiting for INSERT |                  |    0.000 |
 
| 11 | DELAYED | localhost       | test | Killed         | NULL | Waiting for INSERT |                  |    0.000 |
 
| 13 | DELAYED | localhost       | test | Killed         | NULL | Waiting for INSERT |                  |    0.000 |
 
| 14 | DELAYED | localhost       | test | Killed         | NULL | Waiting for INSERT |                  |    0.000 |
 
| 16 | DELAYED | localhost       | test | Delayed insert | NULL | Waiting for INSERT |                  |    0.000 |
 
| 17 | DELAYED | localhost       | test | Killed         | NULL | Waiting for INSERT |                  |    0.000 |
 
| 23 | DELAYED | localhost       | test | Delayed insert | NULL | Waiting for INSERT |                  |    0.000 |
 
| 25 | DELAYED | localhost       | test | Delayed insert | NULL | Waiting for INSERT |                  |    0.000 |
 
| 26 | DELAYED | localhost       | test | Delayed insert | NULL | Waiting for INSERT |                  |    0.000 |
 
| 27 | root    | localhost:38193 | test | Query          |    0 | init               | show processlist |    0.000 |
 
+----+---------+-----------------+------+----------------+------+--------------------+------------------+----------+

And further the server hangs on shutdown. Two thread stacks taken with a few seconds interval are attached, here is the diff:

Stack traces from 10.1 commit a84fae27d9badef8cc89b1369a90cfef1dc3c715

 
< #2  0x00007f85a72d50ed in page_cleaner_sleep_if_needed (next_loop_time=1443813478905) at 10.1/storage/xtradb/buf/buf0flu.cc:2644
 
---
 
> #2  0x00007f85a72d50ed in page_cleaner_sleep_if_needed (next_loop_time=1443813522777) at 10.1/storage/xtradb/buf/buf0flu.cc:2644
 
216c216
 
< #2  0x00007f85a72d50ed in page_cleaner_sleep_if_needed (next_loop_time=1443813478918) at 10.1/storage/xtradb/buf/buf0flu.cc:2644
 
---
 
> #2  0x00007f85a72d50ed in page_cleaner_sleep_if_needed (next_loop_time=1443813522792) at 10.1/storage/xtradb/buf/buf0flu.cc:2644
 
278,285d277
 
< #0  safe_mutex_unlock (mp=0x7f85a8010da0, file=0x7f85a7633ce0 "10.1/sql/sql_insert.cc", line=2886) at 10.1/mysys/thr_mutex.c:452
 
< #1  0x00007f85a6cb30d9 in inline_mysql_mutex_unlock (that=0x7f85a8010da0, src_file=0x7f85a7633ce0 "10.1/sql/sql_insert.cc", src_line=2886) at 10.1/include/mysql/psi/mysql_thread.h:761
 
< #2  0x00007f85a6cb9c19 in handle_delayed_insert (arg=0x7f857fca2070) at 10.1/sql/sql_insert.cc:2886
 
< #3  0x00007f85a63a9b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
 
< #4  0x00007f85a423595d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
 
< #5  0x0000000000000000 in ?? ()
 
< 
< Thread 9 (Thread 0x7f85a4110700 (LWP 5137)):
 
291c283
 
< #5  0x00007f85a6cb9be3 in handle_delayed_insert (arg=0x7f857fcae070) at 10.1/sql/sql_insert.cc:2883
 
---
 
> #5  0x00007f85a6cb9be3 in handle_delayed_insert (arg=0x7f857fca2070) at 10.1/sql/sql_insert.cc:2883
 
294a287,296
 
> 
> Thread 9 (Thread 0x7f85a4110700 (LWP 5137)):
 
> #0  0x00007f85a63ad69e in __pthread_mutex_unlock_usercnt (mutex=0x7f85a8885100, decr=<optimized out>) at pthread_mutex_unlock.c:52
 
> #1  0x00007f85a7588f63 in _db_doprnt_ (format=0x7f85a78acb60 "%s (0x%lx) unlocking") at 10.1/dbug/dbug.c:1284
 
> #2  0x00007f85a756e5ec in safe_mutex_unlock (mp=0x7f857fcb3478, file=0x7f85a7633ce0 "10.1/sql/sql_insert.cc", line=2882) at 10.1/mysys/thr_mutex.c:409
 
> #3  0x00007f85a6cb30d9 in inline_mysql_mutex_unlock (that=0x7f857fcb3478, src_file=0x7f85a7633ce0 "10.1/sql/sql_insert.cc", src_line=2882) at 10.1/include/mysql/psi/mysql_thread.h:761
 
> #4  0x00007f85a6cb9bc8 in handle_delayed_insert (arg=0x7f857fcae070) at 10.1/sql/sql_insert.cc:2882
 
> #5  0x00007f85a63a9b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
 
> #6  0x00007f85a423595d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
 
> #7  0x0000000000000000 in ?? ()

RQG grammar test.yy

 
thread1:
 
	SELECT Id INTO @kill_id FROM INFORMATION_SCHEMA.PROCESSLIST WHERE Info LIKE 'ANALYZE REPLACE DELAYED%' LIMIT 1; KILL QUERY @kill_id ;
 
 
 
query:
 
	ANALYZE REPLACE DELAYED INTO _table ( _field ) VALUES ( NULL ) ;

RQG command line

 
perl ./runall-new.pl --threads=3 --duration=300 --queries=100M --grammar=test.yy --engine=MyISAM --basedir1=<your basedir> --vardir1=<your vardir>

For RQG, I recommend using lp:~elenst/randgen/mariadb-patches

 Comments   
Comment by Sergei Petrunia [ 2015-10-03 ]

@elenst, does the problem occur if the test runs regular INSERT/REPLACE DELAYED instead of ANALYZE ?

Comment by Elena Stepanova [ 2015-10-03 ]

psergey,

I could not reproduce it without ANALYZE. It does not mean it cannot happen on any other circumstances, it might be that I did not find the right mix.
If I add ALTER TABLE, I don't need ANALYZE, but then I get MDEV-8889 as mentioned in the description. If I remove ALTER TABLE and just leave REPLACE and KILL, I am not getting the deadlock.

Comment by Sergei Petrunia [ 2015-10-11 ]

When I attempt to run it in the current tree
I get output like this:

MariaDB [(none)]> show processlist;
 
+----+---------+-----------------+-------+----------------+------+------------------------------+--------------------------------------------------------------------------+----------+
 
| Id | User    | Host            | db    | Command        | Time | State                        | Info                                                                     | Progress |
 
+----+---------+-----------------+-------+----------------+------+------------------------------+--------------------------------------------------------------------------+----------+
 
|  2 | rqg     | localhost:46569 | mysql | Sleep          |  269 |                              | NULL                                                                     |    0.000 |
 
|  8 | rqg     | localhost:46575 | test  | Query          |    0 | init                         | KILL QUERY @kill_id                                                      |    0.000 |
 
|  9 | rqg     | localhost:46576 | test  | Query          |  266 | waiting for handler insert   | ANALYZE REPLACE DELAYED INTO `BB` ( `col_datetime_key` ) VALUES ( NULL ) |    0.000 |
 
| 10 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
 
| 11 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
 
| 12 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
 
| 13 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
 
| 14 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
 
| 15 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
 
| 16 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
 
| 17 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
 
| 18 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for table level lock |                                                                          |    0.000 |
 
| 19 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for table level lock |                                                                          |    0.000 |
 
| 20 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
 
| 21 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
 
| 22 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for table level lock |                                                                          |    0.000 |
 
| 23 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for table level lock |                                                                          |    0.000 |
 
| 24 | rqg     | localhost:46577 | test  | Killed         |  266 | waiting for handler insert   | ANALYZE REPLACE DELAYED INTO `BB` ( `col_int_key` ) VALUES ( NULL )      |    0.000 |
 
| 25 | root    | localhost:46578 | NULL  | Query          |    0 | init                         | show processlist                                                         |    0.000 |
 
+----+---------+-----------------+-------+----------------+------+------------------------------+--------------------------------------------------------------------------+----------+
 
19 rows in set (0.00 sec)

Comment by Sergei Petrunia [ 2015-10-11 ]

However, the server didn't hung:

# 2015-10-11T14:01:36 [29636] GenTest exited with exit status STATUS_OK (0)
 
# 2015-10-11T14:01:36 [29636] Stopping server(s)...
 
# 2015-10-11T14:01:36 [29636] Stopping server on port 19300
 
# 2015-10-11T14:01:36 [29636] [29636] ./runall-new.pl will exit with exit status STATUS_OK (0)

Comment by Sergei Petrunia [ 2015-10-11 ]

Yes it did. After randgen has finished, mysqld process is left running; 

Thread 11 (Thread 0x7f9bc01bc700 (LWP 29731)):
 
#0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
 
#1  0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0
 
#2  0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101
 
#3  0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294
 
#4  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
 
#5  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6000e610) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
 
#6  0x00007f9bc1ffce9a in start_thread (arg=0x7f9bc01bc700) at pthread_create.c:308
 
#7  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
 
#8  0x0000000000000000 in ?? ()
 
...
 
 
 
Thread 10 (Thread 0x7f9bc0173700 (LWP 29732)):
 
#0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
 
#1  0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0
 
#2  0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101
 
#3  0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294
 
#4  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
 
#5  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b60019110) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
 
#6  0x00007f9bc1ffce9a in start_thread (arg=0x7f9bc0173700) at pthread_create.c:308
 
#7  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
 
#8  0x0000000000000000 in ?? ()
 
 
 
Thread 9 (Thread 0x7f9bc00e1700 (LWP 29734)):
 
#0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
 
#1  0x00007f9bc1fff065 in _L_lock_858 () from /lib/x86_64-linux-gnu/libpthread.so.0
 
#2  0x00007f9bc1ffeeba in __pthread_mutex_lock (mutex=0x7f9bc4e6aea0) at pthread_mutex_lock.c:61
 
#3  0x00007f9bc44597a7 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:250
 
#4  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
 
---Type <return> to continue, or q <return> to quit---
 
#5  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6002ba80) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
 
#6  0x00007f9bc1ffce9a in start_thread (arg=0x7f9bc00e1700) at pthread_create.c:308
 
#7  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
 
#8  0x0000000000000000 in ?? ()
 
 
 
Thread 8 (Thread 0x7f9bc004f700 (LWP 29736)):
 
#0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
 
#1  0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0
 
#2  0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101
 
#3  0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294
 
#4  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
 
#5  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6003e600) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
 
#6  0x00007f9bc1ffce9a in start_thread (arg=0x7f9bc004f700) at pthread_create.c:308
 
#7  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
 
#8  0x0000000000000000 in ?? ()
 
 
 
Thread 7 (Thread 0x7f9ba84e9700 (LWP 29739)):
 
#0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
 
#1  0x00007f9bc1fff065 in _L_lock_858 () from /lib/x86_64-linux-gnu/libpthread.so.0
 
#2  0x00007f9bc1ffeeba in __pthread_mutex_lock (mutex=0x7f9bc4e6aea0) at pthread_mutex_lock.c:61
 
#3  0x00007f9bc4459889 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:296
 
#4  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
 
#5  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6005a6c0) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
 
#6  0x00007f9bc1ffce9a in start_thread (arg=0x7f9ba84e9700) at pthread_create.c:308
 
#7  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
 
#8  0x0000000000000000 in ?? ()
 
 
 
Thread 6 (Thread 0x7f9ba84a0700 (LWP 29740)):
 
#0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
 
#1  0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0
 
#2  0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101
 
#3  0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294
 
#4  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
 
#5  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b60063f40) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
 
#6  0x00007f9bc1ffce9a in start_thread (arg=0x7f9ba84a0700) at pthread_create.c:308
 
#7  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
 
#8  0x0000000000000000 in ?? ()
 
 
 
Thread 5 (Thread 0x7f9ba8457700 (LWP 29741)):
 
#0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
 
---Type <return> to continue, or q <return> to quit---
 
#1  0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0
 
#2  0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101
 
#3  0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294
 
#4  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
 
#5  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6006d7c0) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
 
#6  0x00007f9bc1ffce9a in start_thread (arg=0x7f9ba8457700) at pthread_create.c:308
 
#7  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
 
#8  0x0000000000000000 in ?? ()
 
 
 
Thread 4 (Thread 0x7f9ba83c5700 (LWP 29743)):
 
#0  __pthread_mutex_unlock (mutex=0x7f9bc56df220) at pthread_mutex_unlock.c:289
 
#1  0x00007f9bc4459838 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:269
 
#2  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
 
#3  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b600810c0) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
 
#4  0x00007f9bc1ffce9a in start_thread (arg=0x7f9ba83c5700) at pthread_create.c:308
 
#5  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
 
#6  0x0000000000000000 in ?? ()
 
 
 
Thread 3 (Thread 0x7f9ba837c700 (LWP 29744)):
 
#0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
 
#1  0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0
 
#2  0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101
 
#3  0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294
 
#4  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
 
    at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
 
#5  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6008a9e0) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
 
#6  0x00007f9bc1ffce9a in start_thread (arg=0x7f9ba837c700) at pthread_create.c:308
 
#7  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
 
#8  0x0000000000000000 in ?? ()

Generated at Thu Feb 08 07:30:33 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.