Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-8890

Server gets locked after killing ANALYZE INSERT/REPLACE DELAYED

Details

    Description

      Note: see also MDEV-8889, they might be somehow related.

      The test runs ANALYZE REPLACE DELAYED and kills some of these queries in a rapid fashion. Soon after the start the flow gets stuck, and the processlist looks like this:

      +----+---------+-----------------+-------+----------------+------+------------------------------+-------------------------------------------------------------------------+----------+
      | Id | User    | Host            | db    | Command        | Time | State                        | Info                                                                    | Progress |
      +----+---------+-----------------+-------+----------------+------+------------------------------+-------------------------------------------------------------------------+----------+
      |  2 | rqg     | localhost:33215 | mysql | Sleep          |   53 |                              | NULL                                                                    |    0.000 |
      |  8 | rqg     | localhost:33221 | test  | Sleep          |    0 |                              | NULL                                                                    |    0.000 |
      |  9 | rqg     | localhost:33222 | test  | Killed         |   32 | waiting for handler insert   | ANALYZE REPLACE DELAYED INTO `BB` ( `col_varchar_key` ) VALUES ( NULL ) |    0.000 |
      | 10 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
      | 11 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
      | 12 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
      | 13 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
      | 14 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
      | 15 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
      | 16 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for table level lock |                                                                         |    0.000 |
      | 17 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
      | 18 | rqg     | localhost:33223 | test  | Killed         |   31 | waiting for handler insert   | ANALYZE REPLACE DELAYED INTO `BB` ( `col_time_nokey` ) VALUES ( NULL )  |    0.000 |
      | 19 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
      | 20 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
      | 21 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
      | 22 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
      | 23 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for table level lock |                                                                         |    0.000 |
      | 24 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                         |    0.000 |
      | 25 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for table level lock |                                                                         |    0.000 |
      | 26 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for table level lock |                                                                         |    0.000 |
      | 27 | root    | localhost:38193 | test  | Query          |    0 | init                         | show processlist                                                        |    0.000 |
      +----+---------+-----------------+-------+----------------+------+------------------------------+-------------------------------------------------------------------------+----------+

      If lock_wait_timeout is set to a reasonable value, eventually the timeout is reached, and the hanging ANALYZE queries die, along with some of DELAYED connections; but some DELAYED stay:

      MariaDB [test]> show processlist;
      +----+---------+-----------------+------+----------------+------+--------------------+------------------+----------+
      | Id | User    | Host            | db   | Command        | Time | State              | Info             | Progress |
      +----+---------+-----------------+------+----------------+------+--------------------+------------------+----------+
      | 10 | DELAYED | localhost       | test | Killed         | NULL | Waiting for INSERT |                  |    0.000 |
      | 11 | DELAYED | localhost       | test | Killed         | NULL | Waiting for INSERT |                  |    0.000 |
      | 13 | DELAYED | localhost       | test | Killed         | NULL | Waiting for INSERT |                  |    0.000 |
      | 14 | DELAYED | localhost       | test | Killed         | NULL | Waiting for INSERT |                  |    0.000 |
      | 16 | DELAYED | localhost       | test | Delayed insert | NULL | Waiting for INSERT |                  |    0.000 |
      | 17 | DELAYED | localhost       | test | Killed         | NULL | Waiting for INSERT |                  |    0.000 |
      | 23 | DELAYED | localhost       | test | Delayed insert | NULL | Waiting for INSERT |                  |    0.000 |
      | 25 | DELAYED | localhost       | test | Delayed insert | NULL | Waiting for INSERT |                  |    0.000 |
      | 26 | DELAYED | localhost       | test | Delayed insert | NULL | Waiting for INSERT |                  |    0.000 |
      | 27 | root    | localhost:38193 | test | Query          |    0 | init               | show processlist |    0.000 |
      +----+---------+-----------------+------+----------------+------+--------------------+------------------+----------+

      And further the server hangs on shutdown. Two thread stacks taken with a few seconds interval are attached, here is the diff:

      Stack traces from 10.1 commit a84fae27d9badef8cc89b1369a90cfef1dc3c715

      < #2  0x00007f85a72d50ed in page_cleaner_sleep_if_needed (next_loop_time=1443813478905) at 10.1/storage/xtradb/buf/buf0flu.cc:2644
      ---
      > #2  0x00007f85a72d50ed in page_cleaner_sleep_if_needed (next_loop_time=1443813522777) at 10.1/storage/xtradb/buf/buf0flu.cc:2644
      216c216
      < #2  0x00007f85a72d50ed in page_cleaner_sleep_if_needed (next_loop_time=1443813478918) at 10.1/storage/xtradb/buf/buf0flu.cc:2644
      ---
      > #2  0x00007f85a72d50ed in page_cleaner_sleep_if_needed (next_loop_time=1443813522792) at 10.1/storage/xtradb/buf/buf0flu.cc:2644
      278,285d277
      < #0  safe_mutex_unlock (mp=0x7f85a8010da0, file=0x7f85a7633ce0 "10.1/sql/sql_insert.cc", line=2886) at 10.1/mysys/thr_mutex.c:452
      < #1  0x00007f85a6cb30d9 in inline_mysql_mutex_unlock (that=0x7f85a8010da0, src_file=0x7f85a7633ce0 "10.1/sql/sql_insert.cc", src_line=2886) at 10.1/include/mysql/psi/mysql_thread.h:761
      < #2  0x00007f85a6cb9c19 in handle_delayed_insert (arg=0x7f857fca2070) at 10.1/sql/sql_insert.cc:2886
      < #3  0x00007f85a63a9b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
      < #4  0x00007f85a423595d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
      < #5  0x0000000000000000 in ?? ()
      < 
      < Thread 9 (Thread 0x7f85a4110700 (LWP 5137)):
      291c283
      < #5  0x00007f85a6cb9be3 in handle_delayed_insert (arg=0x7f857fcae070) at 10.1/sql/sql_insert.cc:2883
      ---
      > #5  0x00007f85a6cb9be3 in handle_delayed_insert (arg=0x7f857fca2070) at 10.1/sql/sql_insert.cc:2883
      294a287,296
      > 
      > Thread 9 (Thread 0x7f85a4110700 (LWP 5137)):
      > #0  0x00007f85a63ad69e in __pthread_mutex_unlock_usercnt (mutex=0x7f85a8885100, decr=<optimized out>) at pthread_mutex_unlock.c:52
      > #1  0x00007f85a7588f63 in _db_doprnt_ (format=0x7f85a78acb60 "%s (0x%lx) unlocking") at 10.1/dbug/dbug.c:1284
      > #2  0x00007f85a756e5ec in safe_mutex_unlock (mp=0x7f857fcb3478, file=0x7f85a7633ce0 "10.1/sql/sql_insert.cc", line=2882) at 10.1/mysys/thr_mutex.c:409
      > #3  0x00007f85a6cb30d9 in inline_mysql_mutex_unlock (that=0x7f857fcb3478, src_file=0x7f85a7633ce0 "10.1/sql/sql_insert.cc", src_line=2882) at 10.1/include/mysql/psi/mysql_thread.h:761
      > #4  0x00007f85a6cb9bc8 in handle_delayed_insert (arg=0x7f857fcae070) at 10.1/sql/sql_insert.cc:2882
      > #5  0x00007f85a63a9b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
      > #6  0x00007f85a423595d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
      > #7  0x0000000000000000 in ?? ()

      RQG grammar test.yy

      thread1:
      	SELECT Id INTO @kill_id FROM INFORMATION_SCHEMA.PROCESSLIST WHERE Info LIKE 'ANALYZE REPLACE DELAYED%' LIMIT 1; KILL QUERY @kill_id ;
       
      query:
      	ANALYZE REPLACE DELAYED INTO _table ( _field ) VALUES ( NULL ) ;

      RQG command line

      perl ./runall-new.pl --threads=3 --duration=300 --queries=100M --grammar=test.yy --engine=MyISAM --basedir1=<your basedir> --vardir1=<your vardir>

      For RQG, I recommend using lp:~elenst/randgen/mariadb-patches

      Attachments

        1. threads1
          32 kB
        2. threads2
          33 kB

        Activity

          @elenst, does the problem occur if the test runs regular INSERT/REPLACE DELAYED instead of ANALYZE ?

          psergei Sergei Petrunia added a comment - @elenst, does the problem occur if the test runs regular INSERT/REPLACE DELAYED instead of ANALYZE ?

          psergey,

          I could not reproduce it without ANALYZE. It does not mean it cannot happen on any other circumstances, it might be that I did not find the right mix.
          If I add ALTER TABLE, I don't need ANALYZE, but then I get MDEV-8889 as mentioned in the description. If I remove ALTER TABLE and just leave REPLACE and KILL, I am not getting the deadlock.

          elenst Elena Stepanova added a comment - psergey , I could not reproduce it without ANALYZE. It does not mean it cannot happen on any other circumstances, it might be that I did not find the right mix. If I add ALTER TABLE, I don't need ANALYZE, but then I get MDEV-8889 as mentioned in the description. If I remove ALTER TABLE and just leave REPLACE and KILL, I am not getting the deadlock.

          When I attempt to run it in the current tree
          I get output like this:

          MariaDB [(none)]> show processlist;
          +----+---------+-----------------+-------+----------------+------+------------------------------+--------------------------------------------------------------------------+----------+
          | Id | User    | Host            | db    | Command        | Time | State                        | Info                                                                     | Progress |
          +----+---------+-----------------+-------+----------------+------+------------------------------+--------------------------------------------------------------------------+----------+
          |  2 | rqg     | localhost:46569 | mysql | Sleep          |  269 |                              | NULL                                                                     |    0.000 |
          |  8 | rqg     | localhost:46575 | test  | Query          |    0 | init                         | KILL QUERY @kill_id                                                      |    0.000 |
          |  9 | rqg     | localhost:46576 | test  | Query          |  266 | waiting for handler insert   | ANALYZE REPLACE DELAYED INTO `BB` ( `col_datetime_key` ) VALUES ( NULL ) |    0.000 |
          | 10 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
          | 11 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
          | 12 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
          | 13 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
          | 14 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
          | 15 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
          | 16 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
          | 17 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
          | 18 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for table level lock |                                                                          |    0.000 |
          | 19 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for table level lock |                                                                          |    0.000 |
          | 20 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
          | 21 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for INSERT           |                                                                          |    0.000 |
          | 22 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for table level lock |                                                                          |    0.000 |
          | 23 | DELAYED | localhost       | test  | Delayed insert | NULL | Waiting for table level lock |                                                                          |    0.000 |
          | 24 | rqg     | localhost:46577 | test  | Killed         |  266 | waiting for handler insert   | ANALYZE REPLACE DELAYED INTO `BB` ( `col_int_key` ) VALUES ( NULL )      |    0.000 |
          | 25 | root    | localhost:46578 | NULL  | Query          |    0 | init                         | show processlist                                                         |    0.000 |
          +----+---------+-----------------+-------+----------------+------+------------------------------+--------------------------------------------------------------------------+----------+
          19 rows in set (0.00 sec)

          psergei Sergei Petrunia added a comment - When I attempt to run it in the current tree I get output like this: MariaDB [(none)]> show processlist; +----+---------+-----------------+-------+----------------+------+------------------------------+--------------------------------------------------------------------------+----------+ | Id | User | Host | db | Command | Time | State | Info | Progress | +----+---------+-----------------+-------+----------------+------+------------------------------+--------------------------------------------------------------------------+----------+ | 2 | rqg | localhost:46569 | mysql | Sleep | 269 | | NULL | 0.000 | | 8 | rqg | localhost:46575 | test | Query | 0 | init | KILL QUERY @kill_id | 0.000 | | 9 | rqg | localhost:46576 | test | Query | 266 | waiting for handler insert | ANALYZE REPLACE DELAYED INTO `BB` ( `col_datetime_key` ) VALUES ( NULL ) | 0.000 | | 10 | DELAYED | localhost | test | Delayed insert | NULL | Waiting for INSERT | | 0.000 | | 11 | DELAYED | localhost | test | Delayed insert | NULL | Waiting for INSERT | | 0.000 | | 12 | DELAYED | localhost | test | Delayed insert | NULL | Waiting for INSERT | | 0.000 | | 13 | DELAYED | localhost | test | Delayed insert | NULL | Waiting for INSERT | | 0.000 | | 14 | DELAYED | localhost | test | Delayed insert | NULL | Waiting for INSERT | | 0.000 | | 15 | DELAYED | localhost | test | Delayed insert | NULL | Waiting for INSERT | | 0.000 | | 16 | DELAYED | localhost | test | Delayed insert | NULL | Waiting for INSERT | | 0.000 | | 17 | DELAYED | localhost | test | Delayed insert | NULL | Waiting for INSERT | | 0.000 | | 18 | DELAYED | localhost | test | Delayed insert | NULL | Waiting for table level lock | | 0.000 | | 19 | DELAYED | localhost | test | Delayed insert | NULL | Waiting for table level lock | | 0.000 | | 20 | DELAYED | localhost | test | Delayed insert | NULL | Waiting for INSERT | | 0.000 | | 21 | DELAYED | localhost | test | Delayed insert | NULL | Waiting for INSERT | | 0.000 | | 22 | DELAYED | localhost | test | Delayed insert | NULL | Waiting for table level lock | | 0.000 | | 23 | DELAYED | localhost | test | Delayed insert | NULL | Waiting for table level lock | | 0.000 | | 24 | rqg | localhost:46577 | test | Killed | 266 | waiting for handler insert | ANALYZE REPLACE DELAYED INTO `BB` ( `col_int_key` ) VALUES ( NULL ) | 0.000 | | 25 | root | localhost:46578 | NULL | Query | 0 | init | show processlist | 0.000 | +----+---------+-----------------+-------+----------------+------+------------------------------+--------------------------------------------------------------------------+----------+ 19 rows in set (0.00 sec)

          However, the server didn't hung:

          # 2015-10-11T14:01:36 [29636] GenTest exited with exit status STATUS_OK (0)
          # 2015-10-11T14:01:36 [29636] Stopping server(s)...
          # 2015-10-11T14:01:36 [29636] Stopping server on port 19300
          # 2015-10-11T14:01:36 [29636] [29636] ./runall-new.pl will exit with exit status STATUS_OK (0)

          psergei Sergei Petrunia added a comment - However, the server didn't hung: # 2015-10-11T14:01:36 [29636] GenTest exited with exit status STATUS_OK (0) # 2015-10-11T14:01:36 [29636] Stopping server(s)... # 2015-10-11T14:01:36 [29636] Stopping server on port 19300 # 2015-10-11T14:01:36 [29636] [29636] ./runall-new.pl will exit with exit status STATUS_OK (0)

          Yes it did. After randgen has finished, mysqld process is left running;

          Thread 11 (Thread 0x7f9bc01bc700 (LWP 29731)):
          #0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
          #1  0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0
          #2  0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101
          #3  0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
              at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294
          #4  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
              at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
          #5  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6000e610) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
          #6  0x00007f9bc1ffce9a in start_thread (arg=0x7f9bc01bc700) at pthread_create.c:308
          #7  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
          #8  0x0000000000000000 in ?? ()
          ...
           
          Thread 10 (Thread 0x7f9bc0173700 (LWP 29732)):
          #0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
          #1  0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0
          #2  0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101
          #3  0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
              at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294
          #4  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
              at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
          #5  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b60019110) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
          #6  0x00007f9bc1ffce9a in start_thread (arg=0x7f9bc0173700) at pthread_create.c:308
          #7  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
          #8  0x0000000000000000 in ?? ()
           
          Thread 9 (Thread 0x7f9bc00e1700 (LWP 29734)):
          #0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
          #1  0x00007f9bc1fff065 in _L_lock_858 () from /lib/x86_64-linux-gnu/libpthread.so.0
          #2  0x00007f9bc1ffeeba in __pthread_mutex_lock (mutex=0x7f9bc4e6aea0) at pthread_mutex_lock.c:61
          #3  0x00007f9bc44597a7 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
              at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:250
          #4  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
              at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
          ---Type <return> to continue, or q <return> to quit---
          #5  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6002ba80) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
          #6  0x00007f9bc1ffce9a in start_thread (arg=0x7f9bc00e1700) at pthread_create.c:308
          #7  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
          #8  0x0000000000000000 in ?? ()
           
          Thread 8 (Thread 0x7f9bc004f700 (LWP 29736)):
          #0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
          #1  0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0
          #2  0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101
          #3  0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
              at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294
          #4  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
              at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
          #5  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6003e600) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
          #6  0x00007f9bc1ffce9a in start_thread (arg=0x7f9bc004f700) at pthread_create.c:308
          #7  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
          #8  0x0000000000000000 in ?? ()
           
          Thread 7 (Thread 0x7f9ba84e9700 (LWP 29739)):
          #0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
          #1  0x00007f9bc1fff065 in _L_lock_858 () from /lib/x86_64-linux-gnu/libpthread.so.0
          #2  0x00007f9bc1ffeeba in __pthread_mutex_lock (mutex=0x7f9bc4e6aea0) at pthread_mutex_lock.c:61
          #3  0x00007f9bc4459889 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
              at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:296
          #4  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
              at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
          #5  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6005a6c0) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
          #6  0x00007f9bc1ffce9a in start_thread (arg=0x7f9ba84e9700) at pthread_create.c:308
          #7  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
          #8  0x0000000000000000 in ?? ()
           
          Thread 6 (Thread 0x7f9ba84a0700 (LWP 29740)):
          #0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
          #1  0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0
          #2  0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101
          #3  0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
              at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294
          #4  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
              at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
          #5  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b60063f40) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
          #6  0x00007f9bc1ffce9a in start_thread (arg=0x7f9ba84a0700) at pthread_create.c:308
          #7  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
          #8  0x0000000000000000 in ?? ()
           
          Thread 5 (Thread 0x7f9ba8457700 (LWP 29741)):
          #0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
          ---Type <return> to continue, or q <return> to quit---
          #1  0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0
          #2  0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101
          #3  0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
              at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294
          #4  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
              at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
          #5  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6006d7c0) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
          #6  0x00007f9bc1ffce9a in start_thread (arg=0x7f9ba8457700) at pthread_create.c:308
          #7  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
          #8  0x0000000000000000 in ?? ()
           
          Thread 4 (Thread 0x7f9ba83c5700 (LWP 29743)):
          #0  __pthread_mutex_unlock (mutex=0x7f9bc56df220) at pthread_mutex_unlock.c:289
          #1  0x00007f9bc4459838 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
              at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:269
          #2  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
              at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
          #3  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b600810c0) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
          #4  0x00007f9bc1ffce9a in start_thread (arg=0x7f9ba83c5700) at pthread_create.c:308
          #5  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
          #6  0x0000000000000000 in ?? ()
           
          Thread 3 (Thread 0x7f9ba837c700 (LWP 29744)):
          #0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
          #1  0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0
          #2  0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101
          #3  0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883)
              at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294
          #4  0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883)
              at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689
          #5  0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6008a9e0) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883
          #6  0x00007f9bc1ffce9a in start_thread (arg=0x7f9ba837c700) at pthread_create.c:308
          #7  0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
          #8  0x0000000000000000 in ?? ()

          psergei Sergei Petrunia added a comment - Yes it did. After randgen has finished, mysqld process is left running; Thread 11 (Thread 0x7f9bc01bc700 (LWP 29731)): #0 __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132 #1 0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0 #2 0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101 #3 0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883) at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294 #4 0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883) at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689 #5 0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6000e610) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883 #6 0x00007f9bc1ffce9a in start_thread (arg=0x7f9bc01bc700) at pthread_create.c:308 #7 0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #8 0x0000000000000000 in ?? () ...   Thread 10 (Thread 0x7f9bc0173700 (LWP 29732)): #0 __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132 #1 0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0 #2 0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101 #3 0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883) at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294 #4 0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883) at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689 #5 0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b60019110) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883 #6 0x00007f9bc1ffce9a in start_thread (arg=0x7f9bc0173700) at pthread_create.c:308 #7 0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #8 0x0000000000000000 in ?? ()   Thread 9 (Thread 0x7f9bc00e1700 (LWP 29734)): #0 __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132 #1 0x00007f9bc1fff065 in _L_lock_858 () from /lib/x86_64-linux-gnu/libpthread.so.0 #2 0x00007f9bc1ffeeba in __pthread_mutex_lock (mutex=0x7f9bc4e6aea0) at pthread_mutex_lock.c:61 #3 0x00007f9bc44597a7 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883) at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:250 #4 0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883) at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689 ---Type <return> to continue, or q <return> to quit--- #5 0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6002ba80) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883 #6 0x00007f9bc1ffce9a in start_thread (arg=0x7f9bc00e1700) at pthread_create.c:308 #7 0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #8 0x0000000000000000 in ?? ()   Thread 8 (Thread 0x7f9bc004f700 (LWP 29736)): #0 __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132 #1 0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0 #2 0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101 #3 0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883) at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294 #4 0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883) at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689 #5 0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6003e600) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883 #6 0x00007f9bc1ffce9a in start_thread (arg=0x7f9bc004f700) at pthread_create.c:308 #7 0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #8 0x0000000000000000 in ?? ()   Thread 7 (Thread 0x7f9ba84e9700 (LWP 29739)): #0 __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132 #1 0x00007f9bc1fff065 in _L_lock_858 () from /lib/x86_64-linux-gnu/libpthread.so.0 #2 0x00007f9bc1ffeeba in __pthread_mutex_lock (mutex=0x7f9bc4e6aea0) at pthread_mutex_lock.c:61 #3 0x00007f9bc4459889 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883) at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:296 #4 0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883) at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689 #5 0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6005a6c0) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883 #6 0x00007f9bc1ffce9a in start_thread (arg=0x7f9ba84e9700) at pthread_create.c:308 #7 0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #8 0x0000000000000000 in ?? ()   Thread 6 (Thread 0x7f9ba84a0700 (LWP 29740)): #0 __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132 #1 0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0 #2 0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101 #3 0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883) at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294 #4 0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883) at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689 #5 0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b60063f40) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883 #6 0x00007f9bc1ffce9a in start_thread (arg=0x7f9ba84a0700) at pthread_create.c:308 #7 0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #8 0x0000000000000000 in ?? ()   Thread 5 (Thread 0x7f9ba8457700 (LWP 29741)): #0 __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132 ---Type <return> to continue, or q <return> to quit--- #1 0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0 #2 0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101 #3 0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883) at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294 #4 0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883) at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689 #5 0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6006d7c0) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883 #6 0x00007f9bc1ffce9a in start_thread (arg=0x7f9ba8457700) at pthread_create.c:308 #7 0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #8 0x0000000000000000 in ?? ()   Thread 4 (Thread 0x7f9ba83c5700 (LWP 29743)): #0 __pthread_mutex_unlock (mutex=0x7f9bc56df220) at pthread_mutex_unlock.c:289 #1 0x00007f9bc4459838 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883) at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:269 #2 0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883) at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689 #3 0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b600810c0) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883 #4 0x00007f9bc1ffce9a in start_thread (arg=0x7f9ba83c5700) at pthread_create.c:308 #5 0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #6 0x0000000000000000 in ?? ()   Thread 3 (Thread 0x7f9ba837c700 (LWP 29744)): #0 __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132 #1 0x00007f9bc1fff09b in _L_lock_1006 () from /lib/x86_64-linux-gnu/libpthread.so.0 #2 0x00007f9bc1fff01c in __pthread_mutex_lock (mutex=0x7f9bc4e6aec8) at pthread_mutex_lock.c:101 #3 0x00007f9bc4459874 in safe_mutex_lock (mp=0x7f9bc4e6aea0, my_flags=0, file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", line=2883) at /home/psergey/dev-git/10.1-dbg3/mysys/thr_mutex.c:294 #4 0x00007f9bc3bb0523 in inline_mysql_mutex_lock (that=0x7f9bc4e6aea0, src_file=0x7f9bc4514a20 "/home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc", src_line=2883) at /home/psergey/dev-git/10.1-dbg3/include/mysql/psi/mysql_thread.h:689 #5 0x00007f9bc3bb701c in handle_delayed_insert (arg=0x7f9b6008a9e0) at /home/psergey/dev-git/10.1-dbg3/sql/sql_insert.cc:2883 #6 0x00007f9bc1ffce9a in start_thread (arg=0x7f9ba837c700) at pthread_create.c:308 #7 0x00007f9bc172d3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #8 0x0000000000000000 in ?? ()

          People

            psergei Sergei Petrunia
            elenst Elena Stepanova
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.