160530 20:46:51 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see http://kb.askmonty.org/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.0.23-MariaDB-log key_buffer_size=16777216 read_buffer_size=262144 max_used_connections=681 max_threads=1502 thread_count=116 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1199576 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x0x7fb13618d008 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7fb165baeea0 thread_stack 0x48000 InnoDB: Warning: a long semaphore wait: --Thread 140399960110848 has waited at btr0cur.cc line 257 for 241.00 seconds the semaphore: S-lock on RW-latch at 0x7fbad66d51c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: 0 Last time read locked in file btr0cur.cc line 257 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/btr/btr0cur.cc line 257 InnoDB: Warning: semaphore wait: --Thread 140399960110848 has waited at btr0cur.cc line 257 for 241.00 seconds the semaphore: S-lock on RW-latch at 0x7fbad66d51c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: 0 Last time read locked in file btr0cur.cc line 257 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/btr/btr0cur.cc line 257 InnoDB: Warning: semaphore wait: --Thread 140434925299456 has waited at row0row.cc line 815 for 229.00 seconds the semaphore: X-lock on RW-latch at 0x7fb1e1ae68c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: fffffffffff00000 Last time read locked in file row0sel.cc line 3047 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/ibuf/ibuf0ibuf.cc line 4787 InnoDB: Warning: semaphore wait: --Thread 140399993681664 has waited at buf0flu.cc line 1079 for 108.00 seconds the semaphore: S-lock on RW-latch at 0x7fbad66d51c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: 0 Last time read locked in file btr0cur.cc line 257 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/btr/btr0cur.cc line 257 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 ===================================== 2016-05-30 20:52:13 7fb16a3fc700 INNODB MONITOR OUTPUT ===================================== Per second averages calculated from the last 41 seconds ----------------- BACKGROUND THREAD ----------------- srv_master_thread loops: 261872 srv_active, 0 srv_shutdown, 0 srv_idle srv_master_thread log flush and writes: 261871 ---------- SEMAPHORES ---------- OS WAIT ARRAY INFO: reservation count 22138390 --Thread 140399960110848 has waited at btr0cur.cc line 257 for 256.00 seconds the semaphore: S-lock on RW-latch at 0x7fbad66d51c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: 0 Last time read locked in file btr0cur.cc line 257 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/btr/btr0cur.cc line 257 --Thread 140434925299456 has waited at row0row.cc line 815 for 244.00 seconds the semaphore: X-lock on RW-latch at 0x7fb1e1ae68c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: fffffffffff00000 Last time read locked in file row0sel.cc line 3047 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/ibuf/ibuf0ibuf.cc line 4787 --Thread 140399993681664 has waited at buf0flu.cc line 1079 for 123.00 seconds the semaphore: S-lock on RW-latch at 0x7fbad66d51c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: 0 Last time read locked in file btr0cur.cc line 257 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/btr/btr0cur.cc line 257 OS WAIT ARRAY INFO: signal count 18884269 Mutex spin waits 117399520, rounds 477469040, OS waits 5135533 RW-shared spins 12585000, rounds 376779053, OS waits 11436447 RW-excl spins 5427878, rounds 208993635, OS waits 5247311 Spin rounds per wait: 4.07 mutex, 29.94 RW-shared, 38.50 RW-excl ------------------------ LATEST DETECTED DEADLOCK ------------------------ 2016-05-30 20:43:47 7fb165e40700 *** (1) TRANSACTION: TRANSACTION 62420262592, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 4 lock struct(s), heap size 1184, 2 row lock(s), undo log entries 1 MySQL thread id 54783999, OS thread handle 0x7fb165f64700, query id 807293989 xxx xxx xxx update INSERT INTO xxx (xxx,xxx,xxx,xxxx,xxxx,xxxx,xxx) VALUES (2, '2016-05-30 20:43:47', '2016-05-30 20:43:47', 91965929, '20285', NULL, 89058268) *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 69 page no 4302778 n bits 304 index `xxx` of table `xxx`.`xxx` trx table locks 2 total table locks 4 trx id 62420262592 lock mode S waiting lock hold time 0 wait time before grant 0 *** (2) TRANSACTION: TRANSACTION 62420262543, ACTIVE 0 sec starting index read mysql tables in use 1, locked 1 15 lock struct(s), heap size 2936, 12 row lock(s), undo log entries 7 MySQL thread id 54783689, OS thread handle 0x7fb165e40700, query id 807294065 xxx xxx xxx updating UPDATE xxx SET xxx = 332959376, version = version + 1 WHERE id = 91965929 AND version = 1 *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 69 page no 4302778 n bits 304 index `xxx` of table `xxx`.`xxx` trx table locks 7 total table locks 4 trx id 62420262543 lock_mode X locks rec but not gap lock hold time 0 wait time before grant 0 *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 604 page no 441308 n bits 112 index `PRIMARY` of table `xxx`.`xxx` trx table locks 7 total table locks 8 trx id 62420262543 lock_mode X locks rec but not gap waiting lock hold time 0 wait time before grant 0 *** WE ROLL BACK TRANSACTION (1) ------------ TRANSACTIONS ------------ Trx id counter 62420752850 Purge done for trx's n:o < 62420738968 undo n:o < 1 state: running but idle History list length 387 LIST OF TRANSACTIONS FOR EACH SESSION: ---TRANSACTION 62420737013, not started MySQL thread id 54825693, OS thread handle 0x7fb0d7edd700, query id 807980072 xxx xxx xxx cleaning up ---TRANSACTION 62420737547, not started MySQL thread id 54825671, OS thread handle 0x7fb16464c700, query id 807980378 xxx xxx xxx Opening tables SELECT xxx ---TRANSACTION 62420735469, not started MySQL thread id 54825632, OS thread handle 0x7fb98dd19700, query id 807980071 xxx xxx xxx After opening tables SELECT xxx ---TRANSACTION 62420737380, not started mysql tables in use 1, locked 0 MySQL thread id 54825613, OS thread handle 0x7fb165a42700, query id 807980173 xxx xxx xxx Sorting result SELECT xxx ---TRANSACTION 62420737133, not started MySQL thread id 54825556, OS thread handle 0x7fb165baf700, query id 807980062 xxx xxx xxx cleaning up ---TRANSACTION 62420736171, not started mysql tables in use 1, locked 0 MySQL thread id 54825523, OS thread handle 0x7fbad64b8700, query id 807980360 xxx xxx xxx checking permissions SELECT xxx ---TRANSACTION 62420738919, not started estimating records in index range mysql tables in use 2, locked 0 MySQL thread id 54825499, OS thread handle 0x7fbc178e8700, query id 807981971 xxx xxx xxx statistics SELECT xxx ---TRANSACTION 62420742396, not started MySQL thread id 54825403, OS thread handle 0x7fb1dac87700, query id 807985528 xxx xxx xxx updating status SELECT xxx ---TRANSACTION 62420740226, not started MySQL thread id 54825392, OS thread handle 0x7fb19e164700, query id 807983597 xxx xxx xxx cleaning up ---TRANSACTION 62420742862, not started MySQL thread id 54825340, OS thread handle 0x7fb165b66700, query id 807986341 xxx xxx xxx cleaning up ---TRANSACTION 62420739133, not started MySQL thread id 54825337, OS thread handle 0x7fb165122700, query id 807982068 xxx xxx xxx After opening tables SELECT xxx ---TRANSACTION 62420739180, not started MySQL thread id 54825143, OS thread handle 0x7fb323487700, query id 807982121 xxx xxx xxx Opening tables SELECT xxx ---TRANSACTION 62420739027, not started MySQL thread id 54825104, OS thread handle 0x7fb1dadab700, query id 807982378 xxx xxx xxx cleaning up ---TRANSACTION 62420752755, not started MySQL thread id 54269856, OS thread handle 0x7fb163b2d700, query id 807997628 xxx xxx xxx cleaning up ---TRANSACTION 62420752822, not started MySQL thread id 50507965, OS thread handle 0x7fb0d8249700, query id 807997695 xxx xxx xxx cleaning up ---TRANSACTION 62420752849, not started MySQL thread id 50507970, OS thread handle 0x7fb1dad19700, query id 807997722 xxx xxx xxx cleaning up ---TRANSACTION 62420743915, not started MySQL thread id 50507956, OS thread handle 0x7fbc18c59700, query id 807987381 xxx xxx xxx init COMMIT ---TRANSACTION 62420737626, not started mysql tables in use 1, locked 1 MySQL thread id 50507951, OS thread handle 0x7fb3235ab700, query id 807980491 xxx xxx xxx init UPDATE xxx ---TRANSACTION 62420737406, not started MySQL thread id 50507950, OS thread handle 0x7fb1652d8700, query id 807980236 xxx xxx xxx After opening tables UPDATE xxx ---TRANSACTION 62372007607, not started MySQL thread id 46391880, OS thread handle 0x7fbc1797a700, query id 738982310 xxx xxx xxx cleaning up ---TRANSACTION 61903232003, not started MySQL thread id 1, OS thread handle 0x7fbc1b0e7700, query id 0 Waiting for background binlog tasks ---TRANSACTION 62420751099, ACTIVE 187 sec mysql tables in use 1, locked 1 2 lock struct(s), heap size 360, 1 row lock(s), undo log entries 1 MySQL thread id 534334, OS thread handle 0x7fb1662d0700, query id 807995972 localhost monitoring updating UPDATE xxx Trx #rec lock waits 0 #table lock waits 0 Trx total rec lock wait time 0 SEC Trx total table lock wait time 0 SEC ---TRANSACTION 62420750465, ACTIVE 244 sec updating or deleting mysql tables in use 1, locked 1 7 lock struct(s), heap size 1184, 16 row lock(s), undo log entries 16 MySQL thread id 54814524, OS thread handle 0x7fb98dd62700, query id 807995280 xxx xxx xxx updating UPDATE xxx Trx #rec lock waits 0 #table lock waits 0 Trx total rec lock wait time 0 SEC Trx total table lock wait time 0 SEC ---TRANSACTION 62420744005, ACTIVE 318 sec inserting mysql tables in use 1, locked 1 3 lock struct(s), heap size 1184, 1 row lock(s), undo log entries 3 MySQL thread id 50507955, OS thread handle 0x7fb165644700, query id 807987529 xxx xxx xxx update INSERT xxx Trx #rec lock waits 0 #table lock waits 0 Trx total rec lock wait time 0 SEC Trx total table lock wait time 0 SEC ---TRANSACTION 62420743925, ACTIVE 319 sec mysql tables in use 1, locked 1 2 lock struct(s), heap size 360, 1 row lock(s), undo log entries 1 MySQL thread id 50507958, OS thread handle 0x7fb163a9b700, query id 807987393 xxx xxx xxx updating UPDATE xxx Trx #rec lock waits 0 #table lock waits 0 Trx total rec lock wait time 0 SEC Trx total table lock wait time 0 SEC ---TRANSACTION 62420743823, ACTIVE 319 sec 2 lock struct(s), heap size 360, 1 row lock(s), undo log entries 1 MySQL thread id 50507953, OS thread handle 0x7fb16404f700, query id 807987203 xxx xxx xxx closing tables SELECT xxx Trx read view will not see trx with id >= 62420743824, sees < 62420741807 Trx #rec lock waits 0 #table lock waits 0 Trx total rec lock wait time 0 SEC Trx total table lock wait time 0 SEC ---TRANSACTION 62420742064, ACTIVE 321 sec 2 lock struct(s), heap size 360, 1 row lock(s), undo log entries 1 MySQL thread id 50507954, OS thread handle 0x7fb6fcdab700, query id 807985179 xxx xxx xxx updating status SELECT xxx Trx read view will not see trx with id >= 62420742070, sees < 62420741807 Trx #rec lock waits 0 #table lock waits 0 Trx total rec lock wait time 0 SEC Trx total table lock wait time 0 SEC ---TRANSACTION 62420741807, ACTIVE 321 sec 2 lock struct(s), heap size 360, 1 row lock(s), undo log entries 1 MySQL thread id 50507949, OS thread handle 0x7fb1657b1700, query id 807984907 xxx xxx xxx preparing SELECT xxx Trx read view will not see trx with id >= 62420741811, sees < 62420741811 Trx #rec lock waits 0 #table lock waits 0 Trx total rec lock wait time 0 SEC Trx total table lock wait time 0 SEC -------- FILE I/O -------- I/O thread 0 state: waiting for completed aio requests (insert buffer thread) I/O thread 1 state: waiting for completed aio requests (log thread) I/O thread 2 state: waiting for completed aio requests (read thread) I/O thread 3 state: complete io for buf page (read thread) I/O thread 4 state: waiting for completed aio requests (read thread) I/O thread 5 state: waiting for completed aio requests (read thread) I/O thread 6 state: waiting for completed aio requests (write thread) I/O thread 7 state: waiting for completed aio requests (write thread) I/O thread 8 state: waiting for completed aio requests (write thread) I/O thread 9 state: waiting for completed aio requests (write thread) Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] , ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 56923586 OS file reads, 103641674 OS file writes, 53351414 OS fsyncs 0.00 reads/s, 0 avg bytes/read, 0.00 writes/s, 0.00 fsyncs/s ------------------------------------- INSERT BUFFER AND ADAPTIVE HASH INDEX ------------------------------------- Ibuf: size 1, free list len 393220, seg size 393222, 4281741 merges merged operations: insert 1658324, delete mark 23198357, delete 3402074 discarded operations: insert 0, delete mark 0, delete 0 8.78 hash searches/s, 0.39 non-hash searches/s --- LOG --- Log sequence number 5261230715186 Log flushed up to 5261230715146 Pages flushed up to 5261228347163 Last checkpoint at 5261209353612 Max checkpoint age 80826164 Checkpoint age target 78300347 Modified age 2368023 Checkpoint age 21361574 0 pending log writes, 0 pending chkp writes 45346842 log i/o's done, 0.00 log i/o's/second ---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 42926080000; in additional pool allocated 0 Total memory allocated by read views 24936 Internal hash tables (constant factor + variable factor) Adaptive hash index 3623025152 (664054168 + 2958970984) Page hash 5188856 (buffer pool 0 only) Dictionary cache 167525950 (166015184 + 1510766) File system 901648 (812272 + 89376) Lock system 103772664 (103759496 + 13168) Recovery system 0 (0 + 0) Dictionary memory allocated 1510766 InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 140399960110848 has waited at btr0cur.cc line 257 for 272.00 seconds the semaphore: S-lock on RW-latch at 0x7fbad66d51c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: 0 Last time read locked in file btr0cur.cc line 257 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/btr/btr0cur.cc line 257 InnoDB: Warning: a long semaphore wait: --Thread 140434925299456 has waited at row0row.cc line 815 for 260.00 seconds the semaphore: X-lock on RW-latch at 0x7fb1e1ae68c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: fffffffffff00000 Last time read locked in file row0sel.cc line 3047 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/ibuf/ibuf0ibuf.cc line 4787 InnoDB: Warning: semaphore wait: --Thread 140399960110848 has waited at btr0cur.cc line 257 for 272.00 seconds the semaphore: S-lock on RW-latch at 0x7fbad66d51c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: 0 Last time read locked in file btr0cur.cc line 257 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/btr/btr0cur.cc line 257 InnoDB: Warning: semaphore wait: --Thread 140434925299456 has waited at row0row.cc line 815 for 260.00 seconds the semaphore: X-lock on RW-latch at 0x7fb1e1ae68c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: fffffffffff00000 Last time read locked in file row0sel.cc line 3047 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/ibuf/ibuf0ibuf.cc line 4787 InnoDB: Warning: semaphore wait: --Thread 140399993681664 has waited at buf0flu.cc line 1079 for 139.00 seconds the semaphore: S-lock on RW-latch at 0x7fbad66d51c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: 0 Last time read locked in file btr0cur.cc line 257 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/btr/btr0cur.cc line 257 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0 InnoDB: ###### Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 140399960110848 has waited at btr0cur.cc line 257 for 303.00 seconds the semaphore: S-lock on RW-latch at 0x7fbad66d51c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: 0 Last time read locked in file btr0cur.cc line 257 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/btr/btr0cur.cc line 257 InnoDB: Warning: a long semaphore wait: --Thread 140434925299456 has waited at row0row.cc line 815 for 291.00 seconds the semaphore: X-lock on RW-latch at 0x7fb1e1ae68c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: fffffffffff00000 Last time read locked in file row0sel.cc line 3047 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/ibuf/ibuf0ibuf.cc line 4787 InnoDB: Warning: semaphore wait: --Thread 140399960110848 has waited at btr0cur.cc line 257 for 303.00 seconds the semaphore: S-lock on RW-latch at 0x7fbad66d51c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: 0 Last time read locked in file btr0cur.cc line 257 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/btr/btr0cur.cc line 257 InnoDB: Warning: semaphore wait: --Thread 140434925299456 has waited at row0row.cc line 815 for 291.00 seconds the semaphore: X-lock on RW-latch at 0x7fb1e1ae68c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: fffffffffff00000 Last time read locked in file row0sel.cc line 3047 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/ibuf/ibuf0ibuf.cc line 4787 InnoDB: Warning: semaphore wait: --Thread 140399993681664 has waited at buf0flu.cc line 1079 for 170.00 seconds the semaphore: S-lock on RW-latch at 0x7fbad66d51c0 '&block->lock' a writer (thread id 140400899639040) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: 0 Last time read locked in file btr0cur.cc line 257 Last time write locked in file /var/tmp/portage/dev-db/mariadb-10.0.23/work/mysql/storage/xtradb/btr/btr0cur.cc line 257 InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info: InnoDB: Pending preads 0, pwrites 0