[MDEV-21078] MariaDB was crashed with long semaphore wait Created: 2019-11-19  Updated: 2019-12-30  Resolved: 2019-12-30

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.2.15
Fix Version/s: N/A

Type: Bug Priority: Critical
Reporter: Prabha Assignee: Marko Mäkelä
Resolution: Incomplete Votes: 0
Labels: need_feedback


 Description   

MariaDB Was crashed and recovered after 15 mins. Not sure what went wrong, can you please help on this. Below is the log entries in mariadb.log

=====================================
2019-11-18 08:54:07 0x7f79358cb700 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 20 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 40381 srv_active, 0 srv_shutdown, 82038 srv_idle
srv_master_thread log flush and writes: 122417
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 49112997
--Thread 140158456289024 has waited at row0row.cc line 1072 for 953.00 seconds the semaphore:
X-lock on RW-latch at 0x7f9d34ba0880 created in file buf0buf.cc line 1471
a writer (thread id 0) has reserved it in mode  SX
number of readers 0, waiters flag 1, lock_word: 10000000
Last time read locked in file row0ins.cc line 2069
Last time write locked in file buf0flu.cc line 1226
--Thread 140158028384000 has waited at row0undo.cc line 292 for 945.00 seconds the semaphore:
S-lock on RW-latch at 0x558327e61880 created in file dict0dict.cc line 1108
a writer (thread id 140158557792000) has reserved it in mode  wait exclusive
number of readers 4, waiters flag 1, lock_word: fffffffffffffffc
Last time read locked in file row0undo.cc line 292
Last time write locked in file ha_innodb.cc line 12985
--Thread 140158464681728 has waited at row0row.cc line 1072 for 953.00 seconds the semaphore:
X-lock on RW-latch at 0x7f9d34ba0880 created in file buf0buf.cc line 1471
a writer (thread id 0) has reserved it in mode  SX
number of readers 0, waiters flag 1, lock_word: 10000000
Last time read locked in file row0ins.cc line 2069
Last time write locked in file buf0flu.cc line 1226
--Thread 140158030808832 has waited at row0undo.cc line 292 for 83.00 seconds the semaphore:
S-lock on RW-latch at 0x558327e61880 created in file dict0dict.cc line 1108
a writer (thread id 140158557792000) has reserved it in mode  wait exclusive
number of readers 4, waiters flag 1, lock_word: fffffffffffffffc
Last time read locked in file row0undo.cc line 292
Last time write locked in file ha_innodb.cc line 12985
--Thread 140158045054720 has waited at row0row.cc line 1072 for 953.00 seconds the semaphore:
S-lock on RW-latch at 0x7f8619dd8838 created in file buf0buf.cc line 1471
a writer (thread id 140158447896320) has reserved it in mode  wait exclusive
number of readers 2, waiters flag 1, lock_word: fffffffffffffffe
Last time read locked in file row0row.cc line 1072
Last time write locked in file mtr0mtr.ic line 147
--Thread 140158473074432 has waited at btr0cur.cc line 978 for 953.00 seconds the semaphore:
SX-lock on RW-latch at 0x7f75300732d0 created in file dict0dict.cc line 2612
a writer (thread id 140158447896320) has reserved it in mode  SX
number of readers 3, waiters flag 1, lock_word: ffffffd
Last time read locked in file btr0cur.cc line 1010
Last time write locked in file btr0cur.cc line 978
--Thread 140158035052288 has waited at row0undo.cc line 292 for 944.00 seconds the semaphore:
S-lock on RW-latch at 0x558327e61880 created in file dict0dict.cc line 1108
a writer (thread id 140158557792000) has reserved it in mode  wait exclusive
number of readers 4, waiters flag 1, lock_word: fffffffffffffffc
Last time read locked in file row0undo.cc line 292
Last time write locked in file ha_innodb.cc line 12985
--Thread 140158057481984 has waited at row0undo.cc line 292 for 945.00 seconds the semaphore:
S-lock on RW-latch at 0x558327e61880 created in file dict0dict.cc line 1108
a writer (thread id 140158557792000) has reserved it in mode  wait exclusive
number of readers 4, waiters flag 1, lock_word: fffffffffffffffc
Last time read locked in file row0undo.cc line 292
Last time write locked in file ha_innodb.cc line 12985
--Thread 140158557792000 has waited at dict0stats.cc line 2456 for 945.00 seconds the semaphore:
X-lock (wait_ex) on RW-latch at 0x558327e61880 created in file dict0dict.cc line 1108
a writer (thread id 140158557792000) has reserved it in mode  wait exclusive
number of readers 4, waiters flag 1, lock_word: fffffffffffffffc
Last time read locked in file row0undo.cc line 292
Last time write locked in file ha_innodb.cc line 12985
--Thread 140158447896320 has waited at mtr0mtr.ic line 147 for 953.00 seconds the semaphore:
X-lock (wait_ex) on RW-latch at 0x7f8619dd8838 created in file buf0buf.cc line 1471
a writer (thread id 140158447896320) has reserved it in mode  wait exclusive
number of readers 2, waiters flag 1, lock_word: fffffffffffffffe
Last time read locked in file row0row.cc line 1072
Last time write locked in file mtr0mtr.ic line 147
--Thread 140158037780224 has waited at dict0stats.cc line 2456 for 723.00 seconds the semaphore:
X-lock on RW-latch at 0x558327e61880 created in file dict0dict.cc line 1108
a writer (thread id 140158557792000) has reserved it in mode  wait exclusive
number of readers 4, waiters flag 1, lock_word: fffffffffffffffc
Last time read locked in file row0undo.cc line 292
Last time write locked in file ha_innodb.cc line 12985
OS WAIT ARRAY INFO: signal count 52624235
RW-shared spins 0, rounds 69038357, OS waits 17012133
RW-excl spins 0, rounds 472487789, OS waits 7814216
RW-sx spins 30092370, rounds 571255325, OS waits 10951222
Spin rounds per wait: 69038357.00 RW-shared, 472487789.00 RW-excl, 18.98 RW-sx
------------
TRANSACTIONS
------------
Trx id counter 67166267102
Purge done for trx's n:o < 67078595342 undo n:o < 0 state: running
History list length 6307454
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 421795933694472, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421795933682112, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421795933699416, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421795933698592, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421795933696944, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421795933691176, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421795933690352, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421795933682936, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421795933688704, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421795933687880, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421795933687056, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421795933686232, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421795933683760, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421795933695296, not started inserting
mysql tables in use 1, locked 1
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421795933684584, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421795933680464, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 67166267101, ACTIVE 0 sec inserting
mysql tables in use 1, locked 1
0 lock struct(s), heap size 1136, 0 row lock(s)
MySQL thread id 21904, OS thread handle 140158051723008, query id 388751724  update
replace into (`campaign`,`source`,`ad`,`page`,`landing_page`,`date`,`website_visits`,`conv_my_stc_transactions`,`conv_my_stc_revenue`,`conv_form_fills`,`conv_coupon_views`,`datasource`) VALUES(?,?,?,?,?,?,?,?,?,?,?,?)
---TRANSACTION 67164985040, ACTIVE 83 sec inserting
mysql tables in use 1, locked 1
ROLLING BACK 3 lock struct(s), heap size 1136, 2 row lock(s)
MySQL thread id 25756, OS thread handle 140158030808832, query id 388110553  update
replace into  (`ad_id`,`ad_name`,`adset_id`,`adset_name`,`campaign_id`,`campaign_name`,`date_start`,`date_stop`,`account_id`,`rs_score`,`rs_status`,`ad_reach`,`ad_recallers`) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?)
---TRANSACTION 67151659109, ACTIVE 944 sec inserting
mysql tables in use 1, locked 1
ROLLING BACK 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 25749, OS thread handle 140158035052288, query id 381446338  update
replace into (`clicks`,`impressions`,`spend`,`reach`,`objective`,`frequency`,`campaign_name`,`campaign_id`,`ad_id`,`ad_name`,`adset_id`,`adset_name`,`date_start`,`date_stop`,`age`,`gender`,`account_id`,`video_p25_watched`,`video_p50_watched`,`video_p75_watched`,`video_p95_watched`,`video_p100_watched`,`video_avg_watched`,`link_clicks`,`estimated_ad_recall`) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
---TRANSACTION 67151642255, ACTIVE 945 sec inserting
mysql tables in use 1, locked 1
ROLLING BACK 2 lock struct(s), heap size 1136, 6 row lock(s)
MySQL thread id 21920, OS thread handle 140158028384000, query id 381437903  update
replace into  (`casenumber`,`saledate`,`calltype`,`locationcode`,`locname`,`informanthomephone`,`informantworkphone`,`informantmobilephone`,`totalprofessionalservices`,`casket`,`outerburialcontainer`,`totalservicesandmerchandise`,`import_file_name`) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?)
---TRANSACTION 67151642012, ACTIVE 945 sec inserting
mysql tables in use 1, locked 1
ROLLING BACK 3 lock struct(s), heap size 1136, 48 row lock(s)
MySQL thread id 25748, OS thread handle 140158057481984, query id 381437782  update
replace into (`currency`,`campaign_name`,`channel`,`bi_brand_non_brand`,`date`,`impressions`,`clicks`,`cost`,`ecomm_sale_ks_pixel`,`formleads`,`callrail_calls`,`click_to_call`,`account_name`,`network`) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?)
---TRANSACTION 67151469362, ACTIVE 953 sec updating or deleting
mysql tables in use 1, locked 1
3 lock struct(s), heap size 1136, 8 row lock(s), undo log entries 2
MySQL thread id 25736, OS thread handle 140158045054720, query id 381351448  update
replace into (`campaign`,`date`,`ad_group`,`creative`,`ad_server_creative_placement_id`,`advertiser_cost__usd_`,`impressions`,`clicks`,`player_completed_views`,`player_starts`,`all_last_click___view_conversions`) VALUES(?,?,?,?,?,?,?,?,?,?,?)
--------
FILE I/O
--------
I/O thread 0 state: waiting for completed aio requests (insert buffer thread)
I/O thread 1 state: waiting for completed aio requests (log thread)
I/O thread 2 state: waiting for completed aio requests (read thread)
I/O thread 3 state: waiting for completed aio requests (read thread)
I/O thread 4 state: waiting for completed aio requests (read thread)
I/O thread 5 state: waiting for completed aio requests (read thread)
I/O thread 6 state: waiting for completed aio requests (read thread)
I/O thread 7 state: waiting for completed aio requests (read thread)
I/O thread 8 state: waiting for completed aio requests (read thread)
I/O thread 9 state: waiting for completed aio requests (read thread)
I/O thread 10 state: waiting for completed aio requests (read thread)
I/O thread 11 state: waiting for completed aio requests (read thread)
I/O thread 12 state: waiting for completed aio requests (read thread)
I/O thread 13 state: waiting for completed aio requests (read thread)
I/O thread 14 state: waiting for completed aio requests (read thread)
I/O thread 15 state: waiting for completed aio requests (read thread)
I/O thread 16 state: waiting for completed aio requests (read thread)
I/O thread 17 state: waiting for completed aio requests (read thread)
I/O thread 18 state: waiting for completed aio requests (read thread)
I/O thread 19 state: waiting for completed aio requests (read thread)
I/O thread 20 state: waiting for completed aio requests (read thread)
I/O thread 21 state: waiting for completed aio requests (read thread)
I/O thread 22 state: waiting for completed aio requests (read thread)
I/O thread 23 state: waiting for completed aio requests (read thread)
I/O thread 24 state: waiting for completed aio requests (read thread)
I/O thread 25 state: waiting for completed aio requests (read thread)
I/O thread 26 state: waiting for completed aio requests (read thread)
I/O thread 27 state: waiting for completed aio requests (read thread)
I/O thread 28 state: waiting for completed aio requests (read thread)
I/O thread 29 state: waiting for completed aio requests (read thread)
I/O thread 30 state: waiting for completed aio requests (read thread)
I/O thread 31 state: waiting for completed aio requests (read thread)
I/O thread 32 state: waiting for completed aio requests (read thread)
I/O thread 33 state: waiting for completed aio requests (read thread)
I/O thread 34 state: waiting for completed aio requests (write thread)
I/O thread 35 state: waiting for completed aio requests (write thread)
I/O thread 36 state: waiting for completed aio requests (write thread)
I/O thread 37 state: waiting for completed aio requests (write thread)
I/O thread 38 state: waiting for completed aio requests (write thread)
I/O thread 39 state: waiting for completed aio requests (write thread)
I/O thread 40 state: waiting for completed aio requests (write thread)
I/O thread 41 state: waiting for completed aio requests (write thread)
I/O thread 42 state: waiting for completed aio requests (write thread)
I/O thread 43 state: waiting for completed aio requests (write thread)
I/O thread 44 state: waiting for completed aio requests (write thread)
I/O thread 45 state: waiting for completed aio requests (write thread)
I/O thread 46 state: waiting for completed aio requests (write thread)
I/O thread 47 state: waiting for completed aio requests (write thread)
I/O thread 48 state: waiting for completed aio requests (write thread)
I/O thread 49 state: waiting for completed aio requests (write thread)
I/O thread 50 state: waiting for completed aio requests (write thread)
I/O thread 51 state: waiting for completed aio requests (write thread)
I/O thread 52 state: waiting for completed aio requests (write thread)
I/O thread 53 state: waiting for completed aio requests (write thread)
I/O thread 54 state: waiting for completed aio requests (write thread)
I/O thread 55 state: waiting for completed aio requests (write thread)
I/O thread 56 state: waiting for completed aio requests (write thread)
I/O thread 57 state: waiting for completed aio requests (write thread)
I/O thread 58 state: waiting for completed aio requests (write thread)
I/O thread 59 state: waiting for completed aio requests (write thread)
I/O thread 60 state: waiting for completed aio requests (write thread)
I/O thread 61 state: waiting for completed aio requests (write thread)
I/O thread 62 state: waiting for completed aio requests (write thread)
I/O thread 63 state: waiting for completed aio requests (write thread)
I/O thread 64 state: waiting for completed aio requests (write thread)
I/O thread 65 state: waiting for completed aio requests (write thread)
Pending normal aio reads: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] , aio writes: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] ,
 ibuf aio reads:, log i/o's:, sync i/o's:
Pending flushes (fsync) log: 1; buffer pool: 0
22815002 OS file reads, 346374492 OS file writes, 6044672 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 7806.16 writes/s, 0.00 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 2602, free list len 546302, seg size 548905, 722903 merges
merged operations:
 insert 4265123, delete mark 41305593, delete 11844058
discarded operations:
 insert 0, delete mark 0, delete 0
Hash table size 38249683, node heap has 40587 buffer(s)
Hash table size 38249683, node heap has 44853 buffer(s)
Hash table size 38249683, node heap has 2531 buffer(s)
Hash table size 38249683, node heap has 21614 buffer(s)
Hash table size 38249683, node heap has 9901 buffer(s)
Hash table size 38249683, node heap has 13056 buffer(s)
Hash table size 38249683, node heap has 17565 buffer(s)
Hash table size 38249683, node heap has 10678 buffer(s)
2926.90 hash searches/s, 11301.08 non-hash searches/s
---
LOG
---
Log sequence number 123692133745080
Log flushed up to   123688568773877
Pages flushed up to 123666362712167
Last checkpoint at  123666312151975
1 pending log flushes, 0 pending chkp writes
327744157 log i/o's done, 7806.50 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total large memory allocated 158469193728
Dictionary memory allocated 5177753
Buffer pool size   9437184
Free buffers       91
Database pages     9276308
Old database pages 3422953
Modified db pages  1381178
Percent of dirty pages(LRU & free pages): 14.889
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 24, single page 0
Pages made young 7042025, not young 610319239
0.00 youngs/s, 0.00 non-youngs/s
Pages read 22815002, created 7257949, written 16127263
0.00 reads/s, 165.54 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 9276308, unzip_LRU len: 0
I/O sum[14208]:cur[0], unzip sum[0]:cur[0]
----------------------
INDIVIDUAL BUFFER POOL INFO
----------------------
---BUFFER POOL 0
Buffer pool size   147456
Free buffers       1
Database pages     144909
Old database pages 53471
Modified db pages  26128
Percent of dirty pages(LRU & free pages): 18.030
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 112336, not young 9200133
0.00 youngs/s, 0.00 non-youngs/s
Pages read 360569, created 113516, written 503882
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144909, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 1
Buffer pool size   147456
Free buffers       2
Database pages     144953
Old database pages 53488
Modified db pages  20645
Percent of dirty pages(LRU & free pages): 14.242
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 2, single page 0
Pages made young 109701, not young 9621890
0.00 youngs/s, 0.00 non-youngs/s
Pages read 361053, created 114052, written 239003
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144953, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 2
Buffer pool size   147456
Free buffers       1
Database pages     144961
Old database pages 53490
Modified db pages  21043
Percent of dirty pages(LRU & free pages): 14.516
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 110805, not young 9232729
0.00 youngs/s, 0.00 non-youngs/s
Pages read 358067, created 113547, written 255503
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144961, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 3
Buffer pool size   147456
Free buffers       1
Database pages     144943
Old database pages 53484
Modified db pages  21397
Percent of dirty pages(LRU & free pages): 14.762
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 2, single page 0
Pages made young 110217, not young 10219218
0.00 youngs/s, 0.00 non-youngs/s
Pages read 358825, created 114009, written 283134
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144943, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 4
Buffer pool size   147456
Free buffers       1
Database pages     144908
Old database pages 53471
Modified db pages  20903
Percent of dirty pages(LRU & free pages): 14.425
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 109760, not young 9319865
0.00 youngs/s, 0.00 non-youngs/s
Pages read 358994, created 114953, written 274164
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144908, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 5
Buffer pool size   147456
Free buffers       2
Database pages     144980
Old database pages 53498
Modified db pages  20812
Percent of dirty pages(LRU & free pages): 14.355
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 2, single page 0
Pages made young 111352, not young 9896291
0.00 youngs/s, 0.00 non-youngs/s
Pages read 359935, created 113401, written 251532
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144980, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 6
Buffer pool size   147456
Free buffers       2
Database pages     144936
Old database pages 53481
Modified db pages  20869
Percent of dirty pages(LRU & free pages): 14.398
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 107840, not young 8866700
0.00 youngs/s, 0.00 non-youngs/s
Pages read 353825, created 113263, written 243921
0.00 reads/s, 0.05 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144936, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 7
Buffer pool size   147456
Free buffers       1
Database pages     144961
Old database pages 53490
Modified db pages  20931
Percent of dirty pages(LRU & free pages): 14.439
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 111270, not young 9055104
0.00 youngs/s, 0.00 non-youngs/s
Pages read 357151, created 112695, written 240505
0.00 reads/s, 4.75 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144961, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 8
Buffer pool size   147456
Free buffers       2
Database pages     144982
Old database pages 53498
Modified db pages  21173
Percent of dirty pages(LRU & free pages): 14.604
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 110924, not young 10225302
0.00 youngs/s, 0.00 non-youngs/s
Pages read 360676, created 113865, written 242205
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144982, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 9
Buffer pool size   147456
Free buffers       1
Database pages     144967
Old database pages 53493
Modified db pages  20798
Percent of dirty pages(LRU & free pages): 14.347
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 111442, not young 9925783
0.00 youngs/s, 0.00 non-youngs/s
Pages read 360745, created 110977, written 240999
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144967, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 10
Buffer pool size   147456
Free buffers       1
Database pages     144883
Old database pages 53462
Modified db pages  21463
Percent of dirty pages(LRU & free pages): 14.814
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 105130, not young 10225999
0.00 youngs/s, 0.00 non-youngs/s
Pages read 351359, created 113255, written 247109
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144883, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 11
Buffer pool size   147456
Free buffers       2
Database pages     144992
Old database pages 53502
Modified db pages  21453
Percent of dirty pages(LRU & free pages): 14.796
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 107451, not young 9257704
0.00 youngs/s, 0.00 non-youngs/s
Pages read 352717, created 114283, written 245612
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144992, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 12
Buffer pool size   147456
Free buffers       1
Database pages     144953
Old database pages 53488
Modified db pages  20877
Percent of dirty pages(LRU & free pages): 14.402
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 108231, not young 9344128
0.00 youngs/s, 0.00 non-youngs/s
Pages read 356382, created 114699, written 254435
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144953, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 13
Buffer pool size   147456
Free buffers       2
Database pages     144959
Old database pages 53490
Modified db pages  21125
Percent of dirty pages(LRU & free pages): 14.573
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 110097, not young 8964163
0.00 youngs/s, 0.00 non-youngs/s
Pages read 356493, created 112931, written 245242
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144959, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 14
Buffer pool size   147456
Free buffers       1
Database pages     144948
Old database pages 53486
Modified db pages  22120
Percent of dirty pages(LRU & free pages): 15.260
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 108282, not young 9036261
0.00 youngs/s, 0.00 non-youngs/s
Pages read 353896, created 114038, written 247985
0.00 reads/s, 0.05 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144948, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 15
Buffer pool size   147456
Free buffers       1
Database pages     144901
Old database pages 53468
Modified db pages  21116
Percent of dirty pages(LRU & free pages): 14.573
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 112004, not young 9349251
0.00 youngs/s, 0.00 non-youngs/s
Pages read 357733, created 112985, written 238525
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144901, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 16
Buffer pool size   147456
Free buffers       1
Database pages     144955
Old database pages 53488
Modified db pages  21999
Percent of dirty pages(LRU & free pages): 15.176
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 108183, not young 9439602
0.00 youngs/s, 0.00 non-youngs/s
Pages read 351008, created 112313, written 255366
0.00 reads/s, 3.30 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144955, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 17
Buffer pool size   147456
Free buffers       1
Database pages     144990
Old database pages 53501
Modified db pages  21400
Percent of dirty pages(LRU & free pages): 14.759
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 107629, not young 9220428
0.00 youngs/s, 0.00 non-youngs/s
Pages read 354453, created 112579, written 247504
0.00 reads/s, 6.35 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144990, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 18
Buffer pool size   147456
Free buffers       1
Database pages     144905
Old database pages 53470
Modified db pages  21371
Percent of dirty pages(LRU & free pages): 14.748
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 109874, not young 9424295
0.00 youngs/s, 0.00 non-youngs/s
Pages read 357669, created 112543, written 254571
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144905, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 19
Buffer pool size   147456
Free buffers       1
Database pages     144959
Old database pages 53490
Modified db pages  21985
Percent of dirty pages(LRU & free pages): 15.166
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 111878, not young 9788366
0.00 youngs/s, 0.00 non-youngs/s
Pages read 360237, created 115821, written 250548
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144959, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 20
Buffer pool size   147456
Free buffers       1
Database pages     144913
Old database pages 53473
Modified db pages  21675
Percent of dirty pages(LRU & free pages): 14.957
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 111394, not young 9738385
0.00 youngs/s, 0.00 non-youngs/s
Pages read 357634, created 112327, written 244378
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144913, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 21
Buffer pool size   147456
Free buffers       0
Database pages     144936
Old database pages 53481
Modified db pages  21727
Percent of dirty pages(LRU & free pages): 14.991
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 110420, not young 9932824
0.00 youngs/s, 0.00 non-youngs/s
Pages read 359471, created 113119, written 252153
0.00 reads/s, 2.90 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144936, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 22
Buffer pool size   147456
Free buffers       4
Database pages     144955
Old database pages 53488
Modified db pages  22018
Percent of dirty pages(LRU & free pages): 15.189
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 108423, not young 8625866
0.00 youngs/s, 0.00 non-youngs/s
Pages read 351149, created 110052, written 251835
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144955, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 23
Buffer pool size   147456
Free buffers       2
Database pages     144938
Old database pages 53482
Modified db pages  21616
Percent of dirty pages(LRU & free pages): 14.914
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 109927, not young 10250613
0.00 youngs/s, 0.00 non-youngs/s
Pages read 355655, created 112212, written 254677
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144938, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 24
Buffer pool size   147456
Free buffers       3
Database pages     144963
Old database pages 53491
Modified db pages  21364
Percent of dirty pages(LRU & free pages): 14.737
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 112658, not young 9806788
0.00 youngs/s, 0.00 non-youngs/s
Pages read 361200, created 112895, written 249307
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144963, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 25
Buffer pool size   147456
Free buffers       2
Database pages     144937
Old database pages 53482
Modified db pages  21381
Percent of dirty pages(LRU & free pages): 14.752
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 141445, not young 12624903
0.00 youngs/s, 0.00 non-youngs/s
Pages read 362207, created 118198, written 242253
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144937, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 26
Buffer pool size   147456
Free buffers       1
Database pages     144913
Old database pages 53473
Modified db pages  21918
Percent of dirty pages(LRU & free pages): 15.125
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 112236, not young 9945266
0.00 youngs/s, 0.00 non-youngs/s
Pages read 359103, created 115166, written 246984
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144913, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 27
Buffer pool size   147456
Free buffers       2
Database pages     144923
Old database pages 53476
Modified db pages  22580
Percent of dirty pages(LRU & free pages): 15.580
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 110243, not young 10292178
0.00 youngs/s, 0.00 non-youngs/s
Pages read 358103, created 113078, written 253375
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144923, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 28
Buffer pool size   147456
Free buffers       1
Database pages     144956
Old database pages 53489
Modified db pages  21197
Percent of dirty pages(LRU & free pages): 14.623
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 106958, not young 9834381
0.00 youngs/s, 0.00 non-youngs/s
Pages read 353607, created 111102, written 237580
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144956, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 29
Buffer pool size   147456
Free buffers       0
Database pages     144918
Old database pages 53475
Modified db pages  22069
Percent of dirty pages(LRU & free pages): 15.229
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 108537, not young 10126990
0.00 youngs/s, 0.00 non-youngs/s
Pages read 356126, created 112673, written 244877
0.00 reads/s, 2.25 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144918, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 30
Buffer pool size   147456
Free buffers       1
Database pages     144934
Old database pages 53481
Modified db pages  21714
Percent of dirty pages(LRU & free pages): 14.982
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 108750, not young 9375153
0.00 youngs/s, 0.00 non-youngs/s
Pages read 359463, created 113142, written 257622
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144934, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 31
Buffer pool size   147456
Free buffers       3
Database pages     144956
Old database pages 53489
Modified db pages  21517
Percent of dirty pages(LRU & free pages): 14.843
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 2, single page 0
Pages made young 109225, not young 9876443
0.00 youngs/s, 0.00 non-youngs/s
Pages read 354376, created 112530, written 245411
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144956, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 32
Buffer pool size   147456
Free buffers       1
Database pages     144948
Old database pages 53486
Modified db pages  22756
Percent of dirty pages(LRU & free pages): 15.699
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 107776, not young 9302514
0.00 youngs/s, 0.00 non-youngs/s
Pages read 355307, created 113628, written 246290
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144948, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 33
Buffer pool size   147456
Free buffers       2
Database pages     144924
Old database pages 53477
Modified db pages  21921
Percent of dirty pages(LRU & free pages): 15.126
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 2, single page 0
Pages made young 109746, not young 9286110
0.00 youngs/s, 0.00 non-youngs/s
Pages read 353838, created 114277, written 248945
0.00 reads/s, 2.40 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144924, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 34
Buffer pool size   147456
Free buffers       2
Database pages     144999
Old database pages 53505
Modified db pages  20856
Percent of dirty pages(LRU & free pages): 14.383
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 108818, not young 9821739
0.00 youngs/s, 0.00 non-youngs/s
Pages read 355011, created 113121, written 243533
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144999, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 35
Buffer pool size   147456
Free buffers       2
Database pages     144915
Old database pages 53474
Modified db pages  21404
Percent of dirty pages(LRU & free pages): 14.770
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 2, single page 0
Pages made young 109420, not young 9886599
0.00 youngs/s, 0.00 non-youngs/s
Pages read 351533, created 112481, written 240488
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144915, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 36
Buffer pool size   147456
Free buffers       3
Database pages     144933
Old database pages 53480
Modified db pages  21294
Percent of dirty pages(LRU & free pages): 14.692
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 110025, not young 9519484
0.00 youngs/s, 0.00 non-youngs/s
Pages read 355830, created 112532, written 250293
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144933, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 37
Buffer pool size   147456
Free buffers       1
Database pages     144964
Old database pages 53492
Modified db pages  21299
Percent of dirty pages(LRU & free pages): 14.692
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 108996, not young 9556020
0.00 youngs/s, 0.00 non-youngs/s
Pages read 355998, created 113515, written 250275
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144964, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 38
Buffer pool size   147456
Free buffers       2
Database pages     144931
Old database pages 53479
Modified db pages  21363
Percent of dirty pages(LRU & free pages): 14.740
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 109675, not young 8480821
0.00 youngs/s, 0.00 non-youngs/s
Pages read 355217, created 113737, written 241535
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144931, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 39
Buffer pool size   147456
Free buffers       2
Database pages     144951
Old database pages 53487
Modified db pages  21038
Percent of dirty pages(LRU & free pages): 14.514
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 107904, not young 9907361
0.00 youngs/s, 0.00 non-youngs/s
Pages read 352335, created 113158, written 242112
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144951, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 40
Buffer pool size   147456
Free buffers       1
Database pages     144939
Old database pages 53482
Modified db pages  21573
Percent of dirty pages(LRU & free pages): 14.884
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 109289, not young 9601547
0.00 youngs/s, 0.00 non-youngs/s
Pages read 356523, created 114272, written 244932
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144939, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 41
Buffer pool size   147456
Free buffers       2
Database pages     144964
Old database pages 53492
Modified db pages  21778
Percent of dirty pages(LRU & free pages): 15.023
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 2, single page 0
Pages made young 108451, not young 9202651
0.00 youngs/s, 0.00 non-youngs/s
Pages read 351524, created 114148, written 247744
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144964, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 42
Buffer pool size   147456
Free buffers       2
Database pages     144900
Old database pages 53468
Modified db pages  22178
Percent of dirty pages(LRU & free pages): 15.305
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 108064, not young 9175554
0.00 youngs/s, 0.00 non-youngs/s
Pages read 352912, created 114487, written 262151
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144900, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 43
Buffer pool size   147456
Free buffers       2
Database pages     144951
Old database pages 53487
Modified db pages  20693
Percent of dirty pages(LRU & free pages): 14.276
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 109320, not young 8743150
0.00 youngs/s, 0.00 non-youngs/s
Pages read 351134, created 113713, written 243393
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144951, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 44
Buffer pool size   147456
Free buffers       1
Database pages     144916
Old database pages 53474
Modified db pages  21532
Percent of dirty pages(LRU & free pages): 14.858
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 110311, not young 9605638
0.00 youngs/s, 0.00 non-youngs/s
Pages read 354743, created 113571, written 245707
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144916, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 45
Buffer pool size   147456
Free buffers       1
Database pages     144940
Old database pages 53483
Modified db pages  21410
Percent of dirty pages(LRU & free pages): 14.771
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 109462, not young 9071163
0.00 youngs/s, 0.00 non-youngs/s
Pages read 355838, created 113243, written 253390
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144940, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 46
Buffer pool size   147456
Free buffers       1
Database pages     144971
Old database pages 53494
Modified db pages  21366
Percent of dirty pages(LRU & free pages): 14.738
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 108511, not young 8777835
0.00 youngs/s, 0.00 non-youngs/s
Pages read 357139, created 113248, written 243394
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144971, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 47
Buffer pool size   147456
Free buffers       2
Database pages     144921
Old database pages 53476
Modified db pages  21568
Percent of dirty pages(LRU & free pages): 14.882
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 111274, not young 9232989
0.00 youngs/s, 0.00 non-youngs/s
Pages read 360000, created 111524, written 248842
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144921, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 48
Buffer pool size   147456
Free buffers       1
Database pages     144950
Old database pages 53486
Modified db pages  21399
Percent of dirty pages(LRU & free pages): 14.763
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 109729, not young 9153445
0.00 youngs/s, 0.00 non-youngs/s
Pages read 355824, created 112997, written 240555
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144950, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 49
Buffer pool size   147456
Free buffers       1
Database pages     144951
Old database pages 53487
Modified db pages  21387
Percent of dirty pages(LRU & free pages): 14.754
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 108581, not young 9633191
0.00 youngs/s, 0.00 non-youngs/s
Pages read 357554, created 112887, written 245779
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144951, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 50
Buffer pool size   147456
Free buffers       2
Database pages     144900
Old database pages 53468
Modified db pages  20825
Percent of dirty pages(LRU & free pages): 14.372
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 108458, not young 8847389
0.00 youngs/s, 0.00 non-youngs/s
Pages read 353709, created 109697, written 235346
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144900, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 51
Buffer pool size   147456
Free buffers       1
Database pages     144978
Old database pages 53497
Modified db pages  22004
Percent of dirty pages(LRU & free pages): 15.177
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 110072, not young 9459819
0.00 youngs/s, 0.00 non-youngs/s
Pages read 358822, created 114998, written 248437
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144978, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 52
Buffer pool size   147456
Free buffers       1
Database pages     144910
Old database pages 53472
Modified db pages  22143
Percent of dirty pages(LRU & free pages): 15.280
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 107790, not young 9413527
0.00 youngs/s, 0.00 non-youngs/s
Pages read 356598, created 112780, written 251845
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144910, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 53
Buffer pool size   147456
Free buffers       1
Database pages     144925
Old database pages 53477
Modified db pages  21393
Percent of dirty pages(LRU & free pages): 14.761
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 2, single page 0
Pages made young 107890, not young 9241682
0.00 youngs/s, 0.00 non-youngs/s
Pages read 356597, created 111015, written 243544
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144925, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 54
Buffer pool size   147456
Free buffers       1
Database pages     144985
Old database pages 53499
Modified db pages  21459
Percent of dirty pages(LRU & free pages): 14.801
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 106952, not young 8969707
0.00 youngs/s, 0.00 non-youngs/s
Pages read 347490, created 111176, written 236464
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144985, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 55
Buffer pool size   147456
Free buffers       1
Database pages     144969
Old database pages 53493
Modified db pages  21606
Percent of dirty pages(LRU & free pages): 14.904
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 109284, not young 9421591
0.00 youngs/s, 0.00 non-youngs/s
Pages read 357562, created 112863, written 242024
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144969, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 56
Buffer pool size   147456
Free buffers       1
Database pages     144936
Old database pages 53481
Modified db pages  21509
Percent of dirty pages(LRU & free pages): 14.840
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 109139, not young 9371649
0.00 youngs/s, 0.00 non-youngs/s
Pages read 355925, created 114000, written 240569
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144936, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 57
Buffer pool size   147456
Free buffers       1
Database pages     144892
Old database pages 53465
Modified db pages  21458
Percent of dirty pages(LRU & free pages): 14.809
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 111941, not young 10153573
0.00 youngs/s, 0.00 non-youngs/s
Pages read 362308, created 114986, written 255049
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144892, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 58
Buffer pool size   147456
Free buffers       1
Database pages     144933
Old database pages 53480
Modified db pages  21875
Percent of dirty pages(LRU & free pages): 15.093
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 2, single page 0
Pages made young 110014, not young 9371009
0.00 youngs/s, 0.00 non-youngs/s
Pages read 357577, created 114753, written 245289
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144933, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 59
Buffer pool size   147456
Free buffers       0
Database pages     145004
Old database pages 53506
Modified db pages  21695
Percent of dirty pages(LRU & free pages): 14.962
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 110584, not young 10053258
0.00 youngs/s, 0.00 non-youngs/s
Pages read 360602, created 117041, written 247955
0.00 reads/s, 1.50 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 145004, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 60
Buffer pool size   147456
Free buffers       2
Database pages     144903
Old database pages 53469
Modified db pages  22446
Percent of dirty pages(LRU & free pages): 15.490
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 2, single page 0
Pages made young 110566, not young 9547068
0.00 youngs/s, 0.00 non-youngs/s
Pages read 358447, created 115431, written 258934
0.00 reads/s, 1.10 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144903, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 61
Buffer pool size   147456
Free buffers       1
Database pages     144945
Old database pages 53485
Modified db pages  22494
Percent of dirty pages(LRU & free pages): 15.519
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 2, single page 0
Pages made young 110027, not young 9322499
0.00 youngs/s, 0.00 non-youngs/s
Pages read 359557, created 115407, written 247999
0.00 reads/s, 3.30 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144945, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 62
Buffer pool size   147456
Free buffers       1
Database pages     144941
Old database pages 53483
Modified db pages  21110
Percent of dirty pages(LRU & free pages): 14.564
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 2, single page 0
Pages made young 111434, not young 8944235
0.00 youngs/s, 0.00 non-youngs/s
Pages read 357184, created 112681, written 241240
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144941, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 63
Buffer pool size   147456
Free buffers       2
Database pages     144932
Old database pages 53480
Modified db pages  21962
Percent of dirty pages(LRU & free pages): 15.153
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 107900, not young 10201420
0.00 youngs/s, 0.00 non-youngs/s
Pages read 354483, created 114363, written 251013
0.00 reads/s, 3.20 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 144932, unzip_LRU len: 0
I/O sum[222]:cur[0], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
3 read views open inside InnoDB
Process ID=29311, Main thread ID=140158591362816, state: flushing log
Number of rows inserted 271814413, updated 170000953, deleted 61691005, read 48688048564
7815.86 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
Number of system rows inserted 0, updated 0, deleted 0, read 0
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
----------------------------
END OF INNODB MONITOR OUTPUT
============================
InnoDB: ###### Diagnostic info printed to the standard error stream

2019-11-18  8:54:07 140158574577408 [ERROR] [FATAL] InnoDB: Semaphore wait has lasted > 600 seconds. We intentionally crash the server because it appears to be hung.
191118  8:54:07 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
 
Server version: 10.2.15-MariaDB
key_buffer_size=134217728
read_buffer_size=131072
max_used_connections=138
max_threads=2507
thread_count=36
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 5639693 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x49000
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55824a75ff9e]
/usr/sbin/mysqld(handle_fatal_signal+0x355)[0x55824a1eba05]
/lib64/libpthread.so.0(+0xf5e0)[0x7f9f5212d5e0]
/lib64/libc.so.6(gsignal+0x37)[0x7f9f5063a1f7]
/lib64/libc.so.6(abort+0x148)[0x7f9f5063b8e8]
/usr/sbin/mysqld(+0x9a7a73)[0x55824a525a73]
/usr/sbin/mysqld(+0x954ff7)[0x55824a4d2ff7]
/lib64/libpthread.so.0(+0x7e25)[0x7f9f52125e25]
/lib64/libc.so.6(clone+0x6d)[0x7f9f506fd34d]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
2019-11-18  8:54:33 140319486716032 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-11-18  8:54:33 140319486716032 [Note] InnoDB: Uses event mutexes
2019-11-18  8:54:33 140319486716032 [Note] InnoDB: Compressed tables use zlib 1.2.7
2019-11-18  8:54:33 140319486716032 [Note] InnoDB: Using Linux native AIO
2019-11-18  8:54:33 140319486716032 [Note] InnoDB: Number of pools: 1
2019-11-18  8:54:33 140319486716032 [Note] InnoDB: Using SSE2 crc32 instructions
2019-11-18  8:54:33 140319486716032 [Note] InnoDB: Initializing buffer pool, total size = 144G, instances = 64, chunk size = 128M
2019-11-18  8:54:43 140319486716032 [Note] InnoDB: Completed initialization of buffer pool
2019-11-18  8:54:45 140151671248640 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-11-18  8:54:45 140319486716032 [Note] InnoDB: Highest supported file format is Barracuda.
2019-11-18  8:54:47 140319486716032 [Note] InnoDB: Starting crash recovery from checkpoint LSN=123686452084648
2019-11-18  8:54:59 140319486716032 [Note] InnoDB: Read redo log up to LSN=123666746720768
2019-11-18  8:55:14 140319486716032 [Note] InnoDB: Read redo log up to LSN=123667392578048
2019-11-18  8:55:29 140319486716032 [Note] InnoDB: Read redo log up to LSN=123668031750656
2019-11-18  8:55:44 140319486716032 [Note] InnoDB: Read redo log up to LSN=123668676952576
2019-11-18  8:55:59 140319486716032 [Note] InnoDB: Read redo log up to LSN=123669316321792
2019-11-18  8:56:14 140319486716032 [Note] InnoDB: Read redo log up to LSN=123669987344896
2019-11-18  8:56:29 140319486716032 [Note] InnoDB: Read redo log up to LSN=123670659219968
2019-11-18  8:56:44 140319486716032 [Note] InnoDB: Read redo log up to LSN=123671297540608
2019-11-18  8:56:59 140319486716032 [Note] InnoDB: Read redo log up to LSN=123671927800320
2019-11-18  8:57:14 140319486716032 [Note] InnoDB: Read redo log up to LSN=123672564613632
2019-11-18  8:57:29 140319486716032 [Note] InnoDB: Read redo log up to LSN=123673130451456
2019-11-18  8:57:44 140319486716032 [Note] InnoDB: Read redo log up to LSN=123673654542848
2019-11-18  8:57:59 140319486716032 [Note] InnoDB: Read redo log up to LSN=123674199802368
2019-11-18  8:58:14 140319486716032 [Note] InnoDB: Read redo log up to LSN=123674741260800
2019-11-18  8:58:29 140319486716032 [Note] InnoDB: Read redo log up to LSN=123675260502528
2019-11-18  8:58:44 140319486716032 [Note] InnoDB: Read redo log up to LSN=123675796324864
2019-11-18  8:58:59 140319486716032 [Note] InnoDB: Read redo log up to LSN=123676318056960
2019-11-18  8:59:14 140319486716032 [Note] InnoDB: Read redo log up to LSN=123676834087424
2019-11-18  8:59:29 140319486716032 [Note] InnoDB: Read redo log up to LSN=123677359292928
2019-11-18  8:59:44 140319486716032 [Note] InnoDB: Read redo log up to LSN=123677824401920
2019-11-18  8:59:59 140319486716032 [Note] InnoDB: Read redo log up to LSN=123678273913344
2019-11-18  9:00:14 140319486716032 [Note] InnoDB: Read redo log up to LSN=123678723359232
2019-11-18  9:00:29 140319486716032 [Note] InnoDB: Read redo log up to LSN=123679154323968
2019-11-18  9:00:44 140319486716032 [Note] InnoDB: Read redo log up to LSN=123679567987200
2019-11-18  9:00:59 140319486716032 [Note] InnoDB: Read redo log up to LSN=123680011207168
2019-11-18  9:01:14 140319486716032 [Note] InnoDB: Read redo log up to LSN=123680452985344
2019-11-18  9:01:29 140319486716032 [Note] InnoDB: Read redo log up to LSN=123680891093504
2019-11-18  9:01:44 140319486716032 [Note] InnoDB: Read redo log up to LSN=123681314128384
2019-11-18  9:01:59 140319486716032 [Note] InnoDB: Read redo log up to LSN=123681755447808
2019-11-18  9:02:14 140319486716032 [Note] InnoDB: Read redo log up to LSN=123682197291520
2019-11-18  9:02:29 140319486716032 [Note] InnoDB: Read redo log up to LSN=123682643591680
2019-11-18  9:02:44 140319486716032 [Note] InnoDB: Read redo log up to LSN=123683121152512
2019-11-18  9:02:59 140319486716032 [Note] InnoDB: Read redo log up to LSN=123683531211264
2019-11-18  9:03:14 140319486716032 [Note] InnoDB: Read redo log up to LSN=123684056613376
2019-11-18  9:03:29 140319486716032 [Note] InnoDB: Read redo log up to LSN=123684562878976
2019-11-18  9:03:44 140319486716032 [Note] InnoDB: Read redo log up to LSN=123685008785920
2019-11-18  9:03:59 140319486716032 [Note] InnoDB: Read redo log up to LSN=123685441716736
2019-11-18  9:04:14 140319486716032 [Note] InnoDB: Read redo log up to LSN=123685853348352
2019-11-18  9:04:29 140319486716032 [Note] InnoDB: Read redo log up to LSN=123686238306816
2019-11-18  9:04:44 140319486716032 [Note] InnoDB: Read redo log up to LSN=123686647120384
2019-11-18  9:04:59 140319486716032 [Note] InnoDB: Read redo log up to LSN=123687156793856
2019-11-18  9:05:14 140319486716032 [Note] InnoDB: Read redo log up to LSN=123687779058176
2019-11-18  9:05:29 140319486716032 [Note] InnoDB: Read redo log up to LSN=123688427799040
2019-11-18  9:05:44 140319486716032 [Note] InnoDB: Read redo log up to LSN=123689089909248
2019-11-18  9:05:59 140319486716032 [Note] InnoDB: Read redo log up to LSN=123689764733440
2019-11-18  9:06:14 140319486716032 [Note] InnoDB: Read redo log up to LSN=123690432741888
2019-11-18  9:06:29 140319486716032 [Note] InnoDB: Read redo log up to LSN=123691113267712
2019-11-18  9:06:44 140319486716032 [Note] InnoDB: Read redo log up to LSN=123691798643200
2019-11-18  9:06:59 140319486716032 [Note] InnoDB: To recover: 1400082 pages from log
2019-11-18  9:07:14 140319486716032 [Note] InnoDB: To recover: 1399786 pages from log
2019-11-18  9:07:29 140319486716032 [Note] InnoDB: To recover: 1399488 pages from log
2019-11-18  9:07:37 140319486716032 [Note] InnoDB: 7 transaction(s) which must be rolled back or cleaned up in total 6 row operations to undo
2019-11-18  9:07:37 140319486716032 [Note] InnoDB: Trx id counter is 67166278400
2019-11-18  9:07:37 140319486716032 [Note] InnoDB: Starting final batch to recover 1399327 pages from redo log.
2019-11-18  9:07:44 140151990171392 [Note] InnoDB: To recover: 1355738 pages from log
2019-11-18  9:07:59 140152174810880 [Note] InnoDB: To recover: 1239097 pages from log
2019-11-18  9:08:14 140152116061952 [Note] InnoDB: To recover: 1096783 pages from log
2019-11-18  9:08:29 140152032134912 [Note] InnoDB: To recover: 969256 pages from log
2019-11-18  9:08:44 140152107669248 [Note] InnoDB: To recover: 879424 pages from log
2019-11-18  9:08:59 140152149632768 [Note] InnoDB: To recover: 789394 pages from log
2019-11-18  9:09:14 140152116061952 [Note] InnoDB: To recover: 659659 pages from log
2019-11-18  9:09:29 140151956600576 [Note] InnoDB: To recover: 558816 pages from log
2019-11-18  9:09:44 140152099276544 [Note] InnoDB: To recover: 468811 pages from log
2019-11-18  9:09:59 140152006956800 [Note] InnoDB: To recover: 368592 pages from log
2019-11-18  9:10:14 140152199988992 [Note] InnoDB: To recover: 285262 pages from log
2019-11-18  9:10:29 140152191596288 [Note] InnoDB: To recover: 184779 pages from log
2019-11-18  9:10:44 140152141240064 [Note] InnoDB: To recover: 106142 pages from log
2019-11-18  9:10:59 140152006956800 [Note] InnoDB: To recover: 11098 pages from log
2019-11-18  9:11:18 140319486716032 [Note] InnoDB: 128 out of 128 rollback segments are active.
2019-11-18  9:11:18 140147047782144 [Note] InnoDB: Starting in background the rollback of recovered transactions
2019-11-18  9:11:18 140147047782144 [Note] InnoDB: To roll back: 7 transactions, 6 rows
2019-11-18  9:11:18 140147047782144 [Note] InnoDB: Rolled back recovered transaction 67166278063
2019-11-18  9:11:18 140147047782144 [Note] InnoDB: Rolled back recovered transaction 67166278062
2019-11-18  9:11:18 140147047782144 [Note] InnoDB: Rolled back recovered transaction 67166278061
2019-11-18  9:11:18 140147047782144 [Note] InnoDB: Rolled back recovered transaction 67166278052
2019-11-18  9:11:18 140147047782144 [Note] InnoDB: Rolled back recovered transaction 67164985040
2019-11-18  9:11:18 140147047782144 [Note] InnoDB: Rolled back recovered transaction 67151659109
2019-11-18  9:11:18 140319486716032 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2019-11-18  9:11:18 140319486716032 [Note] InnoDB: Creating shared tablespace for temporary tables
2019-11-18  9:11:18 140319486716032 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2019-11-18  9:11:18 140319486716032 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2019-11-18  9:11:18 140147047782144 [Note] InnoDB: Rolled back recovered transaction 67151642012
2019-11-18  9:11:18 140147047782144 [Note] InnoDB: Rollback of non-prepared transactions completed
2019-11-18  9:11:18 140319486716032 [Note] InnoDB: 5.7.22 started; log sequence number 123692136544372
2019-11-18  9:11:18 140155745097472 [Note] InnoDB: Loading buffer pool(s) from /u01/mysql/ib_buffer_pool
2019-11-18  9:11:18 140319486716032 [Note] Plugin 'FEEDBACK' is disabled.
191118  9:11:18 server_audit: MariaDB Audit Plugin version 1.4.4 STARTED.
191118  9:11:18 server_audit: logging started to the file server_audit.log.
2019-11-18  9:11:18 140319486716032 [Note] Recovering after a crash using tc.log
2019-11-18  9:11:18 140319486716032 [Note] Starting crash recovery...
2019-11-18  9:11:18 140319486716032 [Note] Crash recovery finished.
2019-11-18  9:11:18 140319486716032 [Note] Server socket created on IP: '::'.
2019-11-18  9:11:20 140319486716032 [Note] Reading of all Master_info entries succeded
2019-11-18  9:11:20 140319486716032 [Note] Added new Master_info '' to hash table
2019-11-18  9:11:20 140319486716032 [Note] /usr/sbin/mysqld: ready for connections.
Version: '10.2.15-MariaDB'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MariaDB Server
2019-11-18  9:11:51 140155727705856 [Warning] IP address  could not be resolved: Name or service not known
2019-11-18  9:12:17 140155724977920 [Warning] Access denied for user  (using password: YES)
2019-11-18  9:16:43 140155723765504 [Warning] IP address  could not be resolved: Name or service not known
2019-11-18  9:32:53 140155368978176 [Warning] Access denied for user  (using password: NO)
2019-11-18  9:35:02 140155368675072 [Warning] Access denied for user  (using password: NO)



 Comments   
Comment by Prabha [ 2019-11-27 ]

Hi Marko,
Can you please help me.

Comment by Marko Mäkelä [ 2019-11-27 ]

prabha, we have a few such hang reports, but unfortunately not enough input. To better diagnose the hang, I would need the output of the following command during the hang:

gdb -ex "set pagination 0" -ex "print dict_operation_lock" -ex "print dict_sys->mutex" -ex "thread apply all bt" --batch -p $(pgrep -x mysqld)

Substitute $(pgrep -x mysqld) with the process identifier, and be sure to have the debugging symbols installed, so that the variables can be accessed.

Also, the innodb_stats_persistent=ON in particular with innodb_stats_auto_recalc=ON is known to cause hangs.

Comment by Prabha [ 2019-11-27 ]

Marko,
Can you please guide me how to install debug symbols.

   

Comment by Prabha [ 2019-11-28 ]

Marko,
We have installed the debug symbols, Will collect the details and update you. Meantime could you please confirm, If we increase the innodb_fatal_semaphore_wait_threshold value will fix the issue? Also you have mentioned that innodb_stats_persistent=ON in particular with innodb_stats_auto_recalc=ON will cause the hangs, what is the impact if we disable innodb_stats_auto_recalc?

   

Comment by Marko Mäkelä [ 2019-11-29 ]

For support questions, MariaDB Corporation has a commercial offering.

I do not think that increasing innodb_fatal_semaphore_wait_threshold should have any other impact than keeping the server hung for a longer time before the internal watchdog kills the process. Setting it to a larger value could improve your chances to attach gdb to the process while it is hung. But you could also get the requested information from a core dump (if you have enabled them).

Generated at Thu Feb 08 09:04:25 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.