2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor 2018-07-20 11:54:45 128684013562112 [ERROR] Error in accept: Bad file descriptor /home/buildbot/buildbot/build/mariadb-10.1.32/storage/tokudb/ha_tokudb.cc:1315 open_main_dictionary: Assertion `Handlerton: share->file == NULL ' failed (errno=0) : Success Backtrace: (Note: toku_do_assert=0x0x75099093f920) /usr/lib/mysql/plugin/ha_tokudb.so(_Z19db_env_do_backtraceP8_IO_FILE+0x2e)[0x75099093f3ce] /usr/lib/mysql/plugin/ha_tokudb.so(+0x130503)[0x75099093f503] /usr/lib/mysql/plugin/ha_tokudb.so(+0x13067e)[0x75099093f67e] /usr/lib/mysql/plugin/ha_tokudb.so(+0x84bf8)[0x750990893bf8] /usr/lib/mysql/plugin/ha_tokudb.so(_Z21toku_hton_assert_failPKcS0_S0_ii+0x88)[0x75099085ffc8] /usr/lib/mysql/plugin/ha_tokudb.so(_ZN9ha_tokudb20open_main_dictionaryEPKcbP13__toku_db_txn+0x2d1)[0x750990879471] /usr/lib/mysql/plugin/ha_tokudb.so(_ZN9ha_tokudb16initialize_shareEPKci+0x1c3)[0x750990885c23] /usr/lib/mysql/plugin/ha_tokudb.so(_ZN9ha_tokudb4openEPKcij+0x4c4)[0x7509908866b4] /usr/sbin/mysqld(_ZN7handler7ha_openEP5TABLEPKcij+0x33)[0xbb03f9d6633] /usr/sbin/mysqld(+0xa2b855)[0xbb03fe5b855] /usr/sbin/mysqld(_ZN7handler7ha_openEP5TABLEPKcij+0x33)[0xbb03f9d6633] /usr/sbin/mysqld(_Z21open_table_from_shareP3THDP11TABLE_SHAREPKcjjjP5TABLEb+0xb29)[0xbb03f8ed869] /usr/sbin/mysqld(_Z10open_tableP3THDP10TABLE_LISTP18Open_table_context+0xade)[0xbb03f7f4dde] /usr/sbin/mysqld(_Z11open_tablesP3THDRK14DDL_options_stPP10TABLE_LISTPjjP19Prelocking_strategy+0xab2)[0xbb03f7f9142] /usr/sbin/mysqld(_Z30open_normal_and_derived_tablesP3THDP10TABLE_LISTjj+0x59)[0xbb03f7f9ca9] /usr/sbin/mysqld(+0x470199)[0xbb03f8a0199] /usr/sbin/mysqld(_Z14get_all_tablesP3THDP10TABLE_LISTP4Item+0x7e0)[0xbb03f8b3300] /usr/sbin/mysqld(_Z24get_schema_tables_resultP4JOIN23enum_schema_table_state+0x296)[0xbb03f8b4df6] /usr/sbin/mysqld(_ZN4JOIN10exec_innerEv+0x71d)[0xbb03f89972d] /usr/sbin/mysqld(_ZN4JOIN4execEv+0x54)[0xbb03f89ba74] /usr/sbin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xe7)[0xbb03f898227] /usr/sbin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x139)[0xbb03f898bf9] /usr/sbin/mysqld(+0x354a78)[0xbb03f784a78] /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x71bb)[0xbb03f8422cb] /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x311)[0xbb03f8444d1] /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x25a0)[0xbb03f847b80] /usr/sbin/mysqld(_Z10do_commandP3THD+0x219)[0xbb03f848439] /usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x182)[0xbb03f916dd2] /usr/sbin/mysqld(handle_one_connection+0x40)[0xbb03f916f70] /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7509937b46ba] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x750992e5f41d] Engine status: BUILD_ID = 0 Env panic code: 18446744073709551615 Env panic string: /home/buildbot/buildbot/build/mariadb-10.1.32/storage/tokudb/ha_tokudb.cc:1315 open_main_dictionary: Assertion `Handlerton: share->file == NULL ' failed (errno=0) time of environment creation: Sun Nov 27 16:33:50 2016 time of engine startup: Fri Jul 20 11:47:07 2018 time now: Fri Jul 20 11:54:51 2018 db opens: 9616 db closes: 16 num open dbs now: 9600 max open dbs: 9601 period, in ms, that recovery log is automatically fsynced: 0 dictionary inserts: 0 dictionary inserts fail: 0 dictionary deletes: 1608 dictionary deletes fail: 0 dictionary updates: 0 dictionary updates fail: 0 dictionary broadcast updates: 0 dictionary broadcast updates fail: 0 dictionary multi inserts: 0 dictionary multi inserts fail: 0 dictionary multi deletes: 0 dictionary multi deletes fail: 0 dictionary updates multi: 0 dictionary updates multi fail: 0 le: max committed xr: 1 le: max provisional xr: 2 le: expanded: 0 le: max memsize: 68 le: size of leafentries before garbage collection (during message application): 1228139 le: size of leafentries after garbage collection (during message application): 411345 le: size of leafentries before garbage collection (outside message application): 0 le: size of leafentries after garbage collection (outside message application): 0 checkpoint: period: 60 checkpoint: footprint: 40 checkpoint: last checkpoint began: Fri Jul 20 11:54:08 2018 checkpoint: last complete checkpoint began: Fri Jul 20 11:53:08 2018 checkpoint: last complete checkpoint ended: Fri Jul 20 11:53:09 2018 checkpoint: time spent during checkpoint (begin and end phases): 16 checkpoint: time spent during last checkpoint (begin and end phases): 1 checkpoint: last complete checkpoint LSN: 28153374688 checkpoint: checkpoints taken : 7 checkpoint: checkpoints failed: 0 checkpoint: waiters now: 0 checkpoint: waiters max: 0 checkpoint: non-checkpoint client wait on mo lock: 0 checkpoint: non-checkpoint client wait on cs lock: 0 checkpoint: checkpoint begin time: 18896 checkpoint: long checkpoint begin time: 0 checkpoint: long checkpoint begin count: 0 checkpoint: checkpoint end time: 18624856 checkpoint: long checkpoint end time: 0 checkpoint: long checkpoint end count: 0 cachetable: miss: 5071 cachetable: miss time: 1000742 cachetable: prefetches: 182 cachetable: size current: 1199698716 cachetable: size limit: 34896609280 cachetable: size writing: 0 cachetable: size nonleaf: 60687158 cachetable: size leaf: 1139009286 cachetable: size rollback: 2272 cachetable: size cachepressure: 58825654 cachetable: size currently cloned data for checkpoint: 0 cachetable: evictions: 0 cachetable: cleaner executions: 716 cachetable: cleaner period: 1 cachetable: cleaner iterations: 5 cachetable: number of waits on cache pressure: 0 cachetable: time waiting on cache pressure: 0 cachetable: number of long waits on cache pressure: 0 cachetable: long time waiting on cache pressure: 0 cachetable: client pool: number of threads in pool: 12 cachetable: client pool: number of currently active threads in pool: 0 cachetable: client pool: number of currently queued work items: 0 cachetable: client pool: largest number of queued work items: 0 cachetable: client pool: total number of work items processed: 0 cachetable: client pool: total execution time of processing work items: 0 cachetable: cachetable pool: number of threads in pool: 24 cachetable: cachetable pool: number of currently active threads in pool: 0 cachetable: cachetable pool: number of currently queued work items: 0 cachetable: cachetable pool: largest number of queued work items: 1 cachetable: cachetable pool: total number of work items processed: 189 cachetable: cachetable pool: total execution time of processing work items: 1291 cachetable: checkpoint pool: number of threads in pool: 3 cachetable: checkpoint pool: number of currently active threads in pool: 0 cachetable: checkpoint pool: number of currently queued work items: 0 cachetable: checkpoint pool: largest number of queued work items: 0 cachetable: checkpoint pool: total number of work items processed: 0 cachetable: checkpoint pool: total execution time of processing work items: 0 locktree: memory size: 96 locktree: memory size limit: 4294967296 locktree: number of times lock escalation ran: 0 locktree: time spent running escalation (seconds): 0.000000 locktree: latest post-escalation memory size: 0 locktree: number of locktrees open now: 9602 locktree: number of pending lock requests: 0 locktree: number of locktrees eligible for the STO: 1 locktree: number of times a locktree ended the STO early: 1596 locktree: time spent ending the STO early (seconds): 0.027879 locktree: number of wait locks: 0 locktree: time waiting for locks: 0 locktree: number of long wait locks: 0 locktree: long time waiting for locks: 0 locktree: number of lock timeouts: 0 locktree: number of waits on lock escalation: 0 locktree: time waiting on lock escalation: 0 locktree: number of long waits on lock escalation: 0 locktree: long time waiting on lock escalation: 0 ft: dictionary updates: 0 ft: dictionary broadcast updates: 0 ft: descriptor set: 0 ft: messages ignored by leaf due to msn: 2055 ft: total search retries due to TRY_AGAIN: 6701 ft: searches requiring more tries than the height of the tree: 1 ft: searches requiring more tries than the height of the tree plus three: 0 ft: leaf nodes flushed to disk (not for checkpoint): 2 ft: leaf nodes flushed to disk (not for checkpoint) (bytes): 1024 ft: leaf nodes flushed to disk (not for checkpoint) (uncompressed bytes): 550 ft: leaf nodes flushed to disk (not for checkpoint) (seconds): 0.000078 ft: nonleaf nodes flushed to disk (not for checkpoint): 0 ft: nonleaf nodes flushed to disk (not for checkpoint) (bytes): 0 ft: nonleaf nodes flushed to disk (not for checkpoint) (uncompressed bytes): 0 ft: nonleaf nodes flushed to disk (not for checkpoint) (seconds): 0.000000 ft: leaf nodes flushed to disk (for checkpoint): 1657 ft: leaf nodes flushed to disk (for checkpoint) (bytes): 30940672 ft: leaf nodes flushed to disk (for checkpoint) (uncompressed bytes): 102086352 ft: leaf nodes flushed to disk (for checkpoint) (seconds): 0.313292 ft: nonleaf nodes flushed to disk (for checkpoint): 128 ft: nonleaf nodes flushed to disk (for checkpoint) (bytes): 22239744 ft: nonleaf nodes flushed to disk (for checkpoint) (uncompressed bytes): 64555364 ft: nonleaf nodes flushed to disk (for checkpoint) (seconds): 0.098578 ft: uncompressed / compressed bytes written (leaf): UNKNOWN STATUS TYPE: 6 ft: uncompressed / compressed bytes written (nonleaf): UNKNOWN STATUS TYPE: 6 ft: uncompressed / compressed bytes written (overall): UNKNOWN STATUS TYPE: 6 ft: nonleaf node partial evictions: 0 ft: nonleaf node partial evictions (bytes): 0 ft: leaf node partial evictions: 0 ft: leaf node partial evictions (bytes): 0 ft: leaf node full evictions: 0 ft: leaf node full evictions (bytes): 0 ft: nonleaf node full evictions: 0 ft: nonleaf node full evictions (bytes): 0 ft: leaf nodes created: 1 ft: nonleaf nodes created: 0 ft: leaf nodes destroyed: 0 ft: nonleaf nodes destroyed: 0 ft: bytes of messages injected at root (all trees): 0 ft: bytes of messages flushed from h1 nodes to leaves: 4590182 ft: bytes of messages currently in trees (estimate): 18446744073704961434 ft: messages injected at root: 0 ft: broadcast messages injected at root: 0 ft: basements decompressed as a target of a query: 17 ft: basements decompressed for prelocked range: 107 ft: basements decompressed for prefetch: 0 ft: basements decompressed for write: 0 ft: buffers decompressed as a target of a query: 592 ft: buffers decompressed for prelocked range: 42 ft: buffers decompressed for prefetch: 0 ft: buffers decompressed for write: 1842 ft: pivots fetched for query: 1123 ft: pivots fetched for query (bytes): 33492480 ft: pivots fetched for query (seconds): 0.080313 ft: pivots fetched for prefetch: 182 ft: pivots fetched for prefetch (bytes): 5963776 ft: pivots fetched for prefetch (seconds): 0.006657 ft: pivots fetched for write: 1 ft: pivots fetched for write (bytes): 512 ft: pivots fetched for write (seconds): 0.000001 ft: basements fetched as a target of a query: 2956 ft: basements fetched as a target of a query (bytes): 36894208 ft: basements fetched as a target of a query (seconds): 0.022762 ft: basements fetched for prelocked range: 1087 ft: basements fetched for prelocked range (bytes): 8776192 ft: basements fetched for prelocked range (seconds): 0.003646 ft: basements fetched for prefetch: 7571 ft: basements fetched for prefetch (bytes): 63333888 ft: basements fetched for prefetch (seconds): 0.023587 ft: basements fetched for write: 1793 ft: basements fetched for write (bytes): 35568128 ft: basements fetched for write (seconds): 0.204071 ft: buffers fetched as a target of a query: 103 ft: buffers fetched as a target of a query (bytes): 79872 ft: buffers fetched as a target of a query (seconds): 0.000345 ft: buffers fetched for prelocked range: 64 ft: buffers fetched for prelocked range (bytes): 33280 ft: buffers fetched for prelocked range (seconds): 0.000066 ft: buffers fetched for prefetch: 0 ft: buffers fetched for prefetch (bytes): 0 ft: buffers fetched for prefetch (seconds): 0.000000 ft: buffers fetched for write: 501 ft: buffers fetched for write (bytes): 2763264 ft: buffers fetched for write (seconds): 0.007467 ft: leaf compression to memory (seconds): 1.362411 ft: leaf serialization to memory (seconds): 0.231286 ft: leaf decompression to memory (seconds): 3.132026 ft: leaf deserialization to memory (seconds): 0.957900 ft: nonleaf compression to memory (seconds): 0.303836 ft: nonleaf serialization to memory (seconds): 0.213000 ft: nonleaf decompression to memory (seconds): 0.109690 ft: nonleaf deserialization to memory (seconds): 0.367764 ft: promotion: roots split: 0 ft: promotion: leaf roots injected into: 3215 ft: promotion: h1 roots injected into: 0 ft: promotion: injections at depth 0: 0 ft: promotion: injections at depth 1: 0 ft: promotion: injections at depth 2: 0 ft: promotion: injections at depth 3: 0 ft: promotion: injections lower than depth 3: 0 ft: promotion: stopped because of a nonempty buffer: 0 ft: promotion: stopped at height 1: 0 ft: promotion: stopped because the child was locked or not at all in memory: 0 ft: promotion: stopped because the child was not fully in memory: 0 ft: promotion: stopped anyway, after locking the child: 0 ft: basement nodes deserialized with fixed-keysize: 8635 ft: basement nodes deserialized with variable-keysize: 7809 ft: promotion: succeeded in using the rightmost leaf shortcut: 0 ft: promotion: tried the rightmost leaf shorcut but failed (out-of-bounds): 0 ft: promotion: tried the rightmost leaf shorcut but failed (child reactive): 0 ft: cursor skipped deleted leaf entries: 0 ft flusher: total nodes potentially flushed by cleaner thread: 284 ft flusher: height-one nodes flushed by cleaner thread: 169 ft flusher: height-greater-than-one nodes flushed by cleaner thread: 115 ft flusher: nodes cleaned which had empty buffers: 179 ft flusher: nodes dirtied by cleaner thread: 199 ft flusher: max bytes in a buffer flushed by cleaner thread: 634343 ft flusher: min bytes in a buffer flushed by cleaner thread: 28068 ft flusher: total bytes in buffers flushed by cleaner thread: 14239007 ft flusher: max workdone in a buffer flushed by cleaner thread: 28681 ft flusher: min workdone in a buffer flushed by cleaner thread: 0 ft flusher: total workdone in buffers flushed by cleaner thread: 100880 ft flusher: times cleaner thread tries to merge a leaf: 0 ft flusher: cleaner thread leaf merges in progress: 0 ft flusher: cleaner thread leaf merges successful: 0 ft flusher: nodes dirtied by cleaner thread leaf merges: 0 ft flusher: total number of flushes done by flusher threads or cleaner threads: 105 ft flusher: number of in memory flushes: 105 ft flusher: number of flushes that read something off disk: 0 ft flusher: number of flushes that triggered another flush in child: 0 ft flusher: number of flushes that triggered 1 cascading flush: 0 ft flusher: number of flushes that triggered 2 cascading flushes: 0 ft flusher: number of flushes that triggered 3 cascading flushes: 0 ft flusher: number of flushes that triggered 4 cascading flushes: 0 ft flusher: number of flushes that triggered 5 cascading flushes: 0 ft flusher: number of flushes that triggered over 5 cascading flushes: 0 ft flusher: leaf node splits: 1 ft flusher: nonleaf node splits: 0 ft flusher: leaf node merges: 0 ft flusher: nonleaf node merges: 0 ft flusher: leaf node balances: 0 hot: operations ever started: 0 hot: operations successfully completed: 0 hot: operations aborted: 0 hot: max number of flushes from root ever required to optimize a tree: 0 txn: begin: 21209 txn: begin read only: 773 txn: successful commits: 21926 txn: aborts: 55 logger: next LSN: 28153398478 logger: writes: 1595 logger: writes (bytes): 2621362 logger: writes (uncompressed bytes): 2621362 logger: writes (seconds): 0.078234 logger: number of long logger write operations: 0 indexer: number of indexers successfully created: 0 indexer: number of calls to toku_indexer_create_indexer() that failed: 0 indexer: number of calls to indexer->build() succeeded: 0 indexer: number of calls to indexer->build() failed: 0 indexer: number of calls to indexer->close() that succeeded: 0 indexer: number of calls to indexer->close() that failed: 0 indexer: number of calls to indexer->abort(): 0 indexer: number of indexers currently in existence: 0 indexer: max number of indexers that ever existed simultaneously: 0 loader: number of loaders successfully created: 0 loader: number of calls to toku_loader_create_loader() that failed: 0 loader: number of calls to loader->put() succeeded: 0 loader: number of calls to loader->put() failed: 0 loader: number of calls to loader->close() that succeeded: 0 loader: number of calls to loader->close() that failed: 0 loader: number of calls to loader->abort(): 0 loader: number of loaders currently in existence: 0 loader: max number of loaders that ever existed simultaneously: 0 memory: number of malloc operations: 0 memory: number of free operations: 0 memory: number of realloc operations: 0 memory: number of malloc operations that failed: 0 memory: number of realloc operations that failed: 0 memory: number of bytes requested: 0 memory: number of bytes used (requested + overhead): 0 memory: number of bytes freed: 0 memory: largest attempted allocation size: 0 memory: size of the last failed allocation attempt: 0 memory: estimated maximum memory footprint: 0 memory: mallocator version: 3.6.0-0-g46c0af68bd248b04df75e4f92d5fb804c3d75340 memory: mmap threshold: 4194304 filesystem: ENOSPC redzone state: 0 filesystem: threads currently blocked by full disk: 0 filesystem: number of operations rejected by enospc prevention (red zone): 0 filesystem: most recent disk full: Thu Jan 1 00:00:00 1970 filesystem: number of write operations that returned ENOSPC: 0 filesystem: fsync time: 110384047 filesystem: fsync count: 2921 filesystem: long fsync time: 0 filesystem: long fsync count: 0 context: tree traversals blocked by a full fetch: 155 context: tree traversals blocked by a partial fetch: 144 context: tree traversals blocked by a full eviction: 0 context: tree traversals blocked by a partial eviction: 0 context: tree traversals blocked by a message injection: 0 context: tree traversals blocked by a message application: 3 context: tree traversals blocked by a flush: 0 context: tree traversals blocked by a the cleaner thread: 0 context: tree traversals blocked by something uninstrumented: 110 context: promotion blocked by a full fetch (should never happen): 0 context: promotion blocked by a partial fetch (should never happen): 0 context: promotion blocked by a full eviction (should never happen): 0 context: promotion blocked by a partial eviction (should never happen): 0 context: promotion blocked by a message injection: 0 context: promotion blocked by a message application: 0 context: promotion blocked by a flush: 0 context: promotion blocked by the cleaner thread: 0 context: promotion blocked by something uninstrumented: 0 context: something uninstrumented blocked by something uninstrumented: 53 Memory usage: Arena 0: system bytes = 0 in use bytes = 0 Total (incl. mmap): system bytes = 0 in use bytes = 0 max mmap regions = 0 max mmap bytes = 0 180720 11:54:51 [ERROR] mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see https://mariadb.com/kb/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.1.32-MariaDB-1~xenial key_buffer_size=1073741824 read_buffer_size=2097152 max_used_connections=77 max_threads=102 thread_count=63 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 27998262 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x7501a10d6008 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x75074c9bb0b8 thread_stack 0x48400 /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0xbb03fe8b01e] /usr/sbin/mysqld(handle_fatal_signal+0x2f5)[0xbb03f9d21d5] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7509937be390] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x750992d8d428] /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x750992d8f02a] /usr/lib/mysql/plugin/ha_tokudb.so(+0x130508)[0x75099093f508] /usr/lib/mysql/plugin/ha_tokudb.so(+0x13067e)[0x75099093f67e] /usr/lib/mysql/plugin/ha_tokudb.so(+0x84bf8)[0x750990893bf8] /usr/lib/mysql/plugin/ha_tokudb.so(_Z21toku_hton_assert_failPKcS0_S0_ii+0x88)[0x75099085ffc8] /usr/lib/mysql/plugin/ha_tokudb.so(_ZN9ha_tokudb20open_main_dictionaryEPKcbP13__toku_db_txn+0x2d1)[0x750990879471] /usr/lib/mysql/plugin/ha_tokudb.so(_ZN9ha_tokudb16initialize_shareEPKci+0x1c3)[0x750990885c23] /usr/lib/mysql/plugin/ha_tokudb.so(_ZN9ha_tokudb4openEPKcij+0x4c4)[0x7509908866b4] /usr/sbin/mysqld(_ZN7handler7ha_openEP5TABLEPKcij+0x33)[0xbb03f9d6633] /usr/sbin/mysqld(+0xa2b855)[0xbb03fe5b855] /usr/sbin/mysqld(_ZN7handler7ha_openEP5TABLEPKcij+0x33)[0xbb03f9d6633] /usr/sbin/mysqld(_Z21open_table_from_shareP3THDP11TABLE_SHAREPKcjjjP5TABLEb+0xb29)[0xbb03f8ed869] /usr/sbin/mysqld(_Z10open_tableP3THDP10TABLE_LISTP18Open_table_context+0xade)[0xbb03f7f4dde] /usr/sbin/mysqld(_Z11open_tablesP3THDRK14DDL_options_stPP10TABLE_LISTPjjP19Prelocking_strategy+0xab2)[0xbb03f7f9142] /usr/sbin/mysqld(_Z30open_normal_and_derived_tablesP3THDP10TABLE_LISTjj+0x59)[0xbb03f7f9ca9] /usr/sbin/mysqld(+0x470199)[0xbb03f8a0199] /usr/sbin/mysqld(_Z14get_all_tablesP3THDP10TABLE_LISTP4Item+0x7e0)[0xbb03f8b3300] /usr/sbin/mysqld(_Z24get_schema_tables_resultP4JOIN23enum_schema_table_state+0x296)[0xbb03f8b4df6] /usr/sbin/mysqld(_ZN4JOIN10exec_innerEv+0x71d)[0xbb03f89972d] /usr/sbin/mysqld(_ZN4JOIN4execEv+0x54)[0xbb03f89ba74] /usr/sbin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xe7)[0xbb03f898227] /usr/sbin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x139)[0xbb03f898bf9] /usr/sbin/mysqld(+0x354a78)[0xbb03f784a78] /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x71bb)[0xbb03f8422cb] /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x311)[0xbb03f8444d1] /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x25a0)[0xbb03f847b80] /usr/sbin/mysqld(_Z10do_commandP3THD+0x219)[0xbb03f848439] /usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x182)[0xbb03f916dd2] /usr/sbin/mysqld(handle_one_connection+0x40)[0xbb03f916f70] /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7509937b46ba] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x750992e5f41d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x75092284f020): SHOW TABLE STATUS Connection ID (thread ID): 313 Status: NOT_KILLED Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. Fri Jul 20 11:54:58 2018 PerconaFT recovery starting in env /var/lib/mysql/ Fri Jul 20 11:54:58 2018 PerconaFT recovery scanning backward from 28153398472 Fri Jul 20 11:54:58 2018 PerconaFT recovery bw_begin_checkpoint at 28153388869 timestamp 1532087648116967 (bw_newer) Fri Jul 20 11:54:58 2018 PerconaFT recovery bw_end_checkpoint at 28153375388 timestamp 1532087589455194 xid 28153374688 (bw_newer) Fri Jul 20 11:54:58 2018 PerconaFT recovery bw_begin_checkpoint at 28153374688 timestamp 1532087588115738 (bw_between) Fri Jul 20 11:54:58 2018 PerconaFT recovery turning around at begin checkpoint 28153374688 time 1339456 Fri Jul 20 11:54:58 2018 PerconaFT recovery starts scanning forward to 28153398472 from 28153374688 left 23784 (fw_between) Fri Jul 20 11:54:59 2018 PerconaFT recovery closing 9602 dictionaries Fri Jul 20 11:56:23 2018 PerconaFT recovery making a checkpoint Fri Jul 20 11:56:24 2018 PerconaFT recovery done 2018-07-20 11:56:24 110652200757504 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2018-07-20 11:56:24 110652200757504 [Note] InnoDB: The InnoDB memory heap is disabled 2018-07-20 11:56:24 110652200757504 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2018-07-20 11:56:24 110652200757504 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier 2018-07-20 11:56:24 110652200757504 [Note] InnoDB: Compressed tables use zlib 1.2.8 2018-07-20 11:56:24 110652200757504 [Note] InnoDB: Using Linux native AIO 2018-07-20 11:56:24 110652200757504 [Note] InnoDB: Using SSE crc32 instructions 2018-07-20 11:56:24 110652200757504 [Note] InnoDB: Initializing buffer pool, size = 28.0G 2018-07-20 11:56:25 110652200757504 [Note] InnoDB: Completed initialization of buffer pool 2018-07-20 11:56:25 110652200757504 [Note] InnoDB: Highest supported file format is Barracuda. 2018-07-20 11:56:25 110652200757504 [Note] InnoDB: The log sequence number 1208943385836 in ibdata file do not match the log sequence number 1208943825678 in the ib_logfiles! 2018-07-20 11:56:26 110652200757504 [Note] InnoDB: Restoring possible half-written data pages from the doublewrite buffer... InnoDB: Last MySQL binlog file position 0 10826, file name /var/log/mysql/mariadb-bin.049021 2018-07-20 11:56:28 110652200757504 [Note] InnoDB: 128 rollback segment(s) are active. 2018-07-20 11:56:28 110652200757504 [Note] InnoDB: Waiting for purge to start 2018-07-20 11:56:28 110652200757504 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.38-83.0 started; log sequence number 1208943825678 2018-07-20 11:56:28 110617173968640 [Note] InnoDB: Dumping buffer pool(s) not yet started 2018-07-20 11:56:28 110652200757504 [Note] Plugin 'FEEDBACK' is disabled. 2018-07-20 11:56:28 110652200757504 [Note] Recovering after a crash using /var/log/mysql/mariadb-bin 2018-07-20 11:56:28 110652200757504 [Note] Starting crash recovery... 2018-07-20 11:56:28 110652200757504 [Note] Crash recovery finished. 2018-07-20 11:56:29 110652200757504 [Note] Server socket created on IP: '10.1.34.209'. 2018-07-20 11:56:29 110652199594752 [Note] /usr/sbin/mysqld: Normal shutdown 2018-07-20 11:56:29 110652200757504 [Note] Reading of all Master_info entries succeded 2018-07-20 11:56:29 110652200757504 [Note] Added new Master_info '' to hash table 2018-07-20 11:56:29 110652200757504 [Warning] Neither --relay-log nor --relay-log-index were used; so replication may break when this MySQL server acts as a slave and has his hostname changed!! Please use '--log-basename=#' or '--relay-log=mysqld-relay-bin' to avoid this problem. 2018-07-20 11:56:29 110652148878080 [Note] Slave I/O thread killed while connecting to master 2018-07-20 11:56:29 110652148878080 [Note] Slave I/O thread exiting, read up to log 'mysql-bin.000610', position 3794787 2018-07-20 11:56:29 110652200757504 [Note] /usr/sbin/mysqld: ready for connections. Version: '10.1.32-MariaDB-1~xenial' socket: '/var/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution 2018-07-20 11:56:29 110652059814656 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000610' at position 3794787, relay log './mysqld-relay-bin.001348' position: 537 2018-07-20 11:56:29 110652059814656 [Note] Slave SQL thread exiting, replication stopped in log 'mysql-bin.000610' at position 3794787 2018-07-20 11:56:29 110652199594752 [Note] Event Scheduler: Purging the queue. 0 events 2018-07-20 11:56:29 110617208469248 [Note] InnoDB: FTS optimize thread exiting. 2018-07-20 11:56:29 110652199594752 [Note] InnoDB: Starting shutdown... 2018-07-20 11:56:30 110652199594752 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool 2018-07-20 11:56:32 110652199594752 [Note] InnoDB: Shutdown completed; log sequence number 1208943825688 2018-07-20 11:56:33 110652199594752 [Note] /usr/sbin/mysqld: Shutdown complete 2018-07-20 11:56:40 112313184844032 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2018-07-20 11:56:40 112313184844032 [Note] InnoDB: The InnoDB memory heap is disabled 2018-07-20 11:56:40 112313184844032 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2018-07-20 11:56:40 112313184844032 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier 2018-07-20 11:56:40 112313184844032 [Note] InnoDB: Compressed tables use zlib 1.2.8 2018-07-20 11:56:40 112313184844032 [Note] InnoDB: Using Linux native AIO 2018-07-20 11:56:40 112313184844032 [Note] InnoDB: Using SSE crc32 instructions 2018-07-20 11:56:40 112313184844032 [Note] InnoDB: Initializing buffer pool, size = 28.0G 2018-07-20 11:56:41 112313184844032 [Note] InnoDB: Completed initialization of buffer pool 2018-07-20 11:56:41 112313184844032 [Note] InnoDB: Highest supported file format is Barracuda. 2018-07-20 11:56:42 112313184844032 [Note] InnoDB: 128 rollback segment(s) are active. 2018-07-20 11:56:42 112313184844032 [Note] InnoDB: Waiting for purge to start 2018-07-20 11:56:42 112313184844032 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.38-83.0 started; log sequence number 1208943825688 2018-07-20 11:56:42 112278339090176 [Note] InnoDB: Dumping buffer pool(s) not yet started 2018-07-20 11:56:42 112313184844032 [Note] Plugin 'FEEDBACK' is disabled. 2018-07-20 11:56:42 112313184844032 [Note] Server socket created on IP: '10.1.34.209'. 2018-07-20 11:56:42 112313184844032 [Note] Reading of all Master_info entries succeded 2018-07-20 11:56:42 112313184844032 [Note] Added new Master_info '' to hash table 2018-07-20 11:56:42 112313184844032 [Warning] Neither --relay-log nor --relay-log-index were used; so replication may break when this MySQL server acts as a slave and has his hostname changed!! Please use '--log-basename=#' or '--relay-log=mysqld-relay-bin' to avoid this problem. 2018-07-20 11:56:43 112313184844032 [Note] /usr/sbin/mysqld: ready for connections. Version: '10.1.32-MariaDB-1~xenial' socket: '/var/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution 2018-07-20 11:56:43 112313035815680 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000610' at position 3794787, relay log './mysqld-relay-bin.001348' position: 537 2018-07-20 11:56:43 112311935490816 [Warning] IP address '10.1.34.65' could not be resolved: Name or service not known 2018-07-20 11:56:43 112313036823296 [Note] Slave I/O thread: connected to master 'repl@10.1.143.209:3306',replication started in log 'mysql-bin.000610' at position 3794787 2018-07-20 11:56:43 112311935490816 [Warning] IP address '10.1.34.52' could not be resolved: Name or service not known 2018-07-20 11:56:43 112299437738752 [Warning] IP address '10.1.34.52' could not be resolved: Name or service not known 2018-07-20 11:56:43 112313037511424 [Warning] IP address '10.1.34.61' could not be resolved: Name or service not known 2018-07-20 11:56:43 112295490644736 [Warning] IP address '10.1.34.61' could not be resolved: Name or service not known 2018-07-20 11:56:43 112299436624640 [Warning] IP address '10.1.201.50' could not be resolved: Name or service not known 2018-07-20 11:56:43 112313036348160 [Warning] IP address '10.1.201.50' could not be resolved: Name or service not known 2018-07-20 11:56:43 112313035066112 [Warning] IP address '10.1.143.209' could not be resolved: Name or service not known 2018-07-20 11:56:43 112291544021760 [Warning] IP address '10.1.34.218' could not be resolved: Name or service not known 2018-07-20 11:56:43 112287596022528 [Warning] IP address '10.1.31.61' could not be resolved: Name or service not known 2018-07-20 11:56:44 112313035434752 [Warning] IP address '10.1.44.50' could not be resolved: Name or service not known 2018-07-20 11:56:44 112287596022528 [Warning] IP address '10.1.44.50' could not be resolved: Name or service not known 2018-07-20 11:56:44 112313035066112 [Note] Start binlog_dump to slave_server(2), pos(mariadb-bin.049023, 4088)