2018-04-18 9:28:17 140617100650624 [Note] Using unique option prefix 'myisam-recover' is error-prone and can break in the future. Please use the full name 'myisam-recover-options' instead. 2018-04-18 9:28:17 140617100650624 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2018-04-18 9:28:17 140617100650624 [Note] InnoDB: Uses event mutexes 2018-04-18 9:28:17 140617100650624 [Note] InnoDB: Compressed tables use zlib 1.2.7 2018-04-18 9:28:17 140617100650624 [Note] InnoDB: Using Linux native AIO 2018-04-18 9:28:17 140617100650624 [Note] InnoDB: Number of pools: 1 2018-04-18 9:28:17 140617100650624 [Note] InnoDB: Using SSE2 crc32 instructions 2018-04-18 9:28:17 140617100650624 [Note] InnoDB: Initializing buffer pool, total size = 512M, instances = 1, chunk size = 128M 2018-04-18 9:28:17 140617100650624 [Note] InnoDB: Completed initialization of buffer pool 2018-04-18 9:28:17 140615713834752 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2018-04-18 9:28:17 140617100650624 [Note] InnoDB: Highest supported file format is Barracuda. 2018-04-18 9:28:23 140617100650624 [Note] InnoDB: 128 out of 128 rollback segments are active. 2018-04-18 9:28:23 140617100650624 [Note] InnoDB: Creating shared tablespace for temporary tables 2018-04-18 9:28:23 140617100650624 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2018-04-18 9:28:23 140617100650624 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2018-04-18 9:28:23 140617100650624 [Note] InnoDB: Waiting for purge to start 2018-04-18 9:28:23 140617100650624 [Note] InnoDB: 5.7.21 started; log sequence number 7520113289 2018-04-18 9:28:23 140615806613248 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2018-04-18 9:28:23 140617100650624 [Note] Plugin 'FEEDBACK' is disabled. 2018-04-18 9:28:23 140617100650624 [Note] Server socket created on IP: '0.0.0.0'. 2018-04-18 9:28:23 140617100650624 [Warning] 'user' entry 'root@multi2.wadaro.com' ignored in --skip-name-resolve mode. 2018-04-18 9:28:23 140617100650624 [Warning] 'proxies_priv' entry '@% root@multi2.wadaro.com' ignored in --skip-name-resolve mode. 2018-04-18 9:28:23 140617100650624 [Note] /usr/sbin/mysqld: ready for connections. Version: '10.2.14-MariaDB' socket: '/var/lib/mysql/mysqld.sock' port: 3306 MariaDB Server 2018-04-18 9:30:10 140615806613248 [Note] InnoDB: Buffer pool(s) load completed at 180418 9:30:10 2018-04-18 9:36:22 140616634459904 [ERROR] InnoDB: ibuf cursor restoration fails!. ibuf record inserted to page 609:23 2018-04-18 9:36:22 140616634459904 [ERROR] InnoDB: Submit a detailed bug report to https://jira.mariadb.org/ PHYSICAL RECORD: n_fields 1; 1-byte offsets; info bits 0 0: len 8; hex 696e66696d756d00; asc infimum ;; PHYSICAL RECORD: n_fields 6; 1-byte offsets; info bits 32 0: len 4; hex 00000261; asc a;; 1: len 1; hex 00; asc ;; 2: len 4; hex 00000017; asc ;; 3: len 16; hex 023d01010c0f003c00c0860300048000; asc = < ;; 4: SQL NULL, size 0 ; 5: len 4; hex 8000028f; asc ;; DATA TUPLE: 3 fields; 0: len 4; hex 00000261; asc a;; 1: len 1; hex 00; asc ;; 2: len 4; hex 00000000; asc ;; PHYSICAL RECORD: n_fields 6; 1-byte offsets; info bits 32 0: len 4; hex 00000261; asc a;; 1: len 1; hex 00; asc ;; 2: len 4; hex 00000017; asc ;; 3: len 16; hex 023d01010c0f003c00c0860300048000; asc = < ;; 4: SQL NULL, size 0 ; 5: len 4; hex 8000028f; asc ;; 2018-04-18 9:36:22 140616634459904 [ERROR] [FATAL] InnoDB: Failed to restore ibuf position. 180418 9:36:22 [ERROR] mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see https://mariadb.com/kb/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.2.14-MariaDB key_buffer_size=16777216 read_buffer_size=131072 max_used_connections=7 max_threads=153 thread_count=13 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 352557 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x7fe378000a88 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7fe3dc8bdd70 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x5641a76fdcce] /usr/sbin/mysqld(handle_fatal_signal+0x355)[0x5641a7189605] sigaction.c:0(__restore_rt)[0x7fe3f816a5e0] :0(__GI_raise)[0x7fe3f66771f7] :0(__GI_abort)[0x7fe3f66788e8] /usr/sbin/mysqld(+0x9ae963)[0x5641a74c4963] ut/ut0ut.cc:792(ib::fatal::~fatal())[0x5641a739850a] ibuf/ibuf0ibuf.cc:4296(ibuf_restore_pos(unsigned long, unsigned long, dtuple_t const*, unsigned long, btr_pcur_t*, mtr_t*))[0x5641a7399f18] ibuf/ibuf0ibuf.cc:4396(ibuf_delete_rec(unsigned long, unsigned long, btr_pcur_t*, dtuple_t const*, mtr_t*))[0x5641a739b5d9] fil/fil0fil.cc:3234(fil_reinit_space_header_for_table(dict_table_t*, unsigned long, trx_t*))[0x5641a757356f] row/row0trunc.cc:2038(row_truncate_table_for_mysql(dict_table_t*, trx_t*))[0x5641a7457399] handler/ha_innodb.cc:13192(ha_innobase::truncate())[0x5641a7371f08] sql/sql_truncate.cc:261(Sql_cmd_truncate_table::handler_truncate(THD*, TABLE_LIST*, bool))[0x5641a72f6e60] sql/sql_truncate.cc:449(Sql_cmd_truncate_table::truncate_table(THD*, TABLE_LIST*))[0x5641a72f72f0] sql/sql_truncate.cc:498(Sql_cmd_truncate_table::execute(THD*))[0x5641a72f7396] sql/sql_parse.cc:6220(mysql_execute_command(THD*))[0x5641a6ff3d0b] sql/sql_parse.cc:7915(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x5641a6ffbb2e] sql/sql_parse.cc:1815(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x5641a6ffeb33] sql/sql_parse.cc:1371(do_command(THD*))[0x5641a6fff76b] sql/sql_connect.cc:1335(do_handle_one_connection(CONNECT*))[0x5641a70c6d3a] sql/sql_connect.cc:1243(handle_one_connection)[0x5641a70c6e5d] pthread_create.c:0(start_thread)[0x7fe3f8162e25] /lib64/libc.so.6(clone+0x6d)[0x7fe3f673a34d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x7fe37800f170): truncate aggregation_cause_kpi_fleet_day_at Connection ID (thread ID): 304 Status: NOT_KILLED Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 2018-04-18 9:36:30 140209371834496 [Note] Using unique option prefix 'myisam-recover' is error-prone and can break in the future. Please use the full name 'myisam-recover-options' instead. Wed Apr 18 09:36:30 2018 PerconaFT recovery starting in env /var/lib/mysql/ Wed Apr 18 09:36:30 2018 PerconaFT recovery scanning backward from 84625 Wed Apr 18 09:36:30 2018 PerconaFT recovery bw_end_checkpoint at 84625 timestamp 1524044177686707 xid 84621 (bw_newer) Wed Apr 18 09:36:30 2018 PerconaFT recovery bw_begin_checkpoint at 84621 timestamp 1524044177686673 (bw_between) Wed Apr 18 09:36:30 2018 PerconaFT recovery turning around at begin checkpoint 84621 time 34 Wed Apr 18 09:36:30 2018 PerconaFT recovery starts scanning forward to 84625 from 84621 left 4 (fw_between) Wed Apr 18 09:36:30 2018 PerconaFT recovery closing 2 dictionaries Wed Apr 18 09:36:30 2018 PerconaFT recovery making a checkpoint Wed Apr 18 09:36:30 2018 PerconaFT recovery done 2018-04-18 9:36:30 140209371834496 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2018-04-18 9:36:30 140209371834496 [Note] InnoDB: Uses event mutexes 2018-04-18 9:36:30 140209371834496 [Note] InnoDB: Compressed tables use zlib 1.2.7 2018-04-18 9:36:30 140209371834496 [Note] InnoDB: Using Linux native AIO 2018-04-18 9:36:30 140209371834496 [Note] InnoDB: Number of pools: 1 2018-04-18 9:36:30 140209371834496 [Note] InnoDB: Using SSE2 crc32 instructions 2018-04-18 9:36:30 140209371834496 [Note] InnoDB: Initializing buffer pool, total size = 512M, instances = 1, chunk size = 128M 2018-04-18 9:36:30 140209371834496 [Note] InnoDB: Completed initialization of buffer pool 2018-04-18 9:36:30 140207986013952 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2018-04-18 9:36:30 140209371834496 [Note] InnoDB: Highest supported file format is Barracuda. 2018-04-18 9:36:30 140209371834496 [Note] InnoDB: Starting crash recovery from checkpoint LSN=7646325062 2018-04-18 9:36:37 140209371834496 [Note] InnoDB: 1 transaction(s) which must be rolled back or cleaned up in total 0 row operations to undo 2018-04-18 9:36:37 140209371834496 [Note] InnoDB: Trx id counter is 4863488 2018-04-18 9:36:37 140209371834496 [Note] InnoDB: Starting final batch to recover 524 pages from redo log. 2018-04-18 9:36:38 140209371834496 [Note] InnoDB: Rolled back recovered transaction 4863058 2018-04-18 9:36:41 140209371834496 [Note] InnoDB: Completing truncate for table with id (661) residing in file-per-table tablespace with id (609) 2018-04-18 9:36:41 140209371834496 [Note] InnoDB: 128 out of 128 rollback segments are active. 2018-04-18 9:36:41 140209371834496 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1" 2018-04-18 9:36:41 140209371834496 [Note] InnoDB: Creating shared tablespace for temporary tables 2018-04-18 9:36:41 140209371834496 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2018-04-18 9:36:41 140209371834496 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2018-04-18 9:36:41 140209371834496 [Note] InnoDB: Waiting for purge to start 2018-04-18 9:36:41 140209371834496 [Note] InnoDB: 5.7.21 started; log sequence number 7648567406 2018-04-18 9:36:41 140207428974336 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2018-04-18 9:36:41 140209371834496 [Note] Plugin 'FEEDBACK' is disabled. 2018-04-18 9:36:41 140209371834496 [Note] Recovering after a crash using tc.log 2018-04-18 9:36:41 140209371834496 [Note] Starting crash recovery... 2018-04-18 9:36:41 140209371834496 [Note] Crash recovery finished. 2018-04-18 9:36:41 140209371834496 [Note] Server socket created on IP: '0.0.0.0'. 2018-04-18 9:36:41 140209371834496 [Warning] 'user' entry 'root@multi2.wadaro.com' ignored in --skip-name-resolve mode. 2018-04-18 9:36:41 140209371834496 [Warning] 'proxies_priv' entry '@% root@multi2.wadaro.com' ignored in --skip-name-resolve mode. 2018-04-18 9:36:41 140209371834496 [Note] /usr/sbin/mysqld: ready for connections. Version: '10.2.14-MariaDB' socket: '/var/lib/mysql/mysqld.sock' port: 3306 MariaDB Server 2018-04-18 9:38:17 140207428974336 [Note] InnoDB: Buffer pool(s) load completed at 180418 9:38:17