Feb 17 09:18:09 optdb01 mysqld: /home/buildbot/buildbot/build/mariadb-10.0.23/storage/tokudb/PerconaFT/locktree/treenode.cc:338 remove: Assertion `child' failed (errno=2) Feb 17 09:18:09 optdb01 mysqld: : No such file or directory Feb 17 09:18:09 optdb01 mysqld: Backtrace: (Note: toku_do_assert=0x0x7fd2e2c93a30) Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(_Z19db_env_do_backtraceP8_IO_FILE+0x1b)[0x7fd2e2c901fb] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x8e323)[0x7fd2e2c90323] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x8e39a)[0x7fd2e2c9039a] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x117d17)[0x7fd2e2d19d17] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x117c1e)[0x7fd2e2d19c1e] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x117f6e)[0x7fd2e2d19f6e] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x119342)[0x7fd2e2d1b342] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x112635)[0x7fd2e2d14635] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x11292f)[0x7fd2e2d1492f] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x1133f5)[0x7fd2e2d153f5] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x1136eb)[0x7fd2e2d156eb] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(_Z24toku_db_start_range_lockP9__toku_dbP13__toku_db_txnPK10__toku_dbtS5_N4toku12lock_request4typeEPS7_+0xdc)[0x7fd2e2d1ea6c] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(_Z22toku_db_get_range_lockP9__toku_dbP13__toku_db_txnPK10__toku_dbtS5_N4toku12lock_request4typeE+0xe6)[0x7fd2e2d1ebf6] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(_Z19env_update_multipleP13__toku_db_envP9__toku_dbP13__toku_db_txnP10__toku_dbtS6_S6_S6_jPS2_PjjP9DBT_ARRAYjSA_+0x838)[0x7fd2e2d241a8] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(_ZN9ha_tokudb10update_rowEPKhPh+0x572)[0x7fd2e2c6b8a2] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN7handler13ha_update_rowEPKhPh+0x412)[0x745172] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN12multi_update9send_dataER4ListI4ItemE+0x2f9)[0x66c419] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld[0x5fa69d] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld[0x5fc9d7] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x176)[0x60b346] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld[0x5fc9d7] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x1bc)[0x60b38c] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld[0x61686d] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN4JOIN10exec_innerEv+0xa3a)[0x627caa] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN4JOIN4execEv+0x11)[0x629af1] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xc7)[0x6264f7] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_Z18mysql_multi_updateP3THDP10TABLE_LISTP4ListI4ItemES6_PS4_y15enum_duplicatesbP18st_select_lex_unitP13st_select_lexPP12multi_update+0x1af)[0x66ea9f] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x4e81)[0x5d6701] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x34)[0x8527d4] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN13sp_lex_keeper23reset_lex_and_exec_coreEP3THDPjbP8sp_instr+0x80)[0x859c40] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN13sp_instr_stmt7executeEP3THDPj+0x155)[0x85a065] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN7sp_head7executeEP3THDb+0x6be)[0x856d8e] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x664)[0x858374] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x1d41)[0x5d35c1] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x34)[0x8527d4] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN13sp_lex_keeper23reset_lex_and_exec_coreEP3THDPjbP8sp_instr+0x80)[0x859c40] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN13sp_instr_stmt7executeEP3THDPj+0x155)[0x85a065] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN7sp_head7executeEP3THDb+0x6be)[0x856d8e] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x664)[0x858374] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x1d41)[0x5d35c1] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x34)[0x8527d4] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN13sp_lex_keeper23reset_lex_and_exec_coreEP3THDPjbP8sp_instr+0x80)[0x859c40] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN13sp_instr_stmt7executeEP3THDPj+0x155)[0x85a065] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN7sp_head7executeEP3THDb+0x6be)[0x856d8e] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x664)[0x858374] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(_ZN14Event_job_data7executeEP3THDb+0x799)[0x68dc79] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(event_worker_thread+0x109)[0x88de39] Feb 17 09:18:09 optdb01 mysqld: /lib64/libpthread.so.0(+0x7dc5)[0x7fd2ff446dc5] Feb 17 09:18:09 optdb01 mysqld: /lib64/libc.so.6(clone+0x6d)[0x7fd2fdcc321d] Feb 17 09:18:09 optdb01 mysqld: Engine status: Feb 17 09:18:09 optdb01 mysqld: BUILD_ID = 0 Feb 17 09:18:09 optdb01 mysqld: Env panic code: 2 Feb 17 09:18:09 optdb01 mysqld: Env panic string: /home/buildbot/buildbot/build/mariadb-10.0.23/storage/tokudb/PerconaFT/locktree/treenode.cc:338 remove: Assertion `child' failed (errno=2) Feb 17 09:18:09 optdb01 mysqld: Feb 17 09:18:09 optdb01 mysqld: time of environment creation: Mon Oct 12 15:46:17 2015 Feb 17 09:18:09 optdb01 mysqld: time of engine startup: Fri Feb 12 13:38:14 2016 Feb 17 09:18:09 optdb01 mysqld: time now: Wed Feb 17 09:18:09 2016 Feb 17 09:18:09 optdb01 mysqld: db opens: 78524 Feb 17 09:18:09 optdb01 mysqld: db closes: 77250 Feb 17 09:18:09 optdb01 mysqld: num open dbs now: 1274 Feb 17 09:18:09 optdb01 mysqld: max open dbs: 1332 Feb 17 09:18:09 optdb01 mysqld: period, in ms, that recovery log is automatically fsynced: 0 Feb 17 09:18:09 optdb01 mysqld: dictionary inserts: 10868154 Feb 17 09:18:09 optdb01 mysqld: dictionary inserts fail: 383844702 Feb 17 09:18:09 optdb01 mysqld: dictionary deletes: 18739 Feb 17 09:18:09 optdb01 mysqld: dictionary deletes fail: 0 Feb 17 09:18:09 optdb01 mysqld: dictionary updates: 0 Feb 17 09:18:09 optdb01 mysqld: dictionary updates fail: 0 Feb 17 09:18:09 optdb01 mysqld: dictionary broadcast updates: 0 Feb 17 09:18:09 optdb01 mysqld: dictionary broadcast updates fail: 0 Feb 17 09:18:09 optdb01 mysqld: dictionary multi inserts: 1204674407 Feb 17 09:18:09 optdb01 mysqld: dictionary multi inserts fail: 7392867 Feb 17 09:18:09 optdb01 mysqld: dictionary multi deletes: 45288797 Feb 17 09:18:09 optdb01 mysqld: dictionary multi deletes fail: 0 Feb 17 09:18:09 optdb01 mysqld: dictionary updates multi: 4726837421 Feb 17 09:18:09 optdb01 mysqld: dictionary updates multi fail: 0 Feb 17 09:18:09 optdb01 mysqld: le: max committed xr: 6 Feb 17 09:18:09 optdb01 mysqld: le: max provisional xr: 3 Feb 17 09:18:09 optdb01 mysqld: le: expanded: 0 Feb 17 09:18:09 optdb01 mysqld: le: max memsize: 73077 Feb 17 09:18:09 optdb01 mysqld: le: size of leafentries before garbage collection (during message application): 566107532335 Feb 17 09:18:09 optdb01 mysqld: le: size of leafentries after garbage collection (during message application): 322838403284 Feb 17 09:18:09 optdb01 mysqld: le: size of leafentries before garbage collection (outside message application): 0 Feb 17 09:18:09 optdb01 mysqld: le: size of leafentries after garbage collection (outside message application): 0 Feb 17 09:18:09 optdb01 mysqld: checkpoint: period: 60 Feb 17 09:18:09 optdb01 mysqld: checkpoint: footprint: 0 Feb 17 09:18:09 optdb01 mysqld: checkpoint: last checkpoint began: Wed Feb 17 09:17:16 2016 Feb 17 09:18:09 optdb01 mysqld: checkpoint: last complete checkpoint began: Wed Feb 17 09:17:16 2016 Feb 17 09:18:09 optdb01 mysqld: checkpoint: last complete checkpoint ended: Wed Feb 17 09:17:26 2016 Feb 17 09:18:09 optdb01 mysqld: checkpoint: time spent during checkpoint (begin and end phases): 36897 Feb 17 09:18:09 optdb01 mysqld: checkpoint: time spent during last checkpoint (begin and end phases): 10 Feb 17 09:18:09 optdb01 mysqld: checkpoint: last complete checkpoint LSN: 120529185810 Feb 17 09:18:09 optdb01 mysqld: checkpoint: checkpoints taken : 6940 Feb 17 09:18:09 optdb01 mysqld: checkpoint: checkpoints failed: 0 Feb 17 09:18:09 optdb01 mysqld: checkpoint: waiters now: 0 Feb 17 09:18:09 optdb01 mysqld: checkpoint: waiters max: 0 Feb 17 09:18:09 optdb01 mysqld: checkpoint: non-checkpoint client wait on mo lock: 3126 Feb 17 09:18:09 optdb01 mysqld: checkpoint: non-checkpoint client wait on cs lock: 0 Feb 17 09:18:09 optdb01 mysqld: checkpoint: checkpoint begin time: 214915851 Feb 17 09:18:09 optdb01 mysqld: checkpoint: long checkpoint begin time: 0 Feb 17 09:18:09 optdb01 mysqld: checkpoint: long checkpoint begin count: 0 Feb 17 09:18:09 optdb01 mysqld: checkpoint: checkpoint end time: 36380599282 Feb 17 09:18:09 optdb01 mysqld: checkpoint: long checkpoint end time: 0 Feb 17 09:18:09 optdb01 mysqld: checkpoint: long checkpoint end count: 0 Feb 17 09:18:09 optdb01 mysqld: cachetable: miss: 74370 Feb 17 09:18:09 optdb01 mysqld: cachetable: miss time: 205217984 Feb 17 09:18:09 optdb01 mysqld: cachetable: prefetches: 1974 Feb 17 09:18:09 optdb01 mysqld: cachetable: size current: 8334055987 Feb 17 09:18:09 optdb01 mysqld: cachetable: size limit: 8866320384 Feb 17 09:18:09 optdb01 mysqld: cachetable: size writing: 0 Feb 17 09:18:09 optdb01 mysqld: cachetable: size nonleaf: 525454166 Feb 17 09:18:09 optdb01 mysqld: cachetable: size leaf: 7671614045 Feb 17 09:18:09 optdb01 mysqld: cachetable: size rollback: 136987776 Feb 17 09:18:09 optdb01 mysqld: cachetable: size cachepressure: 570815043 Feb 17 09:18:09 optdb01 mysqld: cachetable: size currently cloned data for checkpoint: 0 Feb 17 09:18:09 optdb01 mysqld: cachetable: evictions: 133272 Feb 17 09:18:09 optdb01 mysqld: cachetable: cleaner executions: 488109 Feb 17 09:18:09 optdb01 mysqld: cachetable: cleaner period: 1 Feb 17 09:18:09 optdb01 mysqld: cachetable: cleaner iterations: 5 Feb 17 09:18:09 optdb01 mysqld: cachetable: number of waits on cache pressure: 0 Feb 17 09:18:09 optdb01 mysqld: cachetable: time waiting on cache pressure: 0 Feb 17 09:18:09 optdb01 mysqld: cachetable: number of long waits on cache pressure: 0 Feb 17 09:18:09 optdb01 mysqld: cachetable: long time waiting on cache pressure: 0 Feb 17 09:18:09 optdb01 mysqld: cachetable: number of threads in pool: 4 Feb 17 09:18:09 optdb01 mysqld: cachetable: number of currently active threads in pool: 0 Feb 17 09:18:09 optdb01 mysqld: cachetable: number of currently queued work items: 0 Feb 17 09:18:09 optdb01 mysqld: cachetable: largest number of queued work items: 2 Feb 17 09:18:09 optdb01 mysqld: cachetable: total number of work items processed: 455785 Feb 17 09:18:09 optdb01 mysqld: cachetable: total execution time of processing work items: 4814503 Feb 17 09:18:09 optdb01 mysqld: cachetable: number of threads in pool: 8 Feb 17 09:18:09 optdb01 mysqld: cachetable: number of currently active threads in pool: 0 Feb 17 09:18:09 optdb01 mysqld: cachetable: number of currently queued work items: 0 Feb 17 09:18:09 optdb01 mysqld: cachetable: largest number of queued work items: 73 Feb 17 09:18:09 optdb01 mysqld: cachetable: total number of work items processed: 52808 Feb 17 09:18:09 optdb01 mysqld: cachetable: total execution time of processing work items: 296748 Feb 17 09:18:09 optdb01 mysqld: cachetable: number of threads in pool: 1 Feb 17 09:18:09 optdb01 mysqld: cachetable: number of currently active threads in pool: 0 Feb 17 09:18:09 optdb01 mysqld: cachetable: number of currently queued work items: 0 Feb 17 09:18:09 optdb01 mysqld: cachetable: largest number of queued work items: 72 Feb 17 09:18:09 optdb01 mysqld: cachetable: total number of work items processed: 232320 Feb 17 09:18:09 optdb01 mysqld: cachetable: total execution time of processing work items: 6097528 Feb 17 09:18:09 optdb01 mysqld: locktree: memory size: 520130638 Feb 17 09:18:09 optdb01 mysqld: locktree: memory size limit: 1041181184 Feb 17 09:18:09 optdb01 mysqld: locktree: number of times lock escalation ran: 40 Feb 17 09:18:09 optdb01 mysqld: locktree: time spent running escalation (seconds): 86.928427 Feb 17 09:18:09 optdb01 mysqld: locktree: latest post-escalation memory size: 4211936 Feb 17 09:18:09 optdb01 mysqld: locktree: number of locktrees open now: 1276 Feb 17 09:18:09 optdb01 mysqld: locktree: number of pending lock requests: 1 Feb 17 09:18:09 optdb01 mysqld: locktree: number of locktrees eligible for the STO: 0 Feb 17 09:18:09 optdb01 mysqld: locktree: number of times a locktree ended the STO early: 3259 Feb 17 09:18:09 optdb01 mysqld: locktree: time spent ending the STO early (seconds): 196.743463 Feb 17 09:18:09 optdb01 mysqld: locktree: number of wait locks: 446 Feb 17 09:18:09 optdb01 mysqld: locktree: time waiting for locks: 694950993 Feb 17 09:18:09 optdb01 mysqld: locktree: number of long wait locks: 125 Feb 17 09:18:09 optdb01 mysqld: locktree: long time waiting for locks: 620157538 Feb 17 09:18:09 optdb01 mysqld: locktree: number of lock timeouts: 9 Feb 17 09:18:09 optdb01 mysqld: locktree: number of waits on lock escalation: 85 Feb 17 09:18:09 optdb01 mysqld: locktree: time waiting on lock escalation: 184332378 Feb 17 09:18:09 optdb01 mysqld: locktree: number of long waits on lock escalation: 85 Feb 17 09:18:09 optdb01 mysqld: locktree: long time waiting on lock escalation: 184332378 Feb 17 09:18:09 optdb01 mysqld: ft: dictionary updates: 0 Feb 17 09:18:09 optdb01 mysqld: ft: dictionary broadcast updates: 0 Feb 17 09:18:09 optdb01 mysqld: ft: descriptor set: 17100 Feb 17 09:18:09 optdb01 mysqld: ft: messages ignored by leaf due to msn: 1980402890 Feb 17 09:18:09 optdb01 mysqld: ft: total search retries due to TRY_AGAIN: 2940632 Feb 17 09:18:09 optdb01 mysqld: ft: searches requiring more tries than the height of the tree: 189 Feb 17 09:18:09 optdb01 mysqld: ft: searches requiring more tries than the height of the tree plus three: 130 Feb 17 09:18:09 optdb01 mysqld: ft: leaf nodes flushed to disk (not for checkpoint): 18074 Feb 17 09:18:09 optdb01 mysqld: ft: leaf nodes flushed to disk (not for checkpoint) (bytes): 138286080 Feb 17 09:18:09 optdb01 mysqld: ft: leaf nodes flushed to disk (not for checkpoint) (uncompressed bytes): 1079864083 Feb 17 09:18:09 optdb01 mysqld: ft: leaf nodes flushed to disk (not for checkpoint) (seconds): 1.541877 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf nodes flushed to disk (not for checkpoint): 257 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf nodes flushed to disk (not for checkpoint) (bytes): 53372416 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf nodes flushed to disk (not for checkpoint) (uncompressed bytes): 329393321 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf nodes flushed to disk (not for checkpoint) (seconds): 0.253553 Feb 17 09:18:09 optdb01 mysqld: ft: leaf nodes flushed to disk (for checkpoint): 610896 Feb 17 09:18:09 optdb01 mysqld: ft: leaf nodes flushed to disk (for checkpoint) (bytes): 182649473024 Feb 17 09:18:09 optdb01 mysqld: ft: leaf nodes flushed to disk (for checkpoint) (uncompressed bytes): 1043441964945 Feb 17 09:18:09 optdb01 mysqld: ft: leaf nodes flushed to disk (for checkpoint) (seconds): 556.568653 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf nodes flushed to disk (for checkpoint): 481245 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf nodes flushed to disk (for checkpoint) (bytes): 55014569472 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf nodes flushed to disk (for checkpoint) (uncompressed bytes): 284675842076 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf nodes flushed to disk (for checkpoint) (seconds): 216.262946 Feb 17 09:18:09 optdb01 mysqld: ft: uncompressed / compressed bytes written (leaf): UNKNOWN STATUS TYPE: 6 Feb 17 09:18:09 optdb01 mysqld: ft: uncompressed / compressed bytes written (nonleaf): UNKNOWN STATUS TYPE: 6 Feb 17 09:18:09 optdb01 mysqld: ft: uncompressed / compressed bytes written (overall): UNKNOWN STATUS TYPE: 6 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf node partial evictions: 204230 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf node partial evictions (bytes): 1261364097 Feb 17 09:18:09 optdb01 mysqld: ft: leaf node partial evictions: 4597072 Feb 17 09:18:09 optdb01 mysqld: ft: leaf node partial evictions (bytes): 312710384841 Feb 17 09:18:09 optdb01 mysqld: ft: leaf node full evictions: 109499 Feb 17 09:18:09 optdb01 mysqld: ft: leaf node full evictions (bytes): 207461906234 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf node full evictions: 9046 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf node full evictions (bytes): 20760992111 Feb 17 09:18:09 optdb01 mysqld: ft: leaf nodes created: 89778 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf nodes created: 7481 Feb 17 09:18:09 optdb01 mysqld: ft: leaf nodes destroyed: 0 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf nodes destroyed: 0 Feb 17 09:18:09 optdb01 mysqld: ft: bytes of messages injected at root (all trees): 247517365688 Feb 17 09:18:09 optdb01 mysqld: ft: bytes of messages flushed from h1 nodes to leaves: 235663487694 Feb 17 09:18:09 optdb01 mysqld: ft: bytes of messages currently in trees (estimate): 11853877994 Feb 17 09:18:09 optdb01 mysqld: ft: messages injected at root: 3800842747 Feb 17 09:18:09 optdb01 mysqld: ft: broadcast messages injected at root: 0 Feb 17 09:18:09 optdb01 mysqld: ft: basements decompressed as a target of a query: 35 Feb 17 09:18:09 optdb01 mysqld: ft: basements decompressed for prelocked range: 0 Feb 17 09:18:09 optdb01 mysqld: ft: basements decompressed for prefetch: 0 Feb 17 09:18:09 optdb01 mysqld: ft: basements decompressed for write: 582 Feb 17 09:18:09 optdb01 mysqld: ft: buffers decompressed as a target of a query: 11973 Feb 17 09:18:09 optdb01 mysqld: ft: buffers decompressed for prelocked range: 475 Feb 17 09:18:09 optdb01 mysqld: ft: buffers decompressed for prefetch: 0 Feb 17 09:18:09 optdb01 mysqld: ft: buffers decompressed for write: 198829 Feb 17 09:18:09 optdb01 mysqld: ft: pivots fetched for query: 44260 Feb 17 09:18:09 optdb01 mysqld: ft: pivots fetched for query (bytes): 430127616 Feb 17 09:18:09 optdb01 mysqld: ft: pivots fetched for query (seconds): 66.753970 Feb 17 09:18:09 optdb01 mysqld: ft: pivots fetched for prefetch: 1974 Feb 17 09:18:09 optdb01 mysqld: ft: pivots fetched for prefetch (bytes): 63636480 Feb 17 09:18:09 optdb01 mysqld: ft: pivots fetched for prefetch (seconds): 8.357904 Feb 17 09:18:09 optdb01 mysqld: ft: pivots fetched for write: 5599 Feb 17 09:18:09 optdb01 mysqld: ft: pivots fetched for write (bytes): 42354688 Feb 17 09:18:09 optdb01 mysqld: ft: pivots fetched for write (seconds): 2.977774 Feb 17 09:18:09 optdb01 mysqld: ft: basements fetched as a target of a query: 1191279 Feb 17 09:18:09 optdb01 mysqld: ft: basements fetched as a target of a query (bytes): 18478119936 Feb 17 09:18:09 optdb01 mysqld: ft: basements fetched as a target of a query (seconds): 2394.383495 Feb 17 09:18:09 optdb01 mysqld: ft: basements fetched for prelocked range: 1518914 Feb 17 09:18:09 optdb01 mysqld: ft: basements fetched for prelocked range (bytes): 2766495744 Feb 17 09:18:09 optdb01 mysqld: ft: basements fetched for prelocked range (seconds): 36.265098 Feb 17 09:18:09 optdb01 mysqld: ft: basements fetched for prefetch: 393122 Feb 17 09:18:09 optdb01 mysqld: ft: basements fetched for prefetch (bytes): 2788533760 Feb 17 09:18:09 optdb01 mysqld: ft: basements fetched for prefetch (seconds): 52.415965 Feb 17 09:18:09 optdb01 mysqld: ft: basements fetched for write: 1728537 Feb 17 09:18:09 optdb01 mysqld: ft: basements fetched for write (bytes): 33802406400 Feb 17 09:18:09 optdb01 mysqld: ft: basements fetched for write (seconds): 766.349247 Feb 17 09:18:09 optdb01 mysqld: ft: buffers fetched as a target of a query: 1144 Feb 17 09:18:09 optdb01 mysqld: ft: buffers fetched as a target of a query (bytes): 653824 Feb 17 09:18:09 optdb01 mysqld: ft: buffers fetched as a target of a query (seconds): 2.171251 Feb 17 09:18:09 optdb01 mysqld: ft: buffers fetched for prelocked range: 217 Feb 17 09:18:09 optdb01 mysqld: ft: buffers fetched for prelocked range (bytes): 148992 Feb 17 09:18:09 optdb01 mysqld: ft: buffers fetched for prelocked range (seconds): 0.000466 Feb 17 09:18:09 optdb01 mysqld: ft: buffers fetched for prefetch: 0 Feb 17 09:18:09 optdb01 mysqld: ft: buffers fetched for prefetch (bytes): 0 Feb 17 09:18:09 optdb01 mysqld: ft: buffers fetched for prefetch (seconds): 0.000000 Feb 17 09:18:09 optdb01 mysqld: ft: buffers fetched for write: 7716 Feb 17 09:18:09 optdb01 mysqld: ft: buffers fetched for write (bytes): 38720512 Feb 17 09:18:09 optdb01 mysqld: ft: buffers fetched for write (seconds): 2.240417 Feb 17 09:18:09 optdb01 mysqld: ft: leaf compression to memory (seconds): 20576.846139 Feb 17 09:18:09 optdb01 mysqld: ft: leaf serialization to memory (seconds): 2346.980932 Feb 17 09:18:09 optdb01 mysqld: ft: leaf decompression to memory (seconds): 821.444135 Feb 17 09:18:09 optdb01 mysqld: ft: leaf deserialization to memory (seconds): 574.312437 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf compression to memory (seconds): 6150.237503 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf serialization to memory (seconds): 623.160842 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf decompression to memory (seconds): 2.808899 Feb 17 09:18:09 optdb01 mysqld: ft: nonleaf deserialization to memory (seconds): 8.112770 Feb 17 09:18:09 optdb01 mysqld: ft: promotion: roots split: 7034 Feb 17 09:18:09 optdb01 mysqld: ft: promotion: leaf roots injected into: 516598055 Feb 17 09:18:09 optdb01 mysqld: ft: promotion: h1 roots injected into: 3377723904 Feb 17 09:18:09 optdb01 mysqld: ft: promotion: injections at depth 0: 31825438 Feb 17 09:18:09 optdb01 mysqld: ft: promotion: injections at depth 1: 1130538969 Feb 17 09:18:09 optdb01 mysqld: ft: promotion: injections at depth 2: 81473070 Feb 17 09:18:09 optdb01 mysqld: ft: promotion: injections at depth 3: 9530273 Feb 17 09:18:09 optdb01 mysqld: ft: promotion: injections lower than depth 3: 384964 Feb 17 09:18:09 optdb01 mysqld: ft: promotion: stopped because of a nonempty buffer: 280412038 Feb 17 09:18:09 optdb01 mysqld: ft: promotion: stopped at height 1: 92813 Feb 17 09:18:09 optdb01 mysqld: ft: promotion: stopped because the child was locked or not at all in memory: 316 Feb 17 09:18:09 optdb01 mysqld: ft: promotion: stopped because the child was not fully in memory: 91 Feb 17 09:18:09 optdb01 mysqld: ft: promotion: stopped anyway, after locking the child: 10193 Feb 17 09:18:09 optdb01 mysqld: ft: basement nodes deserialized with fixed-keysize: 4049246 Feb 17 09:18:09 optdb01 mysqld: ft: basement nodes deserialized with variable-keysize: 800670 Feb 17 09:18:09 optdb01 mysqld: ft: promotion: succeeded in using the rightmost leaf shortcut: 337181886 Feb 17 09:18:09 optdb01 mysqld: ft: promotion: tried the rightmost leaf shorcut but failed (out-of-bounds): 44966 Feb 17 09:18:09 optdb01 mysqld: ft: promotion: tried the rightmost leaf shorcut but failed (child reactive): 5773674 Feb 17 09:18:09 optdb01 mysqld: ft: cursor skipped deleted leaf entries: 232284525 Feb 17 09:18:09 optdb01 mysqld: ft flusher: total nodes potentially flushed by cleaner thread: 77367 Feb 17 09:18:09 optdb01 mysqld: ft flusher: height-one nodes flushed by cleaner thread: 66788 Feb 17 09:18:09 optdb01 mysqld: ft flusher: height-greater-than-one nodes flushed by cleaner thread: 10579 Feb 17 09:18:09 optdb01 mysqld: ft flusher: nodes cleaned which had empty buffers: 20119 Feb 17 09:18:09 optdb01 mysqld: ft flusher: nodes dirtied by cleaner thread: 107842 Feb 17 09:18:09 optdb01 mysqld: ft flusher: max bytes in a buffer flushed by cleaner thread: 3760664 Feb 17 09:18:09 optdb01 mysqld: ft flusher: min bytes in a buffer flushed by cleaner thread: 41 Feb 17 09:18:09 optdb01 mysqld: ft flusher: total bytes in buffers flushed by cleaner thread: 3527229087 Feb 17 09:18:09 optdb01 mysqld: ft flusher: max workdone in a buffer flushed by cleaner thread: 3240030 Feb 17 09:18:09 optdb01 mysqld: ft flusher: min workdone in a buffer flushed by cleaner thread: 0 Feb 17 09:18:09 optdb01 mysqld: ft flusher: total workdone in buffers flushed by cleaner thread: 2526847448 Feb 17 09:18:09 optdb01 mysqld: ft flusher: times cleaner thread tries to merge a leaf: 73 Feb 17 09:18:09 optdb01 mysqld: ft flusher: cleaner thread leaf merges in progress: 0 Feb 17 09:18:09 optdb01 mysqld: ft flusher: cleaner thread leaf merges successful: 72 Feb 17 09:18:09 optdb01 mysqld: ft flusher: nodes dirtied by cleaner thread leaf merges: 8 Feb 17 09:18:09 optdb01 mysqld: ft flusher: total number of flushes done by flusher threads or cleaner threads: 509824 Feb 17 09:18:09 optdb01 mysqld: ft flusher: number of in memory flushes: 509824 Feb 17 09:18:09 optdb01 mysqld: ft flusher: number of flushes that read something off disk: 0 Feb 17 09:18:09 optdb01 mysqld: ft flusher: number of flushes that triggered another flush in child: 113 Feb 17 09:18:09 optdb01 mysqld: ft flusher: number of flushes that triggered 1 cascading flush: 110 Feb 17 09:18:09 optdb01 mysqld: ft flusher: number of flushes that triggered 2 cascading flushes: 3 Feb 17 09:18:09 optdb01 mysqld: ft flusher: number of flushes that triggered 3 cascading flushes: 0 Feb 17 09:18:09 optdb01 mysqld: ft flusher: number of flushes that triggered 4 cascading flushes: 0 Feb 17 09:18:09 optdb01 mysqld: ft flusher: number of flushes that triggered 5 cascading flushes: 0 Feb 17 09:18:09 optdb01 mysqld: ft flusher: number of flushes that triggered over 5 cascading flushes: 0 Feb 17 09:18:09 optdb01 mysqld: ft flusher: leaf node splits: 54793 Feb 17 09:18:09 optdb01 mysqld: ft flusher: nonleaf node splits: 326 Feb 17 09:18:09 optdb01 mysqld: ft flusher: leaf node merges: 89 Feb 17 09:18:09 optdb01 mysqld: ft flusher: nonleaf node merges: 6 Feb 17 09:18:09 optdb01 mysqld: ft flusher: leaf node balances: 9 Feb 17 09:18:09 optdb01 mysqld: hot: operations ever started: 0 Feb 17 09:18:09 optdb01 mysqld: hot: operations successfully completed: 0 Feb 17 09:18:09 optdb01 mysqld: hot: operations aborted: 0 Feb 17 09:18:09 optdb01 mysqld: hot: max number of flushes from root ever required to optimize a tree: 0 Feb 17 09:18:09 optdb01 mysqld: txn: begin: 1186484399 Feb 17 09:18:09 optdb01 mysqld: txn: begin read only: 90432 Feb 17 09:18:09 optdb01 mysqld: txn: successful commits: 799370861 Feb 17 09:18:09 optdb01 mysqld: txn: aborts: 387203968 Feb 17 09:18:09 optdb01 mysqld: logger: next LSN: 120530614737 Feb 17 09:18:09 optdb01 mysqld: logger: writes: 9192599 Feb 17 09:18:09 optdb01 mysqld: logger: writes (bytes): 385695439705 Feb 17 09:18:09 optdb01 mysqld: logger: writes (uncompressed bytes): 385695439705 Feb 17 09:18:09 optdb01 mysqld: logger: writes (seconds): 880.279753 Feb 17 09:18:09 optdb01 mysqld: logger: number of long logger write operations: 0 Feb 17 09:18:09 optdb01 mysqld: indexer: number of indexers successfully created: 0 Feb 17 09:18:09 optdb01 mysqld: indexer: number of calls to toku_indexer_create_indexer() that failed: 0 Feb 17 09:18:09 optdb01 mysqld: indexer: number of calls to indexer->build() succeeded: 0 Feb 17 09:18:09 optdb01 mysqld: indexer: number of calls to indexer->build() failed: 0 Feb 17 09:18:09 optdb01 mysqld: indexer: number of calls to indexer->close() that succeeded: 0 Feb 17 09:18:09 optdb01 mysqld: indexer: number of calls to indexer->close() that failed: 0 Feb 17 09:18:09 optdb01 mysqld: indexer: number of calls to indexer->abort(): 0 Feb 17 09:18:09 optdb01 mysqld: indexer: number of indexers currently in existence: 0 Feb 17 09:18:09 optdb01 mysqld: indexer: max number of indexers that ever existed simultaneously: 0 Feb 17 09:18:09 optdb01 mysqld: loader: number of loaders successfully created: 11859 Feb 17 09:18:09 optdb01 mysqld: loader: number of calls to toku_loader_create_loader() that failed: 0 Feb 17 09:18:09 optdb01 mysqld: loader: number of calls to loader->put() succeeded: 19275353 Feb 17 09:18:09 optdb01 mysqld: loader: number of calls to loader->put() failed: 0 Feb 17 09:18:09 optdb01 mysqld: loader: number of calls to loader->close() that succeeded: 11859 Feb 17 09:18:09 optdb01 mysqld: loader: number of calls to loader->close() that failed: 0 Feb 17 09:18:09 optdb01 mysqld: loader: number of calls to loader->abort(): 0 Feb 17 09:18:09 optdb01 mysqld: loader: number of loaders currently in existence: 0 Feb 17 09:18:09 optdb01 mysqld: loader: max number of loaders that ever existed simultaneously: 2 Feb 17 09:18:09 optdb01 mysqld: memory: number of malloc operations: 0 Feb 17 09:18:09 optdb01 mysqld: memory: number of free operations: 0 Feb 17 09:18:09 optdb01 mysqld: memory: number of realloc operations: 0 Feb 17 09:18:09 optdb01 mysqld: memory: number of malloc operations that failed: 0 Feb 17 09:18:09 optdb01 mysqld: memory: number of realloc operations that failed: 14157 Feb 17 09:18:09 optdb01 mysqld: memory: number of bytes requested: 0 Feb 17 09:18:09 optdb01 mysqld: memory: number of bytes used (requested + overhead): 0 Feb 17 09:18:09 optdb01 mysqld: memory: number of bytes freed: 0 Feb 17 09:18:09 optdb01 mysqld: memory: largest attempted allocation size: 0 Feb 17 09:18:09 optdb01 mysqld: memory: size of the last failed allocation attempt: 0 Feb 17 09:18:09 optdb01 mysqld: memory: estimated maximum memory footprint: 0 Feb 17 09:18:09 optdb01 mysqld: memory: mallocator version: 3.6.0-0-g46c0af68bd248b04df75e4f92d5fb804c3d75340 Feb 17 09:18:09 optdb01 mysqld: memory: mmap threshold: 4194304 Feb 17 09:18:09 optdb01 mysqld: filesystem: ENOSPC redzone state: 0 Feb 17 09:18:09 optdb01 mysqld: filesystem: threads currently blocked by full disk: 0 Feb 17 09:18:09 optdb01 mysqld: filesystem: number of operations rejected by enospc prevention (red zone): 0 Feb 17 09:18:09 optdb01 mysqld: filesystem: most recent disk full: Thu Jan 1 00:00:00 1970 Feb 17 09:18:09 optdb01 mysqld: filesystem: number of write operations that returned ENOSPC: 0 Feb 17 09:18:09 optdb01 mysqld: filesystem: fsync time: 6994366715 Feb 17 09:18:09 optdb01 mysqld: filesystem: fsync count: 10314484 Feb 17 09:18:09 optdb01 mysqld: filesystem: long fsync time: 16646283 Feb 17 09:18:09 optdb01 mysqld: filesystem: long fsync count: 14 Feb 17 09:18:09 optdb01 mysqld: context: tree traversals blocked by a full fetch: 0 Feb 17 09:18:09 optdb01 mysqld: context: tree traversals blocked by a partial fetch: 89 Feb 17 09:18:09 optdb01 mysqld: context: tree traversals blocked by a full eviction: 0 Feb 17 09:18:09 optdb01 mysqld: context: tree traversals blocked by a partial eviction: 30 Feb 17 09:18:09 optdb01 mysqld: context: tree traversals blocked by a message injection: 386241 Feb 17 09:18:09 optdb01 mysqld: context: tree traversals blocked by a message application: 662 Feb 17 09:18:09 optdb01 mysqld: context: tree traversals blocked by a flush: 169430 Feb 17 09:18:09 optdb01 mysqld: context: tree traversals blocked by a the cleaner thread: 264 Feb 17 09:18:09 optdb01 mysqld: context: tree traversals blocked by something uninstrumented: 7193 Feb 17 09:18:09 optdb01 mysqld: context: promotion blocked by a full fetch (should never happen): 6 Feb 17 09:18:09 optdb01 mysqld: context: promotion blocked by a partial fetch (should never happen): 0 Feb 17 09:18:09 optdb01 mysqld: context: promotion blocked by a full eviction (should never happen): 0 Feb 17 09:18:09 optdb01 mysqld: context: promotion blocked by a partial eviction (should never happen): 1 Feb 17 09:18:09 optdb01 mysqld: context: promotion blocked by a message injection: 24702 Feb 17 09:18:09 optdb01 mysqld: context: promotion blocked by a message application: 0 Feb 17 09:18:09 optdb01 mysqld: context: promotion blocked by a flush: 2 Feb 17 09:18:09 optdb01 mysqld: context: promotion blocked by the cleaner thread: 35 Feb 17 09:18:09 optdb01 mysqld: context: promotion blocked by something uninstrumented: 178 Feb 17 09:18:09 optdb01 mysqld: context: something uninstrumented blocked by something uninstrumented: 25 Feb 17 09:18:09 optdb01 mysqld: Feb 17 09:18:09 optdb01 mysqld: Memory usage: Feb 17 09:18:09 optdb01 mysqld: Arena 0: Feb 17 09:18:09 optdb01 mysqld: system bytes = 0 Feb 17 09:18:09 optdb01 mysqld: in use bytes = 0 Feb 17 09:18:09 optdb01 mysqld: Total (incl. mmap): Feb 17 09:18:09 optdb01 mysqld: system bytes = 0 Feb 17 09:18:09 optdb01 mysqld: in use bytes = 0 Feb 17 09:18:09 optdb01 mysqld: max mmap regions = 0 Feb 17 09:18:09 optdb01 mysqld: max mmap bytes = 0 Feb 17 09:18:09 optdb01 mysqld: 160217 9:18:09 [ERROR] mysqld got signal 6 ; Feb 17 09:18:09 optdb01 mysqld: This could be because you hit a bug. It is also possible that this binary Feb 17 09:18:09 optdb01 mysqld: or one of the libraries it was linked against is corrupt, improperly built, Feb 17 09:18:09 optdb01 mysqld: or misconfigured. This error can also be caused by malfunctioning hardware. Feb 17 09:18:09 optdb01 mysqld: Feb 17 09:18:09 optdb01 mysqld: To report this bug, see http://kb.askmonty.org/en/reporting-bugs Feb 17 09:18:09 optdb01 mysqld: Feb 17 09:18:09 optdb01 mysqld: We will try our best to scrape up some info that will hopefully help Feb 17 09:18:09 optdb01 mysqld: diagnose the problem, but since we have already crashed, Feb 17 09:18:09 optdb01 mysqld: something is definitely wrong and this may fail. Feb 17 09:18:09 optdb01 mysqld: Feb 17 09:18:09 optdb01 mysqld: Server version: 10.0.23-MariaDB-log Feb 17 09:18:09 optdb01 mysqld: key_buffer_size=134217728 Feb 17 09:18:09 optdb01 mysqld: read_buffer_size=8388608 Feb 17 09:18:09 optdb01 mysqld: max_used_connections=33 Feb 17 09:18:09 optdb01 mysqld: max_threads=76 Feb 17 09:18:09 optdb01 mysqld: thread_count=28 Feb 17 09:18:09 optdb01 mysqld: It is possible that mysqld could use up to Feb 17 09:18:09 optdb01 mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 910793 K bytes of memory Feb 17 09:18:09 optdb01 mysqld: Hope that's ok; if not, decrease some variables in the equation. Feb 17 09:18:09 optdb01 mysqld: Feb 17 09:18:09 optdb01 mysqld: Thread pointer: 0x0x7fcc00704008 Feb 17 09:18:09 optdb01 mysqld: Attempting backtrace. You can use the following information to find out Feb 17 09:18:09 optdb01 mysqld: where mysqld died. If you see no messages after this, something went Feb 17 09:18:09 optdb01 mysqld: terribly wrong... Feb 17 09:18:09 optdb01 mysqld: stack_bottom = 0x7fd2d37fdcef thread_stack 0x48000 Feb 17 09:18:09 optdb01 mysqld: (my_addr_resolve failure: fork) Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x2e) [0xbd0aae] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(handle_fatal_signal+0x390) [0x73aa40] Feb 17 09:18:09 optdb01 mysqld: /lib64/libpthread.so.0(+0xf100) [0x7fd2ff44e100] Feb 17 09:18:09 optdb01 mysqld: /lib64/libc.so.6(gsignal+0x37) [0x7fd2fdc025f7] Feb 17 09:18:09 optdb01 mysqld: /lib64/libc.so.6(abort+0x148) [0x7fd2fdc03ce8] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x8e328) [0x7fd2e2c90328] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x8e39a) [0x7fd2e2c9039a] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x117d17) [0x7fd2e2d19d17] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x117c1e) [0x7fd2e2d19c1e] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x117f6e) [0x7fd2e2d19f6e] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x119342) [0x7fd2e2d1b342] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x112635) [0x7fd2e2d14635] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x11292f) [0x7fd2e2d1492f] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x1133f5) [0x7fd2e2d153f5] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(+0x1136eb) [0x7fd2e2d156eb] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(toku_db_start_range_lock(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, toku::lock_request::type, toku::lock_request*)+0xdc) [0x7fd2e2d1ea6c] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(toku_db_get_range_lock(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, toku::lock_request::type)+0xe6) [0x7fd2e2d1ebf6] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(env_update_multiple(__toku_db_env*, __toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, __toku_dbt*, __toku_dbt*, unsigned int, __toku_db**, unsigned int*, unsigned int, DBT_ARRAY*, unsigned int, DBT_ARRAY*)+0x838) [0x7fd2e2d241a8] Feb 17 09:18:09 optdb01 mysqld: /usr/lib64/mysql/plugin/ha_tokudb.so(ha_tokudb::update_row(unsigned char const*, unsigned char*)+0x572) [0x7fd2e2c6b8a2] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(handler::ha_update_row(unsigned char const*, unsigned char*)+0x412) [0x745172] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(multi_update::send_data(List&)+0x2f9) [0x66c419] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld() [0x5fa69d] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld() [0x5fc9d7] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sub_select(JOIN*, st_join_table*, bool)+0x176) [0x60b346] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld() [0x5fc9d7] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sub_select(JOIN*, st_join_table*, bool)+0x1bc) [0x60b38c] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld() [0x61686d] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(JOIN::exec_inner()+0xa3a) [0x627caa] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(JOIN::exec()+0x11) [0x629af1] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0xc7) [0x6264f7] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(mysql_multi_update(THD*, TABLE_LIST*, List*, List*, Item*, unsigned long long, enum_duplicates, bool, st_select_lex_unit*, st_select_lex*, multi_update**)+0x1af) [0x66ea9f] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(mysql_execute_command(THD*)+0x4e81) [0x5d6701] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sp_instr_stmt::exec_core(THD*, unsigned int*)+0x34) [0x8527d4] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned int*, bool, sp_instr*)+0x80) [0x859c40] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sp_instr_stmt::execute(THD*, unsigned int*)+0x155) [0x85a065] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sp_head::execute(THD*, bool)+0x6be) [0x856d8e] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sp_head::execute_procedure(THD*, List*)+0x664) [0x858374] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(mysql_execute_command(THD*)+0x1d41) [0x5d35c1] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sp_instr_stmt::exec_core(THD*, unsigned int*)+0x34) [0x8527d4] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned int*, bool, sp_instr*)+0x80) [0x859c40] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sp_instr_stmt::execute(THD*, unsigned int*)+0x155) [0x85a065] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sp_head::execute(THD*, bool)+0x6be) [0x856d8e] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sp_head::execute_procedure(THD*, List*)+0x664) [0x858374] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(mysql_execute_command(THD*)+0x1d41) [0x5d35c1] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sp_instr_stmt::exec_core(THD*, unsigned int*)+0x34) [0x8527d4] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned int*, bool, sp_instr*)+0x80) [0x859c40] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sp_instr_stmt::execute(THD*, unsigned int*)+0x155) [0x85a065] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sp_head::execute(THD*, bool)+0x6be) [0x856d8e] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(sp_head::execute_procedure(THD*, List*)+0x664) [0x858374] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(Event_job_data::execute(THD*, bool)+0x799) [0x68dc79] Feb 17 09:18:09 optdb01 mysqld: /usr/sbin/mysqld(event_worker_thread+0x109) [0x88de39] Feb 17 09:18:09 optdb01 mysqld: /lib64/libpthread.so.0(+0x7dc5) [0x7fd2ff446dc5] Feb 17 09:18:09 optdb01 mysqld: /lib64/libc.so.6(clone+0x6d) [0x7fd2fdcc321d] Feb 17 09:18:09 optdb01 mysqld: Feb 17 09:18:09 optdb01 mysqld: Trying to get some variables. Feb 17 09:18:09 optdb01 mysqld: Some pointers may be invalid and cause the dump to abort. Feb 17 09:18:09 optdb01 mysqld: Query (0x7fccb3bc3020): is an invalid pointer Feb 17 09:18:09 optdb01 mysqld: Connection ID (thread ID): 39380 Feb 17 09:18:09 optdb01 mysqld: Status: NOT_KILLED Feb 17 09:18:09 optdb01 mysqld: Feb 17 09:18:09 optdb01 mysqld: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on Feb 17 09:18:09 optdb01 mysqld: Feb 17 09:18:09 optdb01 mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains Feb 17 09:18:09 optdb01 mysqld: information that should help you find out what is causing the crash.