Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-14914

MariaDB 10.2.12 crashes after midnight

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Duplicate
    • 10.2.12
    • N/A
    • Stored routines
    • None
    •  Freshly installed gentoo (4.14.7 kernel), 8core Xeon CPU, 16GB RAM

    Description

      Our database sometimes crashes after midnight. We can't figure out why

      • The CPU load / memory usage seems minimal
      • Happened on the 6th and the 8th of this month, not since

      This is a trace from the 6th of January

      2018-01-06 0:16:00 140365306332928 [Note] Event Scheduler: [root@%].[obfuscated.sphinx_indexujProdukty] event execution failed.
      180106 0:16:08 [ERROR] mysqld got signal 11 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.

      To report this bug, see https://mariadb.com/kb/en/reporting-bugs

      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed,
      something is definitely wrong and this may fail.

      Server version: 10.2.11-MariaDB-log
      key_buffer_size=33554432
      read_buffer_size=1048576
      max_used_connections=10
      max_threads=152
      thread_count=20
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 347227 K bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.

      Thread pointer: 0x7fa929839298
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x7fa95828ddcf thread_stack 0x80000
      /usr/sbin/mysqld(my_print_stacktrace+0x29)[0x55895fb4313d]
      /usr/sbin/mysqld(handle_fatal_signal+0x329)[0x55895f62cacb]
      /lib64/libpthread.so.0(+0x12fd0)[0x7faa09d38fd0]
      /usr/sbin/mysqld(_ZN13Explain_queryD1Ev+0x78)[0x55895f5a8f80]
      /usr/sbin/mysqld(_Z20delete_explain_queryP3LEX+0x15)[0x55895f5adb9f]
      /usr/sbin/mysqld(_ZN12sp_instr_set9exec_coreEP3THDPj+0x45)[0x55895f71d671]
      /usr/sbin/mysqld(_ZN13sp_lex_keeper23reset_lex_and_exec_coreEP3THDPjbP8sp_instr+0x368)[0x55895f7233d4]
      /usr/sbin/mysqld(_ZN7sp_head7executeEP3THDb+0x78b)[0x55895f720175]
      /usr/sbin/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x3be)[0x55895f72131a]
      /usr/sbin/mysqld(+0x4b744f)[0x55895f4cc44f]
      /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x1507)[0x55895f4d43f8]
      /usr/sbin/mysqld(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x14)[0x55895f71d864]
      /usr/sbin/mysqld(_ZN13sp_lex_keeper23reset_lex_and_exec_coreEP3THDPjbP8sp_instr+0x93)[0x55895f7230ff]
      /usr/sbin/mysqld(_ZN13sp_instr_stmt7executeEP3THDPj+0x1fd)[0x55895f7236b1]
      /usr/sbin/mysqld(_ZN7sp_head7executeEP3THDb+0x78b)[0x55895f720175]
      /usr/sbin/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x735)[0x55895f721691]
      /usr/sbin/mysqld(+0x4b744f)[0x55895f4cc44f]
      /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x1507)[0x55895f4d43f8]
      /usr/sbin/mysqld(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x14)[0x55895f71d864]
      /usr/sbin/mysqld(_ZN13sp_lex_keeper23reset_lex_and_exec_coreEP3THDPjbP8sp_instr+0x93)[0x55895f7230ff]
      /usr/sbin/mysqld(_ZN13sp_instr_stmt7executeEP3THDPj+0x1fd)[0x55895f7236b1]
      /usr/sbin/mysqld(_ZN7sp_head7executeEP3THDb+0x78b)[0x55895f720175]
      /usr/sbin/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x3be)[0x55895f72131a]
      /usr/sbin/mysqld(_ZN14Event_job_data7executeEP3THDb+0x867)[0x55895f57cd75]
      /usr/sbin/mysqld(_ZN19Event_worker_thread3runEP3THDP28Event_queue_element_for_exec+0xda)[0x55895f750f3e]
      /usr/sbin/mysqld(event_worker_thread+0x49)[0x55895f750fe9]
      /lib64/libpthread.so.0(+0x74fb)[0x7faa09d2d4fb]
      /lib64/libc.so.6(clone+0x3f)[0x7faa088b3e0f]

      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      Query (0x7fa8b8b78440): is an invalid pointer
      Connection ID (thread ID): 631762
      Status: NOT_KILLED

      Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on

      The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      information that should help you find out what is causing the crash.

      We think the query pointer is invalid, but we will try to print it anyway.
      Query: CALL zakaznik_aktualizujClenyNarozeninovychSkupin()

      This one is from the 8th

      2018-01-08 0:09:25 140627835029248 [Note] Event Scheduler: [root@%].[obfuscated.sphinx_indexujProdukty] event execution failed.
      180108 0:11:26 [ERROR] mysqld got signal 11 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.

      To report this bug, see https://mariadb.com/kb/en/reporting-bugs

      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed,
      something is definitely wrong and this may fail.

      Server version: 10.2.12-MariaDB-log
      key_buffer_size=33554432
      read_buffer_size=1048576
      max_used_connections=4
      max_threads=152
      thread_count=16
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 347227 K bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.

      Thread pointer: 0x7fe6342f1058
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x7fe6880e8dcf thread_stack 0x80000
      /usr/sbin/mysqld(my_print_stacktrace+0x29)[0x562da695b79a]
      /usr/sbin/mysqld(handle_fatal_signal+0x359)[0x562da644ad39]
      /lib64/libpthread.so.0(+0x12fd0)[0x7fe7191f3fd0]
      /usr/sbin/mysqld(_ZN13Explain_query10get_selectEj+0xb)[0x562da63c7245]
      /usr/sbin/mysqld(_ZN13Explain_query8add_nodeEP12Explain_node+0x95)[0x562da63c730f]
      /usr/sbin/mysqld(_ZN4JOIN24save_explain_data_internEP13Explain_querybbbPKc+0xa80)[0x562da63385a8]
      /usr/sbin/mysqld(_ZN4JOIN8optimizeEv+0xad)[0x562da633c9a5]
      /usr/sbin/mysqld(_ZN13st_select_lex31optimize_unflattened_subqueriesEb+0x141)[0x562da62e9cad]
      /usr/sbin/mysqld(_Z12mysql_insertP3THDP10TABLE_LISTR4ListI4ItemERS3_IS5_ES6_S6_15enum_duplicatesb+0x6e7)[0x562da62e0c1b]
      /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x2833)[0x562da62f42dc]
      /usr/sbin/mysqld(_ZN18Prepared_statement7executeEP6Stringb+0x4cf)[0x562da630923b]
      /usr/sbin/mysqld(ZN18Prepared_statement12execute_loopEP6StringbPhS2+0xac)[0x562da630934a]
      /usr/sbin/mysqld(_Z22mysql_sql_stmt_executeP3THD+0x124)[0x562da6309658]
      /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x16e4)[0x562da62f318d]
      /usr/sbin/mysqld(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x14)[0x562da653b71e]
      /usr/sbin/mysqld(_ZN13sp_lex_keeper23reset_lex_and_exec_coreEP3THDPjbP8sp_instr+0x93)[0x562da6540fef]
      /usr/sbin/mysqld(_ZN13sp_instr_stmt7executeEP3THDPj+0x1e8)[0x562da654158c]
      /usr/sbin/mysqld(_ZN7sp_head7executeEP3THDb+0x7bb)[0x562da653e05b]
      /usr/sbin/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x51f)[0x562da653f367]
      /usr/sbin/mysqld(+0x4ba09d)[0x562da62eb09d]
      /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x1506)[0x562da62f2faf]
      /usr/sbin/mysqld(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x14)[0x562da653b71e]
      /usr/sbin/mysqld(_ZN13sp_lex_keeper23reset_lex_and_exec_coreEP3THDPjbP8sp_instr+0x93)[0x562da6540fef]
      /usr/sbin/mysqld(_ZN13sp_instr_stmt7executeEP3THDPj+0x1e8)[0x562da654158c]
      /usr/sbin/mysqld(_ZN7sp_head7executeEP3THDb+0x7bb)[0x562da653e05b]
      /usr/sbin/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x51f)[0x562da653f367]
      /usr/sbin/mysqld(+0x4ba09d)[0x562da62eb09d]
      /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x1506)[0x562da62f2faf]
      /usr/sbin/mysqld(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x14)[0x562da653b71e]
      /usr/sbin/mysqld(_ZN13sp_lex_keeper23reset_lex_and_exec_coreEP3THDPjbP8sp_instr+0x93)[0x562da6540fef]
      /usr/sbin/mysqld(_ZN13sp_instr_stmt7executeEP3THDPj+0x1e8)[0x562da654158c]
      /usr/sbin/mysqld(_ZN7sp_head7executeEP3THDb+0x7bb)[0x562da653e05b]
      /usr/sbin/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x51f)[0x562da653f367]
      /usr/sbin/mysqld(+0x4ba09d)[0x562da62eb09d]
      /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x1506)[0x562da62f2faf]
      /usr/sbin/mysqld(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x14)[0x562da653b71e]
      /usr/sbin/mysqld(_ZN13sp_lex_keeper23reset_lex_and_exec_coreEP3THDPjbP8sp_instr+0x93)[0x562da6540fef]
      /usr/sbin/mysqld(_ZN13sp_instr_stmt7executeEP3THDPj+0x1e8)[0x562da654158c]
      /usr/sbin/mysqld(_ZN7sp_head7executeEP3THDb+0x7bb)[0x562da653e05b]
      /usr/sbin/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x73c)[0x562da653f584]
      /usr/sbin/mysqld(+0x4ba09d)[0x562da62eb09d]
      /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x1506)[0x562da62f2faf]
      /usr/sbin/mysqld(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x14)[0x562da653b71e]
      /usr/sbin/mysqld(_ZN13sp_lex_keeper23reset_lex_and_exec_coreEP3THDPjbP8sp_instr+0x93)[0x562da6540fef]
      /usr/sbin/mysqld(_ZN13sp_instr_stmt7executeEP3THDPj+0x1e8)[0x562da654158c]
      /usr/sbin/mysqld(_ZN7sp_head7executeEP3THDb+0x7bb)[0x562da653e05b]
      /usr/sbin/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x51f)[0x562da653f367]
      /usr/sbin/mysqld(_ZN14Event_job_data7executeEP3THDb+0x86f)[0x562da639af8d]
      /usr/sbin/mysqld(_ZN19Event_worker_thread3runEP3THDP28Event_queue_element_for_exec+0xdf)[0x562da656e3c3]
      /usr/sbin/mysqld(event_worker_thread+0x49)[0x562da656e46e]
      /lib64/libpthread.so.0(+0x74fb)[0x7fe7191e84fb]
      /lib64/libc.so.6(clone+0x3f)[0x7fe717d6ee0f]

      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      Query (0x7fe59c3e36e8): is an invalid pointer
      Connection ID (thread ID): 60936
      Status: NOT_KILLED

      Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on

      The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      information that should help you find out what is causing the crash.

      We think the query pointer is invalid, but we will try to print it anyway.
      Query: INSERT INTO produkt__hodnota_atributu
      SET
      id_produkt = '8883',
      id_atribut = '3',
      id_vazba_atributu_na_skupinu = (SELECT id FROM produkt_atribut_vazba_na_skupiny WHERE id_atribut = '3' AND id_skupina_atributu = 1),
      typ_hodnoty = 'varchar',
      hodnota_varchar = '188.000000',
      datum_vlozeni = NOW()

      This is causing us some unfortunate issues. I wonder if modifing the "optimizer switches" couldn't help us - at lease temporarily....

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              StarOstrich Petr Kubecka
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.