Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Duplicate
    • 10.1.21
    • N/A
    • Query Cache

    Description

      Hello,
      tonight one of our MariaDB Server crashed out of nowhere. Searched the Reason for a few Hours now - but was not able to narrow the cause down! :-/
      Maybe it's a Bug? Maybe you can tell me a bit more from those Errors?

      OS: CentOS Linux release 7.3.1611 (Core)

      Thank you

      170301  0:40:01 [ERROR] mysqld got signal 11 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed, 
      something is definitely wrong and this may fail.
       
      Server version: 10.1.21-MariaDB
      key_buffer_size=3221225472
      read_buffer_size=6291456
      max_used_connections=502
      max_threads=602
      thread_count=79
      It is possible that mysqld could use up to 
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 85762103 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x7f6e58b00a38
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x7f67b2a2ccf0 thread_stack 0x30000
      /usr/libexec/mysqld(my_print_stacktrace+0x2e)[0x7f6e565ce80e]
      /usr/libexec/mysqld(handle_fatal_signal+0x305)[0x7f6e560f5da5]
      /lib64/libpthread.so.0(+0xf370)[0x7f6e55712370]
      /usr/libexec/mysqld(_ZN11Query_cache12move_by_typeEPPhPP17Query_cache_blockPmS3_+0x730)[0x7f6e55f3b920]
      /usr/libexec/mysqld(_ZN11Query_cache10pack_cacheEv+0x71)[0x7f6e55f3bb21]
      /usr/libexec/mysqld(_ZN11Query_cache4packEP3THDmj+0x4e)[0x7f6e55f3be0e]
      /usr/libexec/mysqld(_Z20reload_acl_and_cacheP3THDyP10TABLE_LISTPi+0x690)[0x7f6e5605e970]
      /usr/libexec/mysqld(_Z21mysql_execute_commandP3THD+0x1400)[0x7f6e55f716a0]
      /usr/libexec/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x332)[0x7f6e55f79682]
      /usr/libexec/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x243c)[0x7f6e55f7c99c]
      /usr/libexec/mysqld(_Z10do_commandP3THD+0x14a)[0x7f6e55f7d1fa]
      /usr/libexec/mysqld(_Z24do_handle_one_connectionP3THD+0x18a)[0x7f6e5604445a]
      /usr/libexec/mysqld(handle_one_connection+0x40)[0x7f6e56044600]
      /lib64/libpthread.so.0(+0x7dc5)[0x7f6e5570adc5]
      /lib64/libc.so.6(clone+0x6d)[0x7f6e53f8373d]
       
      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      Query (0x7f68c40008e0): FLUSH QUERY CACHE
      Connection ID (thread ID): 568895
      Status: NOT_KILLED
       
      Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off
      

      Attachments

        Issue Links

          Activity

            added config file

            Futureweb Andreas Schnederle-Wagner added a comment - added config file
            danblack Daniel Black added a comment - - edited

            Definitely a bug. A server should never crash due to a query. Even one like FLUSH QUERY CACHE. No doing this SQL query will probably not affect any server behaviour at all.
            The output from this may be useful too.

             SHOW GLOBAL STATUS LIKE 'Qcache%';
            

            edit: server config provided

            danblack Daniel Black added a comment - - edited Definitely a bug. A server should never crash due to a query. Even one like FLUSH QUERY CACHE. No doing this SQL query will probably not affect any server behaviour at all. The output from this may be useful too. SHOW GLOBAL STATUS LIKE 'Qcache%' ; edit: server config provided
            Futureweb Andreas Schnederle-Wagner added a comment - - edited

            Uptime: 7 Hours

            SHOW GLOBAL STATUS LIKE 'Qcache%' 
             
            Qcache_free_blocks 	2174
            Qcache_free_memory 	104877056
            Qcache_hits 	3047970
            Qcache_inserts 	627341
            Qcache_lowmem_prunes 	63490
            Qcache_not_cached 	366437
            Qcache_queries_in_cache 	15284
            Qcache_total_blocks 	32979
            

            Uptime: 2 Days, 8 Hours, 30 Minutes

            SHOW GLOBAL STATUS LIKE 'Qcache%' 
             
            Qcache_free_blocks 	212
            Qcache_free_memory 	67424424
            Qcache_hits 	28510680
            Qcache_inserts 	4346831
            Qcache_lowmem_prunes 	682635
            Qcache_not_cached 	2673469
            Qcache_queries_in_cache 	36561
            Qcache_total_blocks 	73617
            

            Futureweb Andreas Schnederle-Wagner added a comment - - edited Uptime: 7 Hours SHOW GLOBAL STATUS LIKE 'Qcache%'   Qcache_free_blocks 2174 Qcache_free_memory 104877056 Qcache_hits 3047970 Qcache_inserts 627341 Qcache_lowmem_prunes 63490 Qcache_not_cached 366437 Qcache_queries_in_cache 15284 Qcache_total_blocks 32979 Uptime: 2 Days, 8 Hours, 30 Minutes SHOW GLOBAL STATUS LIKE 'Qcache%'   Qcache_free_blocks 212 Qcache_free_memory 67424424 Qcache_hits 28510680 Qcache_inserts 4346831 Qcache_lowmem_prunes 682635 Qcache_not_cached 2673469 Qcache_queries_in_cache 36561 Qcache_total_blocks 73617

            are there any further information I can supply?

            Futureweb Andreas Schnederle-Wagner added a comment - are there any further information I can supply?

            Hello,
            during the weekend the crash happened 2 more times:

            12.03.2017 08:00:11:

            170312  8:00:02 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware.
             
            To report this bug, see https://mariadb.com/kb/en/reporting-bugs
             
            We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail.
             
            Server version: 10.1.21-MariaDB
            key_buffer_size=3221225472
            read_buffer_size=6291456
            max_used_connections=503
            max_threads=602
            thread_count=73
            It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 85762103 K  bytes of memory Hope that's ok; if not, decrease some variables in the equation.
             
            Thread pointer: 0x7f56d1ef2c48
            Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong...
            stack_bottom = 0x7f54ffc8ccf0 thread_stack 0x30000 /usr/libexec/mysqld(my_print_stacktrace+0x2e)[0x7f56cfbaf80e]
            /usr/libexec/mysqld(handle_fatal_signal+0x305)[0x7f56cf6d6da5]
            /lib64/libpthread.so.0(+0xf370)[0x7f56cecf3370]
            /usr/libexec/mysqld(_ZN11Query_cache12move_by_typeEPPhPP17Query_cache_blockPmS3_+0x730)[0x7f56cf51c920]
            /usr/libexec/mysqld(_ZN11Query_cache10pack_cacheEv+0x71)[0x7f56cf51cb21]
            /usr/libexec/mysqld(_ZN11Query_cache4packEP3THDmj+0x4e)[0x7f56cf51ce0e]
            /usr/libexec/mysqld(_Z20reload_acl_and_cacheP3THDyP10TABLE_LISTPi+0x690)[0x7f56cf63f970]
            /usr/libexec/mysqld(_Z21mysql_execute_commandP3THD+0x1400)[0x7f56cf5526a0]
            /usr/libexec/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x332)[0x7f56cf55a682]
            /usr/libexec/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x243c)[0x7f56cf55d99c]
            /usr/libexec/mysqld(_Z10do_commandP3THD+0x14a)[0x7f56cf55e1fa]
            /usr/libexec/mysqld(_Z24do_handle_one_connectionP3THD+0x18a)[0x7f56cf62545a]
            /usr/libexec/mysqld(handle_one_connection+0x40)[0x7f56cf625600]
            /lib64/libpthread.so.0(+0x7dc5)[0x7f56cecebdc5]
            /lib64/libc.so.6(clone+0x6d)[0x7f56cd56473d]
             
            Trying to get some variables.
            Some pointers may be invalid and cause the dump to abort.
            Query (0x7f528c0008e0): FLUSH QUERY CACHE Connection ID (thread ID): 292308
            Status: NOT_KILLED
             
            Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off
            
            

            Futureweb Andreas Schnederle-Wagner added a comment - Hello, during the weekend the crash happened 2 more times: 12.03.2017 08:00:11: 170312 8:00:02 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware.   To report this bug, see https://mariadb.com/kb/en/reporting-bugs   We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail.   Server version: 10.1.21-MariaDB key_buffer_size=3221225472 read_buffer_size=6291456 max_used_connections=503 max_threads=602 thread_count=73 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 85762103 K bytes of memory Hope that's ok; if not, decrease some variables in the equation.   Thread pointer: 0x7f56d1ef2c48 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7f54ffc8ccf0 thread_stack 0x30000 /usr/libexec/mysqld(my_print_stacktrace+0x2e)[0x7f56cfbaf80e] /usr/libexec/mysqld(handle_fatal_signal+0x305)[0x7f56cf6d6da5] /lib64/libpthread.so.0(+0xf370)[0x7f56cecf3370] /usr/libexec/mysqld(_ZN11Query_cache12move_by_typeEPPhPP17Query_cache_blockPmS3_+0x730)[0x7f56cf51c920] /usr/libexec/mysqld(_ZN11Query_cache10pack_cacheEv+0x71)[0x7f56cf51cb21] /usr/libexec/mysqld(_ZN11Query_cache4packEP3THDmj+0x4e)[0x7f56cf51ce0e] /usr/libexec/mysqld(_Z20reload_acl_and_cacheP3THDyP10TABLE_LISTPi+0x690)[0x7f56cf63f970] /usr/libexec/mysqld(_Z21mysql_execute_commandP3THD+0x1400)[0x7f56cf5526a0] /usr/libexec/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x332)[0x7f56cf55a682] /usr/libexec/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x243c)[0x7f56cf55d99c] /usr/libexec/mysqld(_Z10do_commandP3THD+0x14a)[0x7f56cf55e1fa] /usr/libexec/mysqld(_Z24do_handle_one_connectionP3THD+0x18a)[0x7f56cf62545a] /usr/libexec/mysqld(handle_one_connection+0x40)[0x7f56cf625600] /lib64/libpthread.so.0(+0x7dc5)[0x7f56cecebdc5] /lib64/libc.so.6(clone+0x6d)[0x7f56cd56473d]   Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x7f528c0008e0): FLUSH QUERY CACHE Connection ID (thread ID): 292308 Status: NOT_KILLED   Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off

            13.03.2017 03:10:01:

            170313  3:10:01 [ERROR] mysqld got signal 11 ;
            This could be because you hit a bug. It is also possible that this binary
            or one of the libraries it was linked against is corrupt, improperly built,
            or misconfigured. This error can also be caused by malfunctioning hardware.
             
            To report this bug, see https://mariadb.com/kb/en/reporting-bugs
             
            We will try our best to scrape up some info that will hopefully help
            diagnose the problem, but since we have already crashed, 
            something is definitely wrong and this may fail.
             
            Server version: 10.1.21-MariaDB
            key_buffer_size=3221225472
            read_buffer_size=6291456
            max_used_connections=417
            max_threads=602
            thread_count=48
            It is possible that mysqld could use up to 
            key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 85762103 K  bytes of memory
            Hope that's ok; if not, decrease some variables in the equation.
             
            Thread pointer: 0x7f135263aea8
            Attempting backtrace. You can use the following information to find out
            where mysqld died. If you see no messages after this, something went
            terribly wrong...
            stack_bottom = 0x7f118c344cf0 thread_stack 0x30000
            /usr/libexec/mysqld(my_print_stacktrace+0x2e)[0x7f134e99880e]
            /usr/libexec/mysqld(handle_fatal_signal+0x305)[0x7f134e4bfda5]
            /lib64/libpthread.so.0(+0xf370)[0x7f134dadc370]
            /usr/libexec/mysqld(_ZN11Query_cache12move_by_typeEPPhPP17Query_cache_blockPmS3_+0x730)[0x7f134e305920]
            /usr/libexec/mysqld(_ZN11Query_cache10pack_cacheEv+0x71)[0x7f134e305b21]
            /usr/libexec/mysqld(_ZN11Query_cache4packEP3THDmj+0x4e)[0x7f134e305e0e]
            /usr/libexec/mysqld(_Z20reload_acl_and_cacheP3THDyP10TABLE_LISTPi+0x690)[0x7f134e428970]
            /usr/libexec/mysqld(_Z21mysql_execute_commandP3THD+0x1400)[0x7f134e33b6a0]
            /usr/libexec/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x332)[0x7f134e343682]
            /usr/libexec/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x243c)[0x7f134e34699c]
            /usr/libexec/mysqld(_Z10do_commandP3THD+0x14a)[0x7f134e3471fa]
            /usr/libexec/mysqld(_Z24do_handle_one_connectionP3THD+0x18a)[0x7f134e40e45a]
            /usr/libexec/mysqld(handle_one_connection+0x40)[0x7f134e40e600]
            /lib64/libpthread.so.0(+0x7dc5)[0x7f134dad4dc5]
            /lib64/libc.so.6(clone+0x6d)[0x7f134c34d73d]
             
            Trying to get some variables.
            Some pointers may be invalid and cause the dump to abort.
            Query (0x7f0f2c00bd20): FLUSH QUERY CACHE
            Connection ID (thread ID): 20616
            Status: NOT_KILLED
             
            Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off
            

            Futureweb Andreas Schnederle-Wagner added a comment - 13.03.2017 03:10:01: 170313 3:10:01 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware.   To report this bug, see https://mariadb.com/kb/en/reporting-bugs   We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail.   Server version: 10.1.21-MariaDB key_buffer_size=3221225472 read_buffer_size=6291456 max_used_connections=417 max_threads=602 thread_count=48 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 85762103 K bytes of memory Hope that's ok; if not, decrease some variables in the equation.   Thread pointer: 0x7f135263aea8 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7f118c344cf0 thread_stack 0x30000 /usr/libexec/mysqld(my_print_stacktrace+0x2e)[0x7f134e99880e] /usr/libexec/mysqld(handle_fatal_signal+0x305)[0x7f134e4bfda5] /lib64/libpthread.so.0(+0xf370)[0x7f134dadc370] /usr/libexec/mysqld(_ZN11Query_cache12move_by_typeEPPhPP17Query_cache_blockPmS3_+0x730)[0x7f134e305920] /usr/libexec/mysqld(_ZN11Query_cache10pack_cacheEv+0x71)[0x7f134e305b21] /usr/libexec/mysqld(_ZN11Query_cache4packEP3THDmj+0x4e)[0x7f134e305e0e] /usr/libexec/mysqld(_Z20reload_acl_and_cacheP3THDyP10TABLE_LISTPi+0x690)[0x7f134e428970] /usr/libexec/mysqld(_Z21mysql_execute_commandP3THD+0x1400)[0x7f134e33b6a0] /usr/libexec/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x332)[0x7f134e343682] /usr/libexec/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x243c)[0x7f134e34699c] /usr/libexec/mysqld(_Z10do_commandP3THD+0x14a)[0x7f134e3471fa] /usr/libexec/mysqld(_Z24do_handle_one_connectionP3THD+0x18a)[0x7f134e40e45a] /usr/libexec/mysqld(handle_one_connection+0x40)[0x7f134e40e600] /lib64/libpthread.so.0(+0x7dc5)[0x7f134dad4dc5] /lib64/libc.so.6(clone+0x6d)[0x7f134c34d73d]   Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x7f0f2c00bd20): FLUSH QUERY CACHE Connection ID (thread ID): 20616 Status: NOT_KILLED   Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off
            danblack Daniel Black added a comment -

            The obvious workaround is to not FLUSH QUERY CACHE. It can't be giving you any benefit compared to the crashing.

            danblack Daniel Black added a comment - The obvious workaround is to not FLUSH QUERY CACHE. It can't be giving you any benefit compared to the crashing.

            MDEV-25365 has a test case for what appears to be an identical failure (although not a great test case, but better than nothing). So, while it's generally against common sense to close an older report as a duplicate of a new one, in this case we'll continue tracking this problem in MDEV-25365.

            elenst Elena Stepanova added a comment - MDEV-25365 has a test case for what appears to be an identical failure (although not a great test case, but better than nothing). So, while it's generally against common sense to close an older report as a duplicate of a new one, in this case we'll continue tracking this problem in MDEV-25365 .

            People

              Unassigned Unassigned
              Futureweb Andreas Schnederle-Wagner
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.