Details

      Description

      First time I have seen this on my 3 node Galera cluster. It appears when the daily cron ran to flush logs via logrotate something caused maraidb to crash.

      I don't have much more information except the command which was run by logrotate (below) and the trace from maridadb.

      mysqladmin --defaults-file=/etc/mysql/debian.cnf flush-logs

      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: 161210  6:25:04 [ERROR] mysqld got signal 11 ;
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: This could be because you hit a bug. It is also possible that this binary
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: or one of the libraries it was linked against is corrupt, improperly built,
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: or misconfigured. This error can also be caused by malfunctioning hardware.
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: We will try our best to scrape up some info that will hopefully help
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: diagnose the problem, but since we have already crashed,
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: something is definitely wrong and this may fail.
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: Server version: 10.1.18-MariaDB-1~xenial
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: key_buffer_size=134217728
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: read_buffer_size=2097152
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: max_used_connections=65
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: max_threads=502
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: thread_count=8
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: It is possible that mysqld could use up to
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 3225650 K  bytes of memory
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: Hope that's ok; if not, decrease some variables in the equation.
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: Thread pointer: 0x0x7f571d4fe008
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: Attempting backtrace. You can use the following information to find out
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: where mysqld died. If you see no messages after this, something went
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: terribly wrong...
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: stack_bottom = 0x7f578bd8c088 thread_stack 0x48400
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: (my_addr_resolve failure: fork)
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /usr/sbin/mysqld(my_print_stacktrace+0x2e) [0x561e94e7562e]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /usr/sbin/mysqld(handle_fatal_signal+0x2d5) [0x561e949c43d5]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x113d0) [0x7f583594f3d0]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /usr/sbin/mysqld(MYSQL_BIN_LOG::do_checkpoint_request(unsigned long)+0x4a) [0x561e94a7957a]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /usr/sbin/mysqld(MYSQL_BIN_LOG::checkpoint_and_purge(unsigned long)+0x11) [0x561e94a795f1]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /usr/sbin/mysqld(MYSQL_BIN_LOG::rotate_and_purge(bool)+0x7e) [0x561e94a7baae]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /usr/sbin/mysqld(reload_acl_and_cache(THD*, unsigned long long, TABLE_LIST*, int*)+0x135) [0x561e94925045]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /usr/sbin/mysqld(mysql_execute_command(THD*)+0x1365) [0x561e948360f5]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /usr/sbin/mysqld(mysql_parse(THD*, char*, unsigned int, Parser_state*)+0x311) [0x561e9483e031]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /usr/sbin/mysqld(+0x40f899) [0x561e9483e899]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /usr/sbin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x1c3b) [0x561e94840b8b]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /usr/sbin/mysqld(do_command(THD*)+0x16d) [0x561e948419ad]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /usr/sbin/mysqld(do_handle_one_connection(THD*)+0x182) [0x561e9490af52]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /usr/sbin/mysqld(handle_one_connection+0x40) [0x561e9490b110]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /usr/sbin/mysqld(+0x705fb1) [0x561e94b34fb1]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76fa) [0x7f58359456fa]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5834ff0b5d]
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: Trying to get some variables.
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: Some pointers may be invalid and cause the dump to abort.
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: Query (0x7f57060e1020): flush logs
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: Connection ID (thread ID): 20185572
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: Status: NOT_KILLED
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      Dec 10 06:25:04 ams-prod1-db03 mysqld[2001]: information that should help you find out what is causing the crash.
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sachin.setiya.007 Sachin Setiya
                Reporter:
                madpsy Nathan Madpsy
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: