Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-17044

mysql_upgrade is causing crash of mysqld

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Minor
    • Resolution: Won't Fix
    • 10.2.17, 10.2(EOL), 10.3(EOL)
    • N/A
    • None
    • single server, centos7

    Description

      mysql_upgrade is causing crash of mysqld
      in case of MariaDB downgrade from 10.3 to 10.2.17

      The problem is that after downgrading it's received [ERROR] mysqld:
      An error occurred when initializing system tables
      and then it's recommended to perform mysql_upgrade:
      [Warning] InnoDB: Table mysql/innodb_table_stats has length mismatch in the column name table_name. Please run mysql_upgrade

      But when run mysql_upgrade , mysqld is crashing ; after mysqld reinitialization it's received
      again the message "Please run mysql_upgrade" ;
      then over and over again mysql will crash and start prompting for ' mysql_upgrade'

      [root@t4w5 ~]# mysql_upgrade -u root -p
      Phase 1/7: Checking and upgrading mysql database
      Processing databases
      mysql
      mysql.column_stats                                 OK
      mysql.columns_priv                                 OK
      mysql.db                                           OK
      mysql.event                                        OK
      mysql.func                                         OK
      mysql.gtid_slave_pos                               OK
      mysql.help_category                                OK
      mysql.help_keyword                                 OK
      mysql.help_relation                                OK
      mysql.help_topic                                   OK
      mysql.host                                         OK
      mysql.index_stats                                  OK
      mysql.innodb_index_stats                           OK
      mysql.innodb_table_stats                           OK
      mysql.plugin                                       OK
      mysql.proc                                         OK
      mysql.procs_priv                                   OK
      mysql.proxies_priv                                 OK
      mysql.roles_mapping                                OK
      mysql.servers                                      OK
      mysql.table_stats                                  OK
      mysql.tables_priv                                  OK
      mysql.time_zone                                    OK
      mysql.time_zone_leap_second                        OK
      mysql.time_zone_name                               OK
      mysql.time_zone_transition                         OK
      mysql.time_zone_transition_type                    OK
      mysql.transaction_registry                         OK
      mysql.user                                         OK
      Phase 2/7: Installing used storage engines... Skipped
      Phase 3/7: Fixing views
      Phase 4/7: Running 'mysql_fix_privilege_tables'
      ERROR 1408 (HY000) at line 553: Event Scheduler: An error occurred when initializing system tables. Disabling the Event Scheduler.
      ERROR 1805 (HY000) at line 582: Column count of mysql.proc is wrong. Expected 20, found 21. The table is probably corrupted
      ERROR 1805 (HY000) at line 584: Column count of mysql.proc is wrong. Expected 20, found 21. The table is probably corrupted
      ERROR 1805 (HY000) at line 592: Column count of mysql.proc is wrong. Expected 20, found 21. The table is probably corrupted
      ERROR 1805 (HY000) at line 594: Column count of mysql.proc is wrong. Expected 20, found 21. The table is probably corrupted
      ERROR 2013 (HY000) at line 606: Lost connection to MySQL server during query
      ERROR 2006 (HY000) at line 607: MySQL server has gone away
      ERROR 2006 (HY000) at line 608: MySQL server has gone away
      ERROR 2006 (HY000) at line 609: MySQL server has gone away
      ERROR 2006 (HY000) at line 610: MySQL server has gone away
      ERROR 2006 (HY000) at line 611: MySQL server has gone away
      ERROR 2006 (HY000) at line 612: MySQL server has gone away
      ERROR 2006 (HY000) at line 613: MySQL server has gone away
      ERROR 2006 (HY000) at line 614: MySQL server has gone away
      ERROR 2006 (HY000) at line 615: MySQL server has gone away
      ERROR 2006 (HY000) at line 616: MySQL server has gone away
      ERROR 2006 (HY000) at line 617: MySQL server has gone away
      ERROR 2006 (HY000) at line 618: MySQL server has gone away
      ERROR 2006 (HY000) at line 619: MySQL server has gone away
      ERROR 2006 (HY000) at line 620: MySQL server has gone away
      ERROR 2006 (HY000) at line 621: MySQL server has gone away
      ERROR 2006 (HY000) at line 622: MySQL server has gone away
      ERROR 2006 (HY000) at line 623: MySQL server has gone away
      ERROR 2006 (HY000) at line 624: MySQL server has gone away
      ERROR 2006 (HY000) at line 625: MySQL server has gone away
      ERROR 2006 (HY000) at line 626: MySQL server has gone away
      ERROR 2006 (HY000) at line 627: MySQL server has gone away
      ERROR 2006 (HY000) at line 628: MySQL server has gone away
      ERROR 2006 (HY000) at line 629: MySQL server has gone away
      ERROR 2006 (HY000) at line 630: MySQL server has gone away
      ERROR 2006 (HY000) at line 631: MySQL server has gone away
      ERROR 2006 (HY000) at line 632: MySQL server has gone away
      ERROR 2006 (HY000) at line 633: MySQL server has gone away
      ERROR 2006 (HY000) at line 634: MySQL server has gone away
      ERROR 2006 (HY000) at line 635: MySQL server has gone away
      ERROR 2006 (HY000) at line 636: MySQL server has gone away
      ERROR 2006 (HY000) at line 637: MySQL server has gone away
      ERROR 2006 (HY000) at line 638: MySQL server has gone away
      ERROR 2006 (HY000) at line 639: MySQL server has gone away
      ERROR 2006 (HY000) at line 640: MySQL server has gone away
      ERROR 2006 (HY000) at line 641: MySQL server has gone away
      ERROR 2006 (HY000) at line 642: MySQL server has gone away
      ERROR 2006 (HY000) at line 643: MySQL server has gone away
      ERROR 2006 (HY000) at line 644: MySQL server has gone away
      ERROR 2006 (HY000) at line 645: MySQL server has gone away
      ERROR 2006 (HY000) at line 646: MySQL server has gone away
      ERROR 2006 (HY000) at line 647: MySQL server has gone away
      ERROR 2006 (HY000) at line 648: MySQL server has gone away
      ERROR 2006 (HY000) at line 649: MySQL server has gone away
      ERROR 2006 (HY000) at line 650: MySQL server has gone away
      ERROR 2006 (HY000) at line 651: MySQL server has gone away
      ERROR 2006 (HY000) at line 652: MySQL server has gone away
      ERROR 2006 (HY000) at line 653: MySQL server has gone away
      ERROR 2006 (HY000) at line 654: MySQL server has gone away
      ERROR 2006 (HY000) at line 655: MySQL server has gone away
      ERROR 2006 (HY000) at line 656: MySQL server has gone away
      ERROR 2006 (HY000) at line 657: MySQL server has gone away
      ERROR 2006 (HY000) at line 658: MySQL server has gone away
      FATAL ERROR: Upgrade failed
      
      

      Aug 23 13:31:03 t4w5 mysqld: 2018-08-23 13:31:03 0x7f9d5c143700  InnoDB: Assertion failure in file /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.2.17/storage/innobase/trx/trx0undo.cc line 1516
      Aug 23 13:31:03 t4w5 mysqld: InnoDB: Failing assertion: mach_read_from_2(undo_page + TRX_UNDO_PAGE_HDR + TRX_UNDO_PAGE_TYPE) == TRX_UNDO_UPDATE
      Aug 23 13:31:03 t4w5 mysqld: InnoDB: We intentionally generate a memory trap.
      Aug 23 13:31:03 t4w5 mysqld: InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
      Aug 23 13:31:03 t4w5 mysqld: InnoDB: If you get repeated assertion failures or crashes, even
      Aug 23 13:31:03 t4w5 mysqld: InnoDB: immediately after the mysqld startup, there may be
      Aug 23 13:31:03 t4w5 mysqld: InnoDB: corruption in the InnoDB tablespace. Please refer to
      Aug 23 13:31:03 t4w5 mysqld: InnoDB: https://mariadb.com/kb/en/library/xtradbinnodb-recovery-modes/
      Aug 23 13:31:03 t4w5 mysqld: InnoDB: about forcing recovery.
      Aug 23 13:31:03 t4w5 mysqld: 180823 13:31:03 [ERROR] mysqld got signal 6 ;
      Aug 23 13:31:03 t4w5 mysqld: This could be because you hit a bug. It is also possible that this binary
      Aug 23 13:31:03 t4w5 mysqld: or one of the libraries it was linked against is corrupt, improperly built,
      Aug 23 13:31:03 t4w5 mysqld: or misconfigured. This error can also be caused by malfunctioning hardware.
      Aug 23 13:31:03 t4w5 mysqld: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
      Aug 23 13:31:03 t4w5 mysqld: We will try our best to scrape up some info that will hopefully help
      Aug 23 13:31:03 t4w5 mysqld: diagnose the problem, but since we have already crashed,
      Aug 23 13:31:03 t4w5 mysqld: something is definitely wrong and this may fail.
      Aug 23 13:31:03 t4w5 mysqld: Server version: 10.2.17-MariaDB
      Aug 23 13:31:03 t4w5 mysqld: key_buffer_size=134217728
      Aug 23 13:31:03 t4w5 mysqld: read_buffer_size=131072
      Aug 23 13:31:03 t4w5 mysqld: max_used_connections=2
      Aug 23 13:31:03 t4w5 mysqld: max_threads=153
      Aug 23 13:31:03 t4w5 mysqld: thread_count=8
      Aug 23 13:31:03 t4w5 mysqld: It is possible that mysqld could use up to
      Aug 23 13:31:03 t4w5 mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467246 K  bytes of memory
      Aug 23 13:31:03 t4w5 mysqld: Hope that's ok; if not, decrease some variables in the equation.
      Aug 23 13:31:03 t4w5 mysqld: Thread pointer: 0x7f9d18006df8
      Aug 23 13:31:03 t4w5 mysqld: Attempting backtrace. You can use the following information to find out
      Aug 23 13:31:03 t4w5 mysqld: where mysqld died. If you see no messages after this, something went
      Aug 23 13:31:03 t4w5 mysqld: terribly wrong...
      Aug 23 13:31:03 t4w5 mysqld: stack_bottom = 0x7f9d5c142d70 thread_stack 0x49000
      Aug 23 13:31:03 t4w5 mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55df5be1058e]
      Aug 23 13:31:03 t4w5 mysqld: mysys/stacktrace.c:268(my_print_stacktrace)[0x55df5b88e335]
      Aug 23 13:31:03 t4w5 mysqld: sigaction.c:0(__restore_rt)[0x7f9d79b4c5e0]
      Aug 23 13:31:03 t4w5 mysqld: :0(__GI_raise)[0x7f9d780591f7]
      Aug 23 13:31:03 t4w5 mysqld: :0(__GI_abort)[0x7f9d7805a8e8]
      Aug 23 13:31:04 t4w5 mysqld: /usr/sbin/mysqld(+0x429a88)[0x55df5b63da88]
      Aug 23 13:31:04 t4w5 mysqld: ut/ut0rbt.cc:221(rbt_tree_add_child(ib_rbt_t const*, ib_rbt_bound_t*, ib_rbt_node_t*) [clone .isra.4] [clone .part.5])[0x55df5bbc938c]
      Aug 23 13:31:04 t4w5 mysqld: trx/trx0undo.cc:1503(trx_undo_assign_undo(trx_t*, trx_rseg_t*, trx_undo_t**, unsigned long))[0x55df5bbabd3c]
      Aug 23 13:31:04 t4w5 mysqld: trx/trx0rec.cc:1963(trx_undo_report_row_operation(que_thr_t*, dict_index_t*, dtuple_t const*, upd_t const*, unsigned long, unsigned char const*, unsigned long const*, unsigned long*))[0x55df5bbf0ae3]
      Aug 23 13:31:04 t4w5 mysqld: btr/btr0cur.cc:4833(btr_cur_del_mark_set_clust_rec(buf_block_t*, unsigned char*, dict_index_t*, unsigned long const*, que_thr_t*, dtuple_t const*, mtr_t*))[0x55df5bb7092c]
      Aug 23 13:31:04 t4w5 mysqld: row/row0upd.cc:2727(row_upd_clust_step(upd_node_t*, que_thr_t*))[0x55df5bb7215a]
      Aug 23 13:31:04 t4w5 mysqld: row/row0upd.cc:3281(row_upd_step(que_thr_t*))[0x55df5bb0cee0]
      Aug 23 13:31:04 t4w5 mysqld: que/que0que.cc:1031(que_run_threads(que_thr_t*))[0x55df5bb0d983]
      Aug 23 13:31:04 t4w5 mysqld: que/que0que.cc:1230(que_eval_sql(pars_info_t*, char const*, unsigned long, trx_t*))[0x55df5bb40c23]
      Aug 23 13:31:04 t4w5 mysqld: row/row0mysql.cc:4375(row_rename_table_for_mysql(char const*, char const*, trx_t*, bool))[0x55df5ba7097d]
      Aug 23 13:31:04 t4w5 mysqld: handler/ha_innodb.cc:13600(ha_innobase::rename_table(char const*, char const*))[0x55df5b77b4cb]
      Aug 23 13:31:04 t4w5 mysqld: sql/sql_table.cc:5316(mysql_rename_table(handlerton*, char const*, char const*, char const*, char const*, unsigned int))[0x55df5b783731]
      Aug 23 13:31:04 t4w5 mysqld: sql/sql_table.cc:9627(mysql_alter_table(THD*, char*, char*, HA_CREATE_INFO*, TABLE_LIST*, Alter_info*, unsigned int, st_order*, bool))[0x55df5b7ccd5a]
      Aug 23 13:31:04 t4w5 mysqld: sql/sql_alter.cc:324(Sql_cmd_alter_table::execute(THD*))[0x55df5b6f4435]
      Aug 23 13:31:04 t4w5 mysqld: sql/sql_parse.cc:6225(mysql_execute_command(THD*))[0x55df5b70fade]
      Aug 23 13:31:04 t4w5 mysqld: sql/sql_prepare.cc:4774(Prepared_statement::execute(String*, bool))[0x55df5b70fd60]
      Aug 23 13:31:04 t4w5 mysqld: sql/sql_prepare.cc:4203(Prepared_statement::execute_loop(String*, bool, unsigned char*, unsigned char*))[0x55df5b710246]
      Aug 23 13:31:04 t4w5 mysqld: sql/sql_class.h:1331(Item_change_list_savepoint::rollback(Item_change_list*))[0x55df5b6f71cb]
      Aug 23 13:31:04 t4w5 mysqld: sql/sql_parse.cc:3500(mysql_execute_command(THD*))[0x55df5b6fc68e]
      Aug 23 13:31:04 t4w5 mysqld: sql/sql_parse.cc:8010(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x55df5b6ff8d5]
      Aug 23 13:31:04 t4w5 mysqld: sql/sql_parse.cc:1824(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x55df5b7004f5]
      Aug 23 13:31:04 t4w5 mysqld: sql/sql_parse.cc:1380(do_command(THD*))[0x55df5b7c9eda]
      Aug 23 13:31:04 t4w5 mysqld: sql/sql_connect.cc:1335(do_handle_one_connection(CONNECT*))[0x55df5b7c9ffd]
      Aug 23 13:31:04 t4w5 mysqld: pthread_create.c:0(start_thread)[0x7f9d79b44e25]
      Aug 23 13:31:04 t4w5 mysqld: /lib64/libc.so.6(clone+0x6d)[0x7f9d7811c34d]
      Aug 23 13:31:04 t4w5 mysqld: Trying to get some variables.
      Aug 23 13:31:04 t4w5 mysqld: Some pointers may be invalid and cause the dump to abort.
      Aug 23 13:31:04 t4w5 mysqld: Query (0x7f9d180f35c0): alter table mysql.innodb_index_stats modify last_update timestamp not null default current_timestamp on update current_timestamp, modify table_name varchar(199)
      Aug 23 13:31:04 t4w5 mysqld: Connection ID (thread ID): 16
      Aug 23 13:31:04 t4w5 mysqld: Status: NOT_KILLED
      Aug 23 13:31:04 t4w5 mysqld: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on
      Aug 23 13:31:04 t4w5 mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      Aug 23 13:31:04 t4w5 mysqld: information that should help you find out what is causing the crash.
      Aug 23 13:31:04 t4w5 abrt-hook-ccpp: Process 15310 (mysqld) of user 996 killed by SIGABRT - dumping core
      Aug 23 13:31:07 t4w5 abrt-hook-ccpp: Failed to create core_backtrace: waitpid failed: No child processes
      Aug 23 13:31:07 t4w5 systemd: mariadb.service: main process exited, code=killed, status=6/ABRT
      Aug 23 13:31:07 t4w5 systemd: Unit mariadb.service entered failed state.
      Aug 23 13:31:07 t4w5 systemd: mariadb.service failed.
      
      

      Attachments

        Issue Links

          Activity

            People

              marko Marko Mäkelä
              winstone Zdravelina Sokolovska (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.