Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-31371

Table is lost after the server was killed during the cleanup of failed ALTER TABLE

    XMLWordPrintable

Details

    Description

      mleich produced rr replay traces of a killed server and a crash recovery that leads to the loss of an InnoDB table but not the .frm file.

      On the InnoDB side, everything looks correct. At the time the server was killed, InnoDB had durably written all changes to the ib_logfile0, and the recv_sys.recovered_lsn was identical to that LSN. The last log durable write took place during some ALTER TABLE cleanup:

      #0  0x000055ef174bdd2d in log_write_flush_to_disk_low (lsn=lsn@entry=26335973) at /data/Server/bb-10.6-MDEV-30986A/storage/innobase/log/log0log.cc:654
      #1  0x000055ef174be331 in log_write_up_to (lsn=<optimized out>, lsn@entry=26335973, flush_to_disk=flush_to_disk@entry=true, rotate_key=rotate_key@entry=false, callback=<optimized out>, 
          callback@entry=0x257767553590) at /data/Server/bb-10.6-MDEV-30986A/storage/innobase/log/log0log.cc:854
      #2  0x000055ef17823f51 in trx_flush_log_if_needed_low (lsn=26335973, trx=trx@entry=0x56250ad17640) at /data/Server/bb-10.6-MDEV-30986A/storage/innobase/trx/trx0trx.cc:1158
      #3  0x000055ef17823ff6 in trx_flush_log_if_needed (lsn=<optimized out>, trx=trx@entry=0x56250ad17640) at /data/Server/bb-10.6-MDEV-30986A/storage/innobase/trx/trx0trx.cc:1176
      #4  0x000055ef17835bbf in trx_t::commit_in_memory (mtr=0x257767553bd0, this=0x56250ad17640) at /data/Server/bb-10.6-MDEV-30986A/storage/innobase/trx/trx0trx.cc:1348
      #5  trx_t::commit_low (this=this@entry=0x56250ad17640, mtr=0x257767553bd0) at /data/Server/bb-10.6-MDEV-30986A/storage/innobase/trx/trx0trx.cc:1452
      #6  0x000055ef17835e30 in trx_t::commit_persist (this=this@entry=0x56250ad17640) at /data/Server/bb-10.6-MDEV-30986A/storage/innobase/trx/trx0trx.cc:1466
      #7  0x000055ef17a7355a in trx_t::commit (this=this@entry=0x56250ad17640, deleted=std::vector of length 0, capacity 0) at /data/Server/bb-10.6-MDEV-30986A/storage/innobase/dict/drop.cc:238
      #8  0x000055ef17344eac in ha_innobase::delete_table (this=<optimized out>, name=<optimized out>) at /data/Server/bb-10.6-MDEV-30986A/storage/innobase/handler/ha_innodb.cc:13764
      #9  0x000055ef1688a02a in hton_drop_table (hton=<optimized out>, path=<optimized out>) at /data/Server/bb-10.6-MDEV-30986A/sql/handler.cc:576
      #10 0x000055ef168a10e4 in ha_delete_table (thd=thd@entry=0x62b0001ab218, hton=hton@entry=0x615000001498, path=path@entry=0x257767555ad0 "./test/#sql-alter-1ae231-8b", db=db@entry=0x257767557a60, 
          alias=alias@entry=0x257767557a90, generate_warning=generate_warning@entry=false) at /data/Server/bb-10.6-MDEV-30986A/sql/handler.cc:3153
      #11 0x000055ef1623a4f9 in quick_rm_table (thd=thd@entry=0x62b0001ab218, base=base@entry=0x615000001498, db=db@entry=0x257767557a60, table_name=table_name@entry=0x257767557a90, flags=3, 
          table_path=table_path@entry=0x25776755844f "./test/#sql-alter-1ae231-8b") at /data/Server/bb-10.6-MDEV-30986A/sql/sql_table.cc:1916
      #12 0x000055ef1627d289 in mysql_alter_table (thd=thd@entry=0x62b0001ab218, new_db=new_db@entry=0x62b0001afc48, new_name=new_name@entry=0x62b0001b00a0, create_info=<optimized out>, 
          table_list=table_list@entry=0x62b0001b2438, recreate_info=<optimized out>, alter_info=<optimized out>, order_num=<optimized out>, order=<optimized out>, ignore=<optimized out>, if_exists=<optimized out>)
          at /data/Server/bb-10.6-MDEV-30986A/sql/sql_table.cc:10894
      #13 0x000055ef16410d41 in Sql_cmd_alter_table::execute (this=<optimized out>, thd=0x62b0001ab218) at /data/Server/bb-10.6-MDEV-30986A/sql/sql_alter.cc:553
      #14 0x000055ef15fbdecd in mysql_execute_command (thd=thd@entry=0x62b0001ab218, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /data/Server/bb-10.6-MDEV-30986A/sql/sql_parse.cc:6014
      #15 0x000055ef15fc1434 in mysql_parse (thd=thd@entry=0x62b0001ab218, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x25776755a620)
          at /data/Server/bb-10.6-MDEV-30986A/sql/sql_parse.cc:8036
      #16 0x000055ef15fc7c6f in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x62b0001ab218, 
          packet=packet@entry=0x629000eec219 " ALTER TABLE t3 MODIFY col_text TEXT CHARACTER SET utf8 COLLATE utf8_nopad_bin, LOCK = DEFAULT, ALGORITHM = COPY  /* E_R Thread14 QNO 462 CON_ID 139 */ ", 
          packet_length=packet_length@entry=152, blocking=blocking@entry=true) at /data/Server/bb-10.6-MDEV-30986A/sql/sql_parse.cc:1896
      

      The operation ALTER TABLE t3...ALGORITHM=COPY had failed because of a failure to upgrade a metadata lock, to replace the original table with the rebuilt table:

        if (wait_while_table_is_used(thd, table, HA_EXTRA_PREPARE_FOR_RENAME))
          goto err_new_table_cleanup;
      

      InnoDB is initially recovering test/t3.ibd (tablespace 1188) fine. After the recovery, test/t3.frm will continue to exist but not test/t3.ibd, because the DDL recovery is asking for the table to be dropped in the storage engine:

      10.6 a6c0a2769663850c055e41a83b0cca9dd9ab89a3

      #0  ha_innobase::delete_table (this=0x5597ccfce5d0, name=0x7fff9effd020 "./test/t3") at /mariadb/10.6/storage/innobase/handler/ha_innodb.cc:13470
      #1  0x00005597ca4bdf20 in hton_drop_table (hton=<optimized out>, path=0x20e06929d93d3200 <error: Cannot access memory at address 0x20e06929d93d3200>) at /mariadb/10.6/sql/handler.cc:576
      #2  0x00005597ca87a714 in ddl_log_execute_action (thd=0x5597cd0942b8, mem_root=0x7fff9effcf20, ddl_log_entry=0x7fff9effcf60) at /mariadb/10.6/sql/ddl_log.cc:2103
      #3  ddl_log_execute_entry_no_lock (thd=0x5597cd0942b8, first_entry=<optimized out>) at /mariadb/10.6/sql/ddl_log.cc:2411
      #4  0x00005597ca87b78b in ddl_log_execute_recovery () at /mariadb/10.6/sql/ddl_log.cc:2772
      #5  0x00005597ca49a789 in init_server_components () at /mariadb/10.6/sql/mysqld.cc:5398
      #6  0x00005597ca496ff4 in mysqld_main (argc=<optimized out>, argv=<optimized out>) at /mariadb/10.6/sql/mysqld.cc:5725
      

      The recovery failure can be reproduced with the attached copy of the data directory data.tar.xz that was made between the SIGKILL of the server and attempted recovery, as follows:

      tar xJf data.tar.xz
      10.6/sql/mariadbd --innodb-page-size=8k --innodb-log-file-size=96M --datadir="$(pwd)"/data
      

      Attachments

        Issue Links

          Activity

            People

              monty Michael Widenius
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.