Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-21787

Alter table failure tries to access uninitialized column

Details

    Description

      10.4.13 commit 3c6a3a42158ad3e6199eb690c97b783149d12e41
      Logging: mysql-test/mysql-test-run.pl  --mem --mysqld=--innodb_page_size=4K MDEV-21787
      ....
      TEST                                      RESULT   TIME (ms) or COMMENT
      --------------------------------------------------------------------------
       
      worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
      CREATE TABLE t4 ( col1 INT PRIMARY KEY, col_text_g TEXT GENERATED ALWAYS AS (SUBSTR(col_text,1,499)), col_varchar VARCHAR(500), col_text TEXT ) ENGINE = InnoDB ROW_FORMAT = Compact ;
      ALTER TABLE t4 ADD UNIQUE KEY ( col_text_g(9) ) ;
      ALTER TABLE t4 ADD COLUMN col_text_copy TEXT ;
      ERROR HY000: Lost connection to MySQL server during query
      CREATE TABLE t_end (col1 INT) ENGINE = InnoDB;
      ERROR HY000: MySQL server has gone away
      DROP TABLE t_end;
      ERROR HY000: MySQL server has gone away
      main.MDEV-21787 'innodb'                 [ fail ]
              Test ended at 2020-02-20 18:16:19
       
      CURRENT_TEST: main.MDEV-21787
      ...
      Version: '10.4.13-MariaDB-debug-log'  socket: 'bld_debug/mysql-test/var/tmp/mysqld.1.sock'  port: 16000  Source distribution
      2020-02-20 18:16:18 9 [ERROR] InnoDB: Cannot add field `col_text_copy` in table `test`.`#sql-1b8c_9` because after adding it, the row size is 2105 which is greater than maximum allowed size (1982 bytes) for a record on index leaf page.
      200220 18:16:18 [ERROR] mysqld got signal 11 ;
      ...
      Query (0x7ffa98012660): ALTER TABLE t4 ADD COLUMN col_text_copy TEXT
      Connection ID (thread ID): 9
      Status: NOT_KILLED
      ...
      Thread 1 (Thread 0x7ffae316a700 (LWP 7081)):
      #0  __pthread_kill (threadid=<optimized out>, signo=11) at ../sysdeps/unix/sysv/linux/pthread_kill.c:57
      #1  0x0000563f9671b9b9 in my_write_core (sig=11) at mysys/stacktrace.c:481
      #2  0x0000563f95dc6297 in handle_fatal_signal (sig=11) at sql/signal_handler.cc:343
      #3  <signal handler called>
      #4  0x0000563f96037d78 in dict_col_t::is_virtual (this=0x8f8f8f8f8f8f8f8f) at storage/innobase/include/dict0mem.h:598
      #5  0x0000563f9606eb58 in dict_col_t::detach (this=0x8f8f8f8f8f8f8f8f, index=...) at storage/innobase/include/dict0mem.h:1333
      #6  0x0000563f9606e690 in dict_index_t::detach_columns (this=0x7ffa98168780) at storage/innobase/include/dict0mem.h:1192
      #7  0x0000563f9639bfb9 in dict_mem_index_free (index=0x7ffa98168780) at storage/innobase/dict/dict0mem.cc:1073
      #8  0x0000563f9605da59 in prepare_inplace_alter_table_dict (ha_alter_info=0x7ffae3165770, altered_table=0x7ffae3165800, old_table=0x7ffa9811c1f8, table_name=0x7ffa98118155 "t4", flags=1, flags2=80, fts_doc_id_col=18446744073709551615, add_fts_doc_id=false, add_fts_doc_id_idx=false) at storage/innobase/handler/handler0alter.cc:6763
      #9  0x0000563f960633e6 in ha_innobase::prepare_inplace_alter_table (this=0x7ffa98119450, altered_table=0x7ffae3165800, ha_alter_info=0x7ffae3165770) at storage/innobase/handler/handler0alter.cc:8141
      #10 0x0000563f95dd5f32 in handler::ha_prepare_inplace_alter_table (this=0x7ffa98119450, altered_table=0x7ffae3165800, ha_alter_info=0x7ffae3165770) at sql/handler.cc:4548
      #11 0x0000563f95b5633b in mysql_inplace_alter_table (thd=0x7ffa98000cd8, table_list=0x7ffa98012768, table=0x7ffa9811c1f8, altered_table=0x7ffae3165800, ha_alter_info=0x7ffae3165770, inplace_supported=HA_ALTER_INPLACE_COPY_LOCK, target_mdl_request=0x7ffae31665d0, alter_ctx=0x7ffae3167100) at sql/sql_table.cc:7690
      #12 0x0000563f95b5d3ce in mysql_alter_table (thd=0x7ffa98000cd8, new_db=0x7ffa98005498, new_name=0x7ffa980058a0, create_info=0x7ffae3167ce0, table_list=0x7ffa98012768, alter_info=0x7ffae3167c20, order_num=0, order=0x0, ignore=false) at sql/sql_table.cc:10119
      #13 0x0000563f95bf52ba in Sql_cmd_alter_table::execute (this=0x7ffa98012f58, thd=0x7ffa98000cd8) at sql/sql_alter.cc:502
      #14 0x0000563f95a74059 in mysql_execute_command (thd=0x7ffa98000cd8) at sql/sql_parse.cc:6101
      #15 0x0000563f95a7970d in mysql_parse (thd=0x7ffa98000cd8, rawbuf=0x7ffa98012660 "ALTER TABLE t4 ADD COLUMN col_text_copy TEXT", length=44, parser_state=0x7ffae3169150, is_com_multi=false, is_next_command=false) at sql/sql_parse.cc:7900
      #16 0x0000563f95a648f5 in dispatch_command (command=COM_QUERY, thd=0x7ffa98000cd8, packet=0x7ffa98008199 "ALTER TABLE t4 ADD COLUMN col_text_copy TEXT ", packet_length=45, is_com_multi=false, is_next_command=false) at sql/sql_parse.cc:1841
      #17 0x0000563f95a62f81 in do_command (thd=0x7ffa98000cd8) at sql/sql_parse.cc:1359
      #18 0x0000563f95bebbd9 in do_handle_one_connection (connect=0x563f9957d258) at sql/sql_connect.cc:1412
      #19 0x0000563f95beb928 in handle_one_connection (arg=0x563f9957d258) at sql/sql_connect.cc:1316
      #20 0x0000563f965ef020 in pfs_spawn_thread (arg=0x563f994bde98) at storage/perfschema/pfs.cc:1869
      #21 0x00007ffaeec887fc in start_thread (arg=0x7ffae316a700) at pthread_create.c:465
      #22 0x00007ffaedebeb5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
       
      Replay also on 10.5.2 commit 84e3f9ce84c3e7fce70142cff4bea1c8b916810b
      No replay on 10.3.23 commit b0fa30808622fe12d474a70af1838906e60b9897
      No replay on 10.2.31 commit 7d313214641241e8ce839d9de01529c2335c620f
      
      

      Attachments

        Issue Links

          Activity

            Looks very much like MDEV-20114.

            elenst Elena Stepanova added a comment - Looks very much like MDEV-20114 .

            These SEGV's can be somehow siblings. But the DDL differs.

            mleich Matthias Leich added a comment - These SEGV's can be somehow siblings. But the DDL differs.

            I debugged this a little. The test will not crash if I add the following initialization:

            diff --git a/storage/innobase/row/row0merge.cc b/storage/innobase/row/row0merge.cc
            index 6a428c9ed0b..a49e8e4e6ef 100644
            --- a/storage/innobase/row/row0merge.cc
            +++ b/storage/innobase/row/row0merge.cc
            @@ -4446,6 +4446,8 @@ row_merge_create_index(
             		}
             
             		dict_mem_index_add_field(index, name, ifield->prefix_len);
            +
            +		index->fields[i].col = nullptr;
             	}
             
             	DBUG_RETURN(index);
            

            But, I did not debug deep enough to check whether this actually is the correct fix. The bug might affect 10.3 as well and be caused by MDEV-11369. Here is the relevant output with the above fix:

            10.4 dc22acfdb62745017226a5c698c1bc3ee3e3563e with fix

            CREATE TABLE t4 ( col1 INT PRIMARY KEY, col_text_g TEXT GENERATED ALWAYS AS (SUBSTR(col_text,1,499)), col_varchar VARCHAR(500), col_text TEXT ) ENGINE = InnoDB ROW_FORMAT = Compact ;
            ALTER TABLE t4 ADD UNIQUE KEY ( col_text_g(9) ) ;
            ALTER TABLE t4 ADD COLUMN col_text_copy TEXT ;
            ERROR 42000: Row size too large. The maximum row size for the used table type, not counting BLOBs, is 1982. This includes storage overhead, check the manual. You have to change some columns to TEXT or BLOBs
            

            The crash would occur in the error handling of the last statement. In this case, I believe that we are attempting instant ADD COLUMN. For some reason, the dict_index_t of col_text_g would be uninitialized (col=0x1 in my case).

            Note: remember innodb_page_size=4k when running the test.

            marko Marko Mäkelä added a comment - I debugged this a little. The test will not crash if I add the following initialization: diff --git a/storage/innobase/row/row0merge.cc b/storage/innobase/row/row0merge.cc index 6a428c9ed0b..a49e8e4e6ef 100644 --- a/storage/innobase/row/row0merge.cc +++ b/storage/innobase/row/row0merge.cc @@ -4446,6 +4446,8 @@ row_merge_create_index( } dict_mem_index_add_field(index, name, ifield->prefix_len); + + index->fields[i].col = nullptr; } DBUG_RETURN(index); But, I did not debug deep enough to check whether this actually is the correct fix. The bug might affect 10.3 as well and be caused by MDEV-11369 . Here is the relevant output with the above fix: 10.4 dc22acfdb62745017226a5c698c1bc3ee3e3563e with fix CREATE TABLE t4 ( col1 INT PRIMARY KEY, col_text_g TEXT GENERATED ALWAYS AS (SUBSTR(col_text,1,499)), col_varchar VARCHAR(500), col_text TEXT ) ENGINE = InnoDB ROW_FORMAT = Compact ; ALTER TABLE t4 ADD UNIQUE KEY ( col_text_g(9) ) ; ALTER TABLE t4 ADD COLUMN col_text_copy TEXT ; ERROR 42000: Row size too large. The maximum row size for the used table type, not counting BLOBs, is 1982. This includes storage overhead, check the manual. You have to change some columns to TEXT or BLOBs The crash would occur in the error handling of the last statement. In this case, I believe that we are attempting instant ADD COLUMN . For some reason, the dict_index_t of col_text_g would be uninitialized ( col=0x1 in my case). Note: remember innodb_page_size=4k when running the test.

            Root cause is that Failure of DDL tries to access the uninitialized columns of newly created index. The problem is not there in 10.3
            In dict_index_remove_from_v_col_list(), it checks for index->cached to know whether index is not completely formed. Problem
            exist only in 10.4+. It is caused by MDEV-19606. The following fix can be applied:

            diff --git a/storage/innobase/include/dict0mem.h b/storage/innobase/include/dict0mem.h
            index 35f05f40e2c..259da23fcd9 100644
            --- a/storage/innobase/include/dict0mem.h
            +++ b/storage/innobase/include/dict0mem.h
            @@ -1203,7 +1203,7 @@ struct dict_index_t {
               @param   whether to reset fields[].col */
               void detach_columns(bool clear= false)
               {
            -    if (!has_virtual())
            +    if (!has_virtual() || !cached)
                   return;
                 for (unsigned i= 0; i < n_fields; i++)
                 {
            
            

            thiru Thirunarayanan Balathandayuthapani added a comment - - edited Root cause is that Failure of DDL tries to access the uninitialized columns of newly created index. The problem is not there in 10.3 In dict_index_remove_from_v_col_list(), it checks for index->cached to know whether index is not completely formed. Problem exist only in 10.4+. It is caused by MDEV-19606 . The following fix can be applied: diff --git a/storage/innobase/include/dict0mem.h b/storage/innobase/include/dict0mem.h index 35f05f40e2c..259da23fcd9 100644 --- a/storage/innobase/include/dict0mem.h +++ b/storage/innobase/include/dict0mem.h @@ -1203,7 +1203,7 @@ struct dict_index_t { @param whether to reset fields[].col */ void detach_columns( bool clear= false ) { - if (!has_virtual()) + if (!has_virtual() || !cached) return ; for (unsigned i= 0; i < n_fields; i++) {

            Pushed the fix:

            commit fdb37d084b04be35c080de78ac01823bfaaf02a4 (HEAD -> 10.4-sample, origin/HEAD, origin/10.4)
            Author: Thirunarayanan Balathandayuthapani <thiru@mariadb.com>
            Date:   Tue May 26 20:25:41 2020 +0530
             
                MDEV-21787 Alter table failure tries to access uninitialized column
                
            

            thiru Thirunarayanan Balathandayuthapani added a comment - Pushed the fix: commit fdb37d084b04be35c080de78ac01823bfaaf02a4 (HEAD -> 10.4-sample, origin/HEAD, origin/10.4) Author: Thirunarayanan Balathandayuthapani <thiru@mariadb.com> Date: Tue May 26 20:25:41 2020 +0530   MDEV-21787 Alter table failure tries to access uninitialized column

            People

              thiru Thirunarayanan Balathandayuthapani
              mleich Matthias Leich
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.