Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-34919

Aria crashes with high-level (vector) indexes

Details

    Description

      Test:

      create table t1(v blob not null, vector index(v)) engine=aria;
      insert into t1 values(x'e360d63ebe554f3fcdbc523f4522193f5236083d');
      

      There're at least 2 crashes, one concerns rec_reflength, on create we get 5 (which looks correct), on open we get 6 (which doesn't look right):

      maria/ma_blockrec.c:2855(write_block_record)[0x56440c6d483e]
      maria/ma_blockrec.c:3584(allocate_and_write_block_record)[0x56440c6d6ab6]
      maria/ma_blockrec.c:3624(_ma_write_init_block_record)[0x56440c6d6bd0]
      maria/ma_write.c:158(maria_write)[0x56440c6ea1de]
      maria/ha_maria.cc:1235(ha_maria::write_row(unsigned char const*))[0x56440c66423b]
      sql/handler.cc:8131(handler::ha_write_row(unsigned char const*))[0x56440c3a57dd]
      sql/vector_mhnsw.cc:1015(FVectorNode::save(TABLE*))[0x56440c5578e9]
      sql/vector_mhnsw.cc:1203(mhnsw_insert(TABLE*, st_key*))[0x56440c558be1]
      sql/sql_base.cc:9926(TABLE::hlindexes_on_insert())[0x56440bec8a15]
      sql/handler.cc:8135(handler::ha_write_row(unsigned char const*))[0x56440c3a5840]
      sql/sql_insert.cc:2322(write_record(THD*, TABLE*, st_copy_info*, select_result*))[0x56440bf21e14]
      sql/sql_insert.cc:1179(mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool, select_result*))[0x56440bf1e30a]
      sql/sql_parse.cc:4458(mysql_execute_command(THD*, bool))[0x56440bf760d9]
      sql/sql_parse.cc:7873(mysql_parse(THD*, char*, unsigned int, Parser_state*))[0x56440bf816d1]
      sql/sql_parse.cc:1894(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool))[0x56440bf6e244]
      sql/sql_parse.cc:1405(do_command(THD*, bool))[0x56440bf6cbf8]
      sql/sql_connect.cc:1448(do_handle_one_connection(CONNECT*, bool))[0x56440c1694f3]
      sql/sql_connect.cc:1352(handle_one_connection)[0x56440c16924f]
      perfschema/pfs.cc:2200(pfs_spawn_thread)[0x56440c75b73f]
      nptl/pthread_create.c:478(start_thread)[0x7f73302de609]
      

      Another - missing start_trans:

      maria/ma_loghandler.c:6417(translog_write_record)[0x561dcefa6afd]
      maria/ma_blockrec.c:3162(write_block_record)[0x561dceff695f]
      maria/ma_blockrec.c:3584(allocate_and_write_block_record)[0x561dceff7b08]
      maria/ma_blockrec.c:3624(_ma_write_init_block_record)[0x561dceff7c22]
      maria/ma_write.c:158(maria_write)[0x561dcf00b230]
      maria/ha_maria.cc:1235(ha_maria::write_row(unsigned char const*))[0x561dcef8528d]
      sql/handler.cc:8131(handler::ha_write_row(unsigned char const*))[0x561dcecc67dd]
      sql/vector_mhnsw.cc:1015(FVectorNode::save(TABLE*))[0x561dcee788e9]
      sql/vector_mhnsw.cc:1203(mhnsw_insert(TABLE*, st_key*))[0x561dcee79be1]
      sql/sql_base.cc:9926(TABLE::hlindexes_on_insert())[0x561dce7e9a15]
      sql/handler.cc:8135(handler::ha_write_row(unsigned char const*))[0x561dcecc6840]
      sql/sql_insert.cc:2322(write_record(THD*, TABLE*, st_copy_info*, select_result*))[0x561dce842e14]
      sql/sql_insert.cc:1179(mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool, select_result*))[0x561dce83f30a]
      sql/sql_parse.cc:4458(mysql_execute_command(THD*, bool))[0x561dce8970d9]
      sql/sql_parse.cc:7873(mysql_parse(THD*, char*, unsigned int, Parser_state*))[0x561dce8a26d1]
      sql/sql_parse.cc:1894(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool))[0x561dce88f244]
      sql/sql_parse.cc:1405(do_command(THD*, bool))[0x561dce88dbf8]
      sql/sql_connect.cc:1448(do_handle_one_connection(CONNECT*, bool))[0x561dcea8a4f3]
      sql/sql_connect.cc:1352(handle_one_connection)[0x561dcea8a24f]
      perfschema/pfs.cc:2200(pfs_spawn_thread)[0x561dcf07c791]
      nptl/pthread_create.c:478(start_thread)[0x7f00c7366609]
      

      rec_reflength fix prototype:

      diff --git a/storage/maria/ma_info.c b/storage/maria/ma_info.c
      index 8434709997a..882c8f4c5c0 100644
      --- a/storage/maria/ma_info.c
      +++ b/storage/maria/ma_info.c
      @@ -78,8 +78,12 @@ int maria_status(MARIA_HA *info, register MARIA_INFO *x, uint flag)
           x->filenr   = info->dfile.file;
           x->options  = share->options;
           x->create_time=share->state.create_time;
      -    x->reflength= maria_get_pointer_length(share->base.max_data_file_length,
      -                                        maria_data_pointer_size);
      +    if (share->data_file_type == BLOCK_RECORD)
      +      x->reflength= maria_get_pointer_length(share->base.max_data_file_length /
      +                                             share->base.block_size * 2, 4) + 1;
      +    else
      +      x->reflength= maria_get_pointer_length(share->base.max_data_file_length,
      +                                             maria_data_pointer_size);
           x->record_offset= (info->s->data_file_type == STATIC_RECORD ?
                              share->base.pack_reclength: 0);
           x->sortkey= -1;                            /* No clustering */
      

      Relevant commits:

      commit fe87d24f13fead09fb73db948e8953aca21a6637
      Author: Sergey Vojtovich <svoj@mariadb.org>
      Date:   Fri Sep 13 11:44:35 2024 +0400
       
          Disabled high-level indexes with Aria
       
          ... until a few bugs that cause server crash are fixed.
       
       
      commit ca5ec0200b4a87a9618590961c71d37afbfa1c38
      Author: Sergei Golubchik <serg@mariadb.org>
      Date:   Fri Jun 7 13:50:13 2024 +0200
       
          mhnsw: build indexes with the columns of exactly right size
      

      Attachments

        Issue Links

          Activity

            The first crash was fixed by

            commit b29dca95bd255ae9551fd319f1e8ecbaaae4b4ac
            Author: Sergei Golubchik <serg@mariadb.org>
            Date:   Sun Oct 6 20:09:08 2024 +0200
             
                make MyISAM and Aria report correct reflength to the server
             
                MyISAM and Aria used to lie to the server about the reflength value.
                One value was used internally, it was stored on disk, e.g. in indexes,
                and couldn't be changed without full table rebuild. A differently
                calculated value was reported to the server - that value was sometimes
                larger than the true reflength.
             
                That caused the server to allocate more memory per position than
                necessary - affecting filesort, join buffer usage, optimizer cost
                calculations, and may be more.
            

            svoj Sergey Vojtovich added a comment - The first crash was fixed by commit b29dca95bd255ae9551fd319f1e8ecbaaae4b4ac Author: Sergei Golubchik <serg@mariadb.org> Date: Sun Oct 6 20:09:08 2024 +0200   make MyISAM and Aria report correct reflength to the server   MyISAM and Aria used to lie to the server about the reflength value. One value was used internally, it was stored on disk, e.g. in indexes, and couldn't be changed without full table rebuild. A differently calculated value was reported to the server - that value was sometimes larger than the true reflength.   That caused the server to allocate more memory per position than necessary - affecting filesort, join buffer usage, optimizer cost calculations, and may be more.

            People

              serg Sergei Golubchik
              svoj Sergey Vojtovich
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.