Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-29856

ASAN heap-use-after-poison in row_merge_spatial_rows() with PRIMARY KEY on column prefix

Details

    Description

      It looks very similar to MDEV-29520 which is already fixed in all branches.

      --source include/have_innodb.inc
       
      CREATE TABLE t (id INT, f VARCHAR(8192), s POINT NOT NULL DEFAULT POINT(0,0), PRIMARY KEY(id,f(8)), SPATIAL(s)) ENGINE=InnoDB;
      INSERT INTO t (id,f) VALUES (1,REPEAT('x',8067));
      INSERT INTO t (id,f) VALUES (2,'');
      ALTER TABLE t FORCE;
       
      # Cleanup
      DROP TABLE t;
      

      10.3 9de37e07

      ==3901273==ERROR: AddressSanitizer: use-after-poison on address 0x63100008c9d0 at pc 0x7f2a5c459983 bp 0x7f2a451ee140 sp 0x7f2a451ed8f0
      READ of size 8 at 0x63100008c9d0 thread T27
          #0 0x7f2a5c459982 in __interceptor_memcpy ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:806
          #1 0x56212087568e in rec_convert_dtuple_to_rec_comp<false> /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/innobase/rem/rem0rec.cc:1597
          #2 0x56212086a750 in rec_convert_dtuple_to_rec_new /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/innobase/rem/rem0rec.cc:1633
          #3 0x56212086aa99 in rec_convert_dtuple_to_rec(unsigned char*, dict_index_t const*, dtuple_t const*, unsigned long) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/innobase/rem/rem0rec.cc:1662
          #4 0x562120b3c6dc in page_cur_tuple_insert /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/innobase/include/page0cur.inl:277
          #5 0x562120b541af in btr_cur_optimistic_insert(unsigned long, btr_cur_t*, unsigned short**, mem_block_info_t**, dtuple_t*, unsigned char**, big_rec_t**, unsigned long, que_thr_t*, mtr_t*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/innobase/btr/btr0cur.cc:3457
          #6 0x5621208fc6ee in index_tuple_info_t::insert(unsigned long, mem_block_info_t*, btr_pcur_t*, mtr_t*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/innobase/row/row0merge.cc:186
          #7 0x5621208e2e30 in row_merge_spatial_rows /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/innobase/row/row0merge.cc:1594
          #8 0x5621208e4e7c in row_merge_read_clustered_index /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/innobase/row/row0merge.cc:1949
          #9 0x5621208f7ab2 in row_merge_build_indexes(trx_t*, dict_table_t*, dict_table_t*, bool, dict_index_t**, unsigned long const*, unsigned long, TABLE*, dtuple_t const*, unsigned long const*, unsigned long, ib_sequence_t&, bool, ut_stage_alter_t*, dict_add_v_col_t const*, TABLE*, bool) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/innobase/row/row0merge.cc:4736
          #10 0x56212069ebec in ha_innobase::inplace_alter_table(TABLE*, Alter_inplace_info*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/innobase/handler/handler0alter.cc:7209
          #11 0x56211fc33395 in handler::ha_inplace_alter_table(TABLE*, Alter_inplace_info*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/handler.h:4147
          #12 0x56211fc183c6 in mysql_inplace_alter_table /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_table.cc:7774
          #13 0x56211fc2845e in mysql_alter_table(THD*, st_mysql_const_lex_string const*, st_mysql_const_lex_string const*, HA_CREATE_INFO*, TABLE_LIST*, Alter_info*, unsigned int, st_order*, bool) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_table.cc:10159
          #14 0x56211fd70a41 in Sql_cmd_alter_table::execute(THD*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_alter.cc:512
          #15 0x56211fa0b459 in mysql_execute_command(THD*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_parse.cc:6076
          #16 0x56211fa16017 in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_parse.cc:7855
          #17 0x56211f9ef291 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_parse.cc:1852
          #18 0x56211f9ec4d6 in do_command(THD*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_parse.cc:1398
          #19 0x56211fd61f98 in do_handle_one_connection(CONNECT*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_connect.cc:1403
          #20 0x56211fd61894 in handle_one_connection /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_connect.cc:1308
          #21 0x562121218e6c in pfs_spawn_thread /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/perfschema/pfs.cc:1869
          #22 0x7f2a5bbc3ea6 in start_thread nptl/pthread_create.c:477
          #23 0x7f2a5bae3aee in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xfcaee)
       
      0x63100008c9d0 is located 464 bytes inside of 65664-byte region [0x63100008c800,0x63100009c880)
      allocated by thread T27 here:
          #0 0x7f2a5c4c9e8f in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
          #1 0x5621207a507f in mem_heap_create_block_func(mem_block_info_t*, unsigned long, char const*, unsigned int, unsigned long) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/innobase/mem/mem0mem.cc:277
          #2 0x5621208cf916 in mem_heap_create_func /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/innobase/include/mem0mem.inl:375
          #3 0x5621208e4a01 in row_merge_read_clustered_index /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/innobase/row/row0merge.cc:1891
          #4 0x5621208f7ab2 in row_merge_build_indexes(trx_t*, dict_table_t*, dict_table_t*, bool, dict_index_t**, unsigned long const*, unsigned long, TABLE*, dtuple_t const*, unsigned long const*, unsigned long, ib_sequence_t&, bool, ut_stage_alter_t*, dict_add_v_col_t const*, TABLE*, bool) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/innobase/row/row0merge.cc:4736
          #5 0x56212069ebec in ha_innobase::inplace_alter_table(TABLE*, Alter_inplace_info*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/innobase/handler/handler0alter.cc:7209
          #6 0x56211fc33395 in handler::ha_inplace_alter_table(TABLE*, Alter_inplace_info*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/handler.h:4147
          #7 0x56211fc183c6 in mysql_inplace_alter_table /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_table.cc:7774
          #8 0x56211fc2845e in mysql_alter_table(THD*, st_mysql_const_lex_string const*, st_mysql_const_lex_string const*, HA_CREATE_INFO*, TABLE_LIST*, Alter_info*, unsigned int, st_order*, bool) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_table.cc:10159
          #9 0x56211fd70a41 in Sql_cmd_alter_table::execute(THD*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_alter.cc:512
          #10 0x56211fa0b459 in mysql_execute_command(THD*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_parse.cc:6076
          #11 0x56211fa16017 in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_parse.cc:7855
          #12 0x56211f9ef291 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_parse.cc:1852
          #13 0x56211f9ec4d6 in do_command(THD*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_parse.cc:1398
          #14 0x56211fd61f98 in do_handle_one_connection(CONNECT*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_connect.cc:1403
          #15 0x56211fd61894 in handle_one_connection /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_connect.cc:1308
          #16 0x562121218e6c in pfs_spawn_thread /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/perfschema/pfs.cc:1869
          #17 0x7f2a5bbc3ea6 in start_thread nptl/pthread_create.c:477
       
      Thread T27 created by T0 here:
          #0 0x7f2a5c4752a2 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:214
          #1 0x562121219259 in spawn_thread_v1 /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/perfschema/pfs.cc:1919
          #2 0x56211f763d76 in inline_mysql_thread_create /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/include/mysql/psi/mysql_thread.h:1275
          #3 0x56211f779d43 in create_thread_to_handle_connection(CONNECT*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/mysqld.cc:6678
          #4 0x56211f77a2df in create_new_thread /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/mysqld.cc:6748
          #5 0x56211f77b34c in handle_connections_sockets() /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/mysqld.cc:7006
          #6 0x56211f7794bd in mysqld_main(int, char**) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/mysqld.cc:6300
          #7 0x56211f762604 in main /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/main.cc:25
          #8 0x7f2a5ba0ad09 in __libc_start_main ../csu/libc-start.c:308
       
      SUMMARY: AddressSanitizer: use-after-poison ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:806 in __interceptor_memcpy
      Shadow bytes around the buggy address:
        0x0c62800098e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x0c62800098f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x0c6280009900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
        0x0c6280009910: f7 00 00 00 00 02 f7 00 00 00 00 00 00 00 00 00
        0x0c6280009920: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      =>0x0c6280009930: f7 00 00 00 00 00 00 03 f7 f7[f7]f7 f7 f7 f7 f7
        0x0c6280009940: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
        0x0c6280009950: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
        0x0c6280009960: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
        0x0c6280009970: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
        0x0c6280009980: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
      Shadow byte legend (one shadow byte represents 8 application bytes):
        Addressable:           00
        Partially addressable: 01 02 03 04 05 06 07 
        Heap left redzone:       fa
        Freed heap region:       fd
        Stack left redzone:      f1
        Stack mid redzone:       f2
        Stack right redzone:     f3
        Stack after return:      f5
        Stack use after scope:   f8
        Global redzone:          f9
        Global init order:       f6
        Poisoned by user:        f7
        Container overflow:      fc
        Array cookie:            ac
        Intra object redzone:    bb
        ASan internal:           fe
        Left alloca redzone:     ca
        Right alloca redzone:    cb
        Shadow gap:              cc
      ==3901273==ABORTING
      

      Attachments

        Issue Links

          Activity

            The memory was poisoned earlier in the same thread:

            10.3 01ac7455e2c974affbf335c4bc1acbfe71b7127b

            #3  0x0000557689c3fe10 in mem_heap_empty (heap=0x63100008c800)
                at /mariadb/10.3/storage/innobase/include/mem0mem.inl:290
            #4  0x0000557689c57165 in row_merge_read_clustered_index (…)
                at /mariadb/10.3/storage/innobase/row/row0merge.cc:2034
            

            Before this, the memory had been initialized in:

            #1  0x000055768978e70b in mem_heap_dup (heap=0x610000008140, 
                data=0x63100008c9d0, len=8)
                at /mariadb/10.3/storage/innobase/include/mem0mem.h:241
            #2  0x0000557689c42738 in dfield_dup (field=field@entry=0x615000092060, 
                heap=<optimized out>)
                at /mariadb/10.3/storage/innobase/include/data0data.inl:173
            #3  0x0000557689c4e164 in row_merge_buf_add (buf=buf@entry=0x615000092008, 
                fts_index=fts_index@entry=0x0, old_table=old_table@entry=0x6190000d5308, 
                new_table=new_table@entry=0x6190000da308, psort_info=psort_info@entry=0x0, 
                row=row@entry=0x63100008c8c8, ext=<optimized out>, doc_id=<optimized out>, 
                conv_heap=<optimized out>, err=<optimized out>, v_heap=<optimized out>, 
                my_table=<optimized out>, trx=<optimized out>)
                at /mariadb/10.3/storage/innobase/row/row0merge.cc:839
            #4  0x0000557689c59508 in row_merge_read_clustered_index (…)
                at /mariadb/10.3/storage/innobase/row/row0merge.cc:2355
            

            There is some strange caching logic in row_merge_spatial_rows() that appears to cover several rows, separately from the merge_buf[]->heap.

            marko Marko Mäkelä added a comment - The memory was poisoned earlier in the same thread: 10.3 01ac7455e2c974affbf335c4bc1acbfe71b7127b #3 0x0000557689c3fe10 in mem_heap_empty (heap=0x63100008c800) at /mariadb/10.3/storage/innobase/include/mem0mem.inl:290 #4 0x0000557689c57165 in row_merge_read_clustered_index (…) at /mariadb/10.3/storage/innobase/row/row0merge.cc:2034 Before this, the memory had been initialized in: #1 0x000055768978e70b in mem_heap_dup (heap=0x610000008140, data=0x63100008c9d0, len=8) at /mariadb/10.3/storage/innobase/include/mem0mem.h:241 #2 0x0000557689c42738 in dfield_dup (field=field@entry=0x615000092060, heap=<optimized out>) at /mariadb/10.3/storage/innobase/include/data0data.inl:173 #3 0x0000557689c4e164 in row_merge_buf_add (buf=buf@entry=0x615000092008, fts_index=fts_index@entry=0x0, old_table=old_table@entry=0x6190000d5308, new_table=new_table@entry=0x6190000da308, psort_info=psort_info@entry=0x0, row=row@entry=0x63100008c8c8, ext=<optimized out>, doc_id=<optimized out>, conv_heap=<optimized out>, err=<optimized out>, v_heap=<optimized out>, my_table=<optimized out>, trx=<optimized out>) at /mariadb/10.3/storage/innobase/row/row0merge.cc:839 #4 0x0000557689c59508 in row_merge_read_clustered_index (…) at /mariadb/10.3/storage/innobase/row/row0merge.cc:2355 There is some strange caching logic in row_merge_spatial_rows() that appears to cover several rows, separately from the merge_buf[]->heap .

            I think that I finally figured it out. The ext that is being passed to index_tuple_info_t::add() was allocated from row_heap, which would have been emptied before the index_tuple_info_t::insert() is executed. The lifetime of row_heap should be only a single row of the clustered index, while the row_merge_spatial_rows() will be invoked at the end of a larger batch of rows.

            The memory management of the SPATIAL INDEX specific data structure index_tuple_info_t is very poor. After this fix, the sp_heap (or its replacement) may consume excessive amounts of memory, per batch.

            marko Marko Mäkelä added a comment - I think that I finally figured it out. The ext that is being passed to index_tuple_info_t::add() was allocated from row_heap , which would have been emptied before the index_tuple_info_t::insert() is executed. The lifetime of row_heap should be only a single row of the clustered index, while the row_merge_spatial_rows() will be invoked at the end of a larger batch of rows. The memory management of the SPATIAL INDEX specific data structure index_tuple_info_t is very poor. After this fix, the sp_heap (or its replacement) may consume excessive amounts of memory, per batch.

            I tried to simplify the test case, but the simplest I came up with was this:

            --source include/have_innodb.inc
            CREATE TABLE t (id INT, f TEXT, s POINT NOT NULL,
                            PRIMARY KEY(id,f(1)), SPATIAL(s)) ENGINE=InnoDB;
            INSERT INTO t VALUES
            (1,REPEAT('x',8192),@p:=ST_GeomFromText('POINT(0 0)')),(2,'',@p);
            ALTER TABLE t FORCE;
            DROP TABLE t;
            

            For some reason, if I use id=1 for both records or if I remove the column id, the test will pass even when the my was absent. Using a different value for s did not help.

            I would think that column prefixes in the PRIMARY KEY are not awfully common.

            marko Marko Mäkelä added a comment - I tried to simplify the test case, but the simplest I came up with was this: --source include/have_innodb.inc CREATE TABLE t (id INT , f TEXT, s POINT NOT NULL , PRIMARY KEY (id,f(1)), SPATIAL(s)) ENGINE=InnoDB; INSERT INTO t VALUES (1,REPEAT( 'x' ,8192),@p:=ST_GeomFromText( 'POINT(0 0)' )),(2, '' ,@p); ALTER TABLE t FORCE ; DROP TABLE t; For some reason, if I use id=1 for both records or if I remove the column id , the test will pass even when the my was absent. Using a different value for s did not help. I would think that column prefixes in the PRIMARY KEY are not awfully common.

            I was able to remove sp_heap as part of fixing this. The memory usage during SPATIAL INDEX creation should be rather reasonable now.

            marko Marko Mäkelä added a comment - I was able to remove sp_heap as part of fixing this. The memory usage during SPATIAL INDEX creation should be rather reasonable now.

            People

              marko Marko Mäkelä
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.