Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-19114

gcol.innodb_virtual_debug failed in buildbot with Assertion `n_fields > 0' failure

Details

    Description

      http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/16894/steps/mtr_ps_emb/logs/stdio

      10.3 f03f4da66373161d604b8ecf3c23ae18d08c0461

      		 
      gcol.innodb_virtual_debug 'innodb'       w4 [ fail ]
      		 
              Test ended at 2019-03-25 05:16:44
      		 
       
      		 
      CURRENT_TEST: gcol.innodb_virtual_debug
      		 
      Warning: /mnt/buildbot/build/mariadb-10.3.14/libmysqld/examples/mysqltest_embedded: unknown variable 'loose-ssl-ca=/mnt/buildbot/build/mariadb-10.3.14/mysql-test/std_data/cacert.pem'
      		 
      Warning: /mnt/buildbot/build/mariadb-10.3.14/libmysqld/examples/mysqltest_embedded: unknown variable 'loose-ssl-cert=/mnt/buildbot/build/mariadb-10.3.14/mysql-test/std_data/client-cert.pem'
      		 
      Warning: /mnt/buildbot/build/mariadb-10.3.14/libmysqld/examples/mysqltest_embedded: unknown variable 'loose-ssl-key=/mnt/buildbot/build/mariadb-10.3.14/mysql-test/std_data/client-key.pem'
      		 
      Warning: /mnt/buildbot/build/mariadb-10.3.14/libmysqld/examples/mysqltest_embedded: unknown option '--loose-skip-ssl'
      		 
      mysqltest_embedded: /home/buildbot/buildbot/build/mariadb-10.3.14/storage/innobase/include/rem0rec.ic:1244: void rec_offs_set_n_fields(ulint*, ulint): Assertion `n_fields > 0' failed.
      		 
      mysqltest got signal 6
      		 
      read_command_buf (0x82d91c98): ALTE
      		 
      conn->name (0x82e14dd8): 
      conn->cur_query (0x83081df8): ALTER TABLE t ADD INDEX idc(c)
      		 
      Attempting backtrace...
      		 
      stack_bottom = 0x0 thread_stack 0x49000
      		 
      /mnt/buildbot/build/mariadb-10.3.14/libmysqld/examples/mysqltest_embedded(my_print_stacktrace+0x3c)[0x8038ddb6]
      		 
      /mnt/buildbot/build/mariadb-10.3.14/libmysqld/examples/mysqltest_embedded(+0x30e5bd)[0x803705bd]
      		 
      mysys/stacktrace.c:269(my_print_stacktrace)[0x803705fb]
      		 
      addr2line: '': No such file
      		 
      [0xb77a9c14]
      		 
      [0xb77a9c31]
      		 
      /lib/i386-linux-gnu/libc.so.6(gsignal+0x39)[0xb7115e89]
      		 
      /lib/i386-linux-gnu/libc.so.6(abort+0x157)[0xb71173e7]
      		 
      /lib/i386-linux-gnu/libc.so.6(+0x24d07)[0xb710ed07]
      		 
      /lib/i386-linux-gnu/libc.so.6(+0x24d8b)[0xb710ed8b]
      		 
      /mnt/buildbot/build/mariadb-10.3.14/libmysqld/examples/mysqltest_embedded(+0xb4bd35)[0x80badd35]
      		 
      /mnt/buildbot/build/mariadb-10.3.14/libmysqld/examples/mysqltest_embedded(+0xb4e0c8)[0x80bb00c8]
      		 
      /mnt/buildbot/build/mariadb-10.3.14/libmysqld/examples/mysqltest_embedded(+0xafcc24)[0x80b5ec24]
      		 
      /mnt/buildbot/build/mariadb-10.3.14/libmysqld/examples/mysqltest_embedded(+0xace12a)[0x80b3012a]
      		 
      include/rem0rec.ic:1245(rec_offs_set_n_fields(unsigned int*, unsigned int))[0x80ab9962]
      		 
      rem/rem0rec.cc:867(rec_get_offsets_func(unsigned char const*, dict_index_t const*, unsigned int*, bool, unsigned int, char const*, unsigned int, mem_block_info_t**))[0x80abc1bb]
      		 
      page/page0cur.cc:745(page_cur_search_with_match_bytes(buf_block_t const*, dict_index_t const*, dtuple_t const*, page_cur_mode_t, unsigned int*, unsigned int*, unsigned int*, unsigned int*, page_cur_t*))[0x80bea076]
      		 
      btr/btr0cur.cc:1837(btr_cur_search_to_nth_level_func(dict_index_t*, unsigned int, dtuple_t const*, page_cur_mode_t, unsigned int, btr_cur_t*, rw_lock_t*, char const*, unsigned int, mtr_t*, unsigned long long))[0x80bea630]
      		 
      include/btr0pcur.ic:459(btr_pcur_open_low(dict_index_t*, unsigned int, dtuple_t const*, page_cur_mode_t, unsigned int, btr_pcur_t*, char const*, unsigned int, unsigned long long, mtr_t*))[0x80beb251]
      		 
      row/row0row.cc:1136(row_search_index_entry(dict_index_t*, dtuple_t const*, unsigned int, btr_pcur_t*, mtr_t*))[0x80bebfd7]
      		 
      row/row0purge.cc:569(row_purge_remove_sec_if_poss_leaf(purge_node_t*, dict_index_t*, dtuple_t const*))[0x80bec195]
      		 
      row/row0purge.cc:893(row_purge_upd_exist_or_extern_func(que_thr_t const*, purge_node_t*, unsigned char*))[0x80bec3ca]
      		 
      row/row0purge.cc:1206(row_purge_record_func(purge_node_t*, unsigned char*, que_thr_t const*, bool))[0x80bf29d2]
      		 
      row/row0purge.cc:1250(row_purge(purge_node_t*, unsigned char*, que_thr_t*))[0x80bf2c17]
      		 
      row/row0purge.cc:1311(row_purge_step(que_thr_t*))[0x80bf2e19]
      		 
      que/que0que.cc:1042(que_thr_step(que_thr_t*))[0x80aab044]
      		 
      trx/trx0purge.cc:1603(trx_purge(unsigned int, bool))[0x80d007c4]
      		 
      srv/srv0srv.cc:2595(srv_do_purge(unsigned int*))[0x80d00c09]
      		 
      /lib/i386-linux-gnu/libpthread.so.0(+0x62b5)[0xb77802b5]
      		 
      /lib/i386-linux-gnu/libc.so.6(clone+0x6e)[0xb71d116e]
      		 
      Writing a core file...

      Attachments

        Issue Links

          is caused by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-16376 ASAN: heap-use-after-free in gcol.innodb_virtual_debug

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            View 4 older comments
            marko Marko Mäkelä added a comment -

            The n_fields assignment was moved in MDEV-22456 from dict_index_t::detach_columns() to ha_innobase_inplace_ctx::clear_added_indexes(), but it should not affect this failure.

            It may feel strange that purge is operating on an uncommitted secondary index, but it must be so by design, because concurrent DML must update a completely created secondary index as if the change had already been committed. Only duplicate key errors must be reported by the not-yet-committed DDL transaction, leading to a rollback of the operation.

            In the case of this failure, no concurrent DML is involved, though. The index is created by the following at event 119954:

            ALTER TABLE t3 ADD UNIQUE INDEX `uidx1` ( col1, col_int ), ALGORITHM = DEFAULT, LOCK = EXCLUSIVE;

            At event 120108 we cleared the n_fields from 3 to 0, because we flagged an error (most likely, a unique key violation). The assignment is only needed for avoiding a duplicated execution of dict_col_t::detach() on indexed virtual columns. It was originally added in MDEV-16376.

            I will try to find a better solution, which could involve resetting dict_field_t::col to NULL in dict_index_t::detach_columns().

            marko Marko Mäkelä added a comment - The n_fields assignment was moved in MDEV-22456 from dict_index_t::detach_columns() to ha_innobase_inplace_ctx::clear_added_indexes() , but it should not affect this failure. It may feel strange that purge is operating on an uncommitted secondary index, but it must be so by design, because concurrent DML must update a completely created secondary index as if the change had already been committed. Only duplicate key errors must be reported by the not-yet-committed DDL transaction, leading to a rollback of the operation. In the case of this failure, no concurrent DML is involved, though. The index is created by the following at event 119954: ALTER TABLE t3 ADD UNIQUE INDEX `uidx1` ( col1, col_int ), ALGORITHM = DEFAULT , LOCK = EXCLUSIVE; At event 120108 we cleared the n_fields from 3 to 0, because we flagged an error (most likely, a unique key violation). The assignment is only needed for avoiding a duplicated execution of dict_col_t::detach() on indexed virtual columns. It was originally added in MDEV-16376 . I will try to find a better solution, which could involve resetting dict_field_t::col to NULL in dict_index_t::detach_columns() .
            marko Marko Mäkelä added a comment -

            mleich, please test bb-10.2-marko with CREATE UNIQUE INDEX on virtual columns (expecting duplicate key errors) and avoiding not only table rebuilds, but also add/drop virtual column. The fix passed my local ASAN test.

            marko Marko Mäkelä added a comment - mleich , please test bb-10.2-marko with CREATE UNIQUE INDEX on virtual columns (expecting duplicate key errors) and avoiding not only table rebuilds, but also add/drop virtual column. The fix passed my local ASAN test.
            mleich Matthias Leich added a comment - 

            commit 4c0f3b3be76343c155b00feff3309d91e1e11441 origin/bb-10.2-marko
            		 
            I made ~ 4000 runs of the RQG test the rr trace above belongs to. No replay of the assert.
            		 
            Two other asserts which showed up were already known and observed months ago.
            		 
            Also the battery for broad range coverage did not show unknown/new problems.

            mleich Matthias Leich added a comment - commit 4c0f3b3be76343c155b00feff3309d91e1e11441 origin/bb-10.2-marko I made ~ 4000 runs of the RQG test the rr trace above belongs to. No replay of the assert. Two other asserts which showed up were already known and observed months ago. Also the battery for broad range coverage did not show unknown/new problems.
            marko Marko Mäkelä added a comment -

            This bug is not limited to virtual columns, and it is not an innocent test failure. It could affect any user of ALTER TABLE…ADD [UNIQUE] INDEX or CREATE [UNIQUE] INDEX when the operation is rolled back due to MDL upgrade timeout or duplicate key error.

            marko Marko Mäkelä added a comment - This bug is not limited to virtual columns, and it is not an innocent test failure. It could affect any user of ALTER TABLE…ADD [UNIQUE] INDEX or CREATE [UNIQUE] INDEX when the operation is rolled back due to MDL upgrade timeout or duplicate key error.
            marko Marko Mäkelä added a comment -

            I got yet another crash on INSERT after a failed ADD UNIQUE INDEX on a virtual column, when merging this to 10.5. So, I pushed a follow-up fix. For some reason, I was unable to reproduce that crash on 10.4. The crash occurred every now and then for the following snippet of gcol.innodb_virtual_basic:

            # Bug 21941320 - GCOLS: FAILING ASSERTION: N_IDX > 0
            		 
            create table t(a int) engine=innodb;
            		 
            insert into t set a=1;
            		 
            alter table t add column c int generated always as (1) virtual;
            		 
            insert into t set a=2;
            		 
             
            		 
            # Following will cause create index fail, we need to make sure the column
            		 
            # ord_part is reset
            		 
            --error ER_DUP_ENTRY
            		 
            alter table t add unique index(c);
            		 
            insert into t set a=1;
            		 
            drop table t;

            marko Marko Mäkelä added a comment - I got yet another crash on INSERT after a failed ADD UNIQUE INDEX on a virtual column, when merging this to 10.5. So, I pushed a follow-up fix . For some reason, I was unable to reproduce that crash on 10.4. The crash occurred every now and then for the following snippet of gcol.innodb_virtual_basic : # Bug 21941320 - GCOLS: FAILING ASSERTION: N_IDX > 0 create table t(a int ) engine=innodb; insert into t set a=1; alter table t add column c int generated always as (1) virtual; insert into t set a=2;   # Following will cause create index fail, we need to make sure the column # ord_part is reset --error ER_DUP_ENTRY alter table t add unique index (c); insert into t set a=1; drop table t;

            People

              marko Marko Mäkelä
              elenst Elena Stepanova
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.