Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-17483

Insert on delete-marked record can wrongly inherit old values for instantly added column

Details

    Description

      The following test attempts to free the value of column b twice, resulting in a debug assertion failure:

      --source include/have_innodb.inc
      SET @saved_frequency = @@GLOBAL.innodb_purge_rseg_truncate_frequency;
      SET GLOBAL innodb_purge_rseg_truncate_frequency = 1;
       
      CREATE TABLE t1 (a INT PRIMARY KEY) ENGINE = InnoDB  ;
      INSERT INTO t1 SET a = 1;
      ALTER TABLE t1 ADD COLUMN b TEXT;
      BEGIN;
      UPDATE t1 SET b = REPEAT('1', @@innodb_page_size / 2);
      UPDATE t1 SET a = 2;
      INSERT INTO t1 SET a = 1;
      DELETE FROM t1;
      COMMIT;
      --source include/wait_all_purged.inc
      DROP TABLE t1;
      SET GLOBAL innodb_purge_rseg_truncate_frequency = @saved_frequency;
      

      10.3 7e869a2767c849d266ffbf1def9b93ffae7cd428

      mysqld: /mariadb/10.3/storage/innobase/buf/buf0buf.cc:4948: buf_block_t* buf_page_get_gen(const page_id_t&, const page_size_t&, ulint, buf_block_t*, ulint, const char*, unsigned int, mtr_t*, dberr_t*): Assertion `mode == 16 || mode == 12 || !fix_block->page.file_page_was_freed' failed.
      …
      buf/buf0buf.cc:4953(buf_page_get_gen(page_id_t const&, page_size_t const&, unsigned long, buf_block_t*, unsigned long, char const*, unsigned int, mtr_t*, dberr_t*))[0x55fedca5a27c]
      btr/btr0cur.cc:7848(btr_free_externally_stored_field(dict_index_t*, unsigned char*, unsigned char const*, unsigned long const*, page_zip_des_t*, unsigned long, bool, mtr_t*))[0x55fedca5a7dd]
      btr/btr0cur.cc:7950(btr_rec_free_externally_stored_fields(dict_index_t*, unsigned char*, unsigned long const*, page_zip_des_t*, bool, mtr_t*))[0x55fedca54d2e]
      btr/btr0cur.cc:5651(btr_cur_pessimistic_delete(dberr_t*, unsigned long, btr_cur_t*, unsigned long, bool, mtr_t*))[0x55fedc96fd8a]
      row/row0purge.cc:178(row_purge_remove_clust_if_poss_low(purge_node_t*, unsigned long))[0x55fedc96feb8]
      row/row0purge.cc:223(row_purge_remove_clust_if_poss(purge_node_t*))[0x55fedc971766]
      row/row0purge.cc:806(row_purge_del_mark(purge_node_t*))[0x55fedc972c55]
      row/row0purge.cc:1187(row_purge_record_func(purge_node_t*, unsigned char*, que_thr_t const*, bool))[0x55fedc972f6d]
      

      Attachments

        Issue Links

          Activity

            I believe that this is because we are failing to set the ‘disown’ flag (confusingly named BTR_EXTERN_OWNER_FLAG) on column b in the old record a=1 when the PRIMARY KEY is updated, and the ownership of the off-page column b should transfer to the new record a=2.
            Purge would first free the BLOB for a=2, then a=1, with the following stack trace:

            #1  0x00005555563417dd in btr_rec_free_externally_stored_fields (
                index=0x7fff9c020058, rec=0x7ffff45b00dd "\200", offsets=0x7fffd0015908, 
                page_zip=0x0, rollback=false, mtr=0x7fffe0c0d180)
                at /mariadb/10.3/storage/innobase/btr/btr0cur.cc:7950
            #2  0x000055555633bd2e in btr_cur_pessimistic_delete (err=0x7fffe0c0ce3c, 
                has_reserved_extents=0, cursor=0x555557ce4cb0, flags=0, rollback=false, 
                mtr=0x7fffe0c0d180) at /mariadb/10.3/storage/innobase/btr/btr0cur.cc:5643
            #3  0x0000555556256d8a in row_purge_remove_clust_if_poss_low (
                node=0x555557ce4c18, mode=65569)
                at /mariadb/10.3/storage/innobase/row/row0purge.cc:174
            #4  0x0000555556256eb8 in row_purge_remove_clust_if_poss (node=0x555557ce4c18)
                at /mariadb/10.3/storage/innobase/row/row0purge.cc:223
            #5  0x0000555556258766 in row_purge_del_mark (node=0x555557ce4c18)
                at /mariadb/10.3/storage/innobase/row/row0purge.cc:806
            #6  0x0000555556259c55 in row_purge_record_func (node=0x555557ce4c18, 
                undo_rec=0x7fffd00136b8 "", thr=0x555557ce4770, updated_extern=false)
                at /mariadb/10.3/storage/innobase/row/row0purge.cc:1187
            

            The BLOB pointers of the two delete-marked records are identical, except that on a=2 the BTR_EXTERN_INHERITED_FLAG is set. If the pointer in the a=1 record carried the BTR_EXTERN_OWNER_FLAG, then it should be skipped by purge.
            If the , FORCE clause is added to the ALTER TABLE statement, then there will be only one call to free the BLOB:

            #0  btr_free_externally_stored_field (index=0x7fff9c033a28, 
                field_ref=0x7ffff415043d "", rec=0x0, offsets=0x0, page_zip=0x0, i=0, 
                rollback=false, local_mtr=0x7fffe0c0d1e0)
                at /mariadb/10.3/storage/innobase/btr/btr0cur.cc:7759
            #1  0x000055555625956b in row_purge_upd_exist_or_extern_func (
                thr=0x555557ce4dc8, node=0x555557ce4e80, undo_rec=0x555557ce5978 "")
                at /mariadb/10.3/storage/innobase/row/row0purge.cc:1000
            #2  0x0000555556259dd3 in row_purge_record_func (node=0x555557ce4e80, 
                undo_rec=0x555557ce5978 "", thr=0x555557ce4dc8, updated_extern=true)
                at /mariadb/10.3/storage/innobase/row/row0purge.cc:1210
            

            In this case, the BLOB would be freed based on the pointer in the undo log record. At this point of time, the 2 delete-marked records still exist in the clustered index. The record a=1 is carrying the BTR_EXTERN_OWNER_FLAG on b, and the record a=2 is carrying BTR_EXTERN_INHERITED_FLAG.

            marko Marko Mäkelä added a comment - I believe that this is because we are failing to set the ‘disown’ flag (confusingly named BTR_EXTERN_OWNER_FLAG ) on column b in the old record a=1 when the PRIMARY KEY is updated, and the ownership of the off-page column b should transfer to the new record a=2 . Purge would first free the BLOB for a=2 , then a=1 , with the following stack trace: #1 0x00005555563417dd in btr_rec_free_externally_stored_fields ( index=0x7fff9c020058, rec=0x7ffff45b00dd "\200", offsets=0x7fffd0015908, page_zip=0x0, rollback=false, mtr=0x7fffe0c0d180) at /mariadb/10.3/storage/innobase/btr/btr0cur.cc:7950 #2 0x000055555633bd2e in btr_cur_pessimistic_delete (err=0x7fffe0c0ce3c, has_reserved_extents=0, cursor=0x555557ce4cb0, flags=0, rollback=false, mtr=0x7fffe0c0d180) at /mariadb/10.3/storage/innobase/btr/btr0cur.cc:5643 #3 0x0000555556256d8a in row_purge_remove_clust_if_poss_low ( node=0x555557ce4c18, mode=65569) at /mariadb/10.3/storage/innobase/row/row0purge.cc:174 #4 0x0000555556256eb8 in row_purge_remove_clust_if_poss (node=0x555557ce4c18) at /mariadb/10.3/storage/innobase/row/row0purge.cc:223 #5 0x0000555556258766 in row_purge_del_mark (node=0x555557ce4c18) at /mariadb/10.3/storage/innobase/row/row0purge.cc:806 #6 0x0000555556259c55 in row_purge_record_func (node=0x555557ce4c18, undo_rec=0x7fffd00136b8 "", thr=0x555557ce4770, updated_extern=false) at /mariadb/10.3/storage/innobase/row/row0purge.cc:1187 The BLOB pointers of the two delete-marked records are identical, except that on a=2 the BTR_EXTERN_INHERITED_FLAG is set. If the pointer in the a=1 record carried the BTR_EXTERN_OWNER_FLAG , then it should be skipped by purge. If the , FORCE clause is added to the ALTER TABLE statement, then there will be only one call to free the BLOB: #0 btr_free_externally_stored_field (index=0x7fff9c033a28, field_ref=0x7ffff415043d "", rec=0x0, offsets=0x0, page_zip=0x0, i=0, rollback=false, local_mtr=0x7fffe0c0d1e0) at /mariadb/10.3/storage/innobase/btr/btr0cur.cc:7759 #1 0x000055555625956b in row_purge_upd_exist_or_extern_func ( thr=0x555557ce4dc8, node=0x555557ce4e80, undo_rec=0x555557ce5978 "") at /mariadb/10.3/storage/innobase/row/row0purge.cc:1000 #2 0x0000555556259dd3 in row_purge_record_func (node=0x555557ce4e80, undo_rec=0x555557ce5978 "", thr=0x555557ce4dc8, updated_extern=true) at /mariadb/10.3/storage/innobase/row/row0purge.cc:1210 In this case, the BLOB would be freed based on the pointer in the undo log record. At this point of time, the 2 delete-marked records still exist in the clustered index. The record a=1 is carrying the BTR_EXTERN_OWNER_FLAG on b , and the record a=2 is carrying BTR_EXTERN_INHERITED_FLAG .

            When the PRIMARY KEY was updated, the BLOB pointer was correctly marked as ‘disowned’ in the delete-marked record with the old key. The problem is that a subsequent INSERT with the old key wrongly cleared the ‘disown’ flag:

            #3  0x000055555633ecd1 in btr_cur_unmark_extern_fields (page_zip=0x0, 
                rec=0x7ffff45b00af "\200", index=0x7fff9c020058, offsets=0x7fffefdb1b40, 
                mtr=0x7fffefdb1e60) at /mariadb/10.3/storage/innobase/btr/btr0cur.cc:6961
            #4  0x0000555556336bd6 in btr_cur_update_in_place (flags=0, 
                cursor=0x7fffefdb1a40, offsets=0x7fffefdb1b40, update=0x7fff9c0316e8, 
                cmpl_info=0, thr=0x7fff9c022e08, trx_id=42, mtr=0x7fffefdb1e60)
                at /mariadb/10.3/storage/innobase/btr/btr0cur.cc:4103
            #5  0x00005555563375c2 in btr_cur_optimistic_update (flags=0, 
                cursor=0x7fffefdb1a40, offsets=0x7fffefdb19e0, heap=0x7fffefdb19d8, 
                update=0x7fff9c0316e8, cmpl_info=0, thr=0x7fff9c022e08, trx_id=42, 
                mtr=0x7fffefdb1e60) at /mariadb/10.3/storage/innobase/btr/btr0cur.cc:4262
            #6  0x000055555620d931 in row_ins_clust_index_entry_by_modify (
                pcur=0x7fffefdb1a40, flags=0, mode=2, offsets=0x7fffefdb19e0, 
                offsets_heap=0x7fffefdb19d8, heap=0x7fff9c031660, entry=0x7fff9c01f228, 
                thr=0x7fff9c022e08, mtr=0x7fffefdb1e60)
                at /mariadb/10.3/storage/innobase/row/row0ins.cc:368
            

            Here is a slightly different test case that demonstrates the same problem:

            --source include/have_innodb.inc
            SET @saved_frequency = @@GLOBAL.innodb_purge_rseg_truncate_frequency;
            SET GLOBAL innodb_purge_rseg_truncate_frequency = 1;
             
            CREATE TABLE t1 (a INT PRIMARY KEY) ENGINE = InnoDB;
            # Create non-empty table so that instant ADD will be used
            INSERT INTO t1 SET a = 0;
            ALTER TABLE t1 ADD COLUMN b TEXT;
            BEGIN;
            --echo # The dummy record inserted above does not really matter.
            DELETE FROM t1;
            --echo # In the previous test, we added the BLOB by UPDATE. Here we do it by INSERT:
            INSERT INTO t1 SET a = 1, b = REPEAT('1', @@innodb_page_size / 2);
            --echo # Update the PRIMARY KEY. a=1 will be delete-marked, and will ‘disown’ b.
            UPDATE t1 SET a = 2;
            --echo # This will wrongly attach b to the record, also marking it as ‘owned’!
            INSERT INTO t1 SET a = 1;
            --echo # This should return b=NULL for a=1, but returns the BLOB for both rows!
            SELECT * FROM t1;
            DELETE FROM t1;
            COMMIT;
            --source include/wait_all_purged.inc
            DROP TABLE t1;
            SET GLOBAL innodb_purge_rseg_truncate_frequency = @saved_frequency;
            

            marko Marko Mäkelä added a comment - When the PRIMARY KEY was updated, the BLOB pointer was correctly marked as ‘disowned’ in the delete-marked record with the old key. The problem is that a subsequent INSERT with the old key wrongly cleared the ‘disown’ flag: #3 0x000055555633ecd1 in btr_cur_unmark_extern_fields (page_zip=0x0, rec=0x7ffff45b00af "\200", index=0x7fff9c020058, offsets=0x7fffefdb1b40, mtr=0x7fffefdb1e60) at /mariadb/10.3/storage/innobase/btr/btr0cur.cc:6961 #4 0x0000555556336bd6 in btr_cur_update_in_place (flags=0, cursor=0x7fffefdb1a40, offsets=0x7fffefdb1b40, update=0x7fff9c0316e8, cmpl_info=0, thr=0x7fff9c022e08, trx_id=42, mtr=0x7fffefdb1e60) at /mariadb/10.3/storage/innobase/btr/btr0cur.cc:4103 #5 0x00005555563375c2 in btr_cur_optimistic_update (flags=0, cursor=0x7fffefdb1a40, offsets=0x7fffefdb19e0, heap=0x7fffefdb19d8, update=0x7fff9c0316e8, cmpl_info=0, thr=0x7fff9c022e08, trx_id=42, mtr=0x7fffefdb1e60) at /mariadb/10.3/storage/innobase/btr/btr0cur.cc:4262 #6 0x000055555620d931 in row_ins_clust_index_entry_by_modify ( pcur=0x7fffefdb1a40, flags=0, mode=2, offsets=0x7fffefdb19e0, offsets_heap=0x7fffefdb19d8, heap=0x7fff9c031660, entry=0x7fff9c01f228, thr=0x7fff9c022e08, mtr=0x7fffefdb1e60) at /mariadb/10.3/storage/innobase/row/row0ins.cc:368 Here is a slightly different test case that demonstrates the same problem: --source include/have_innodb.inc SET @saved_frequency = @@ GLOBAL .innodb_purge_rseg_truncate_frequency; SET GLOBAL innodb_purge_rseg_truncate_frequency = 1;   CREATE TABLE t1 (a INT PRIMARY KEY ) ENGINE = InnoDB; # Create non-empty table so that instant ADD will be used INSERT INTO t1 SET a = 0; ALTER TABLE t1 ADD COLUMN b TEXT; BEGIN ; --echo # The dummy record inserted above does not really matter. DELETE FROM t1; --echo # In the previous test, we added the BLOB by UPDATE. Here we do it by INSERT: INSERT INTO t1 SET a = 1, b = REPEAT( '1' , @@innodb_page_size / 2); --echo # Update the PRIMARY KEY. a=1 will be delete-marked, and will ‘disown’ b. UPDATE t1 SET a = 2; --echo # This will wrongly attach b to the record, also marking it as ‘owned’! INSERT INTO t1 SET a = 1; --echo # This should return b=NULL for a=1, but returns the BLOB for both rows! SELECT * FROM t1; DELETE FROM t1; COMMIT ; --source include/wait_all_purged.inc DROP TABLE t1; SET GLOBAL innodb_purge_rseg_truncate_frequency = @saved_frequency;

            The problem is not limited to BLOBs. The following test will return (1,3) instead of the correct value (1,NULL):

            --source include/have_innodb.inc
            CREATE TABLE t1 (a INT PRIMARY KEY) ENGINE = InnoDB;
            INSERT INTO t1 SET a = 1;
            ALTER TABLE t1 ADD COLUMN b INT;
            UPDATE t1 SET b = 3;
            BEGIN;
            DELETE FROM t1;
            INSERT INTO t1 SET a = 1;
            COMMIT;
            SELECT * FROM t1;
            DROP TABLE t1;
            

            Without the BEGIN and COMMIT, the test could occasionally return the correct result (1,NULL) if purge processed the DELETE statement.

            marko Marko Mäkelä added a comment - The problem is not limited to BLOBs. The following test will return (1,3) instead of the correct value (1,NULL): --source include/have_innodb.inc CREATE TABLE t1 (a INT PRIMARY KEY ) ENGINE = InnoDB; INSERT INTO t1 SET a = 1; ALTER TABLE t1 ADD COLUMN b INT ; UPDATE t1 SET b = 3; BEGIN ; DELETE FROM t1; INSERT INTO t1 SET a = 1; COMMIT ; SELECT * FROM t1; DROP TABLE t1; Without the BEGIN and COMMIT , the test could occasionally return the correct result (1,NULL) if purge processed the DELETE statement.

            The problem appears to be that row_upd_build_difference_binary() is wrongly returning an empty update vector, even though instantly added columns are being replaced with absent columns (when the entry has fewer fields than rec).
            This does not seem to affect normal UPDATE: updating the wrongly inherited column values to default values seems to work as expected.

            marko Marko Mäkelä added a comment - The problem appears to be that row_upd_build_difference_binary() is wrongly returning an empty update vector, even though instantly added columns are being replaced with absent columns (when the entry has fewer fields than rec ). This does not seem to affect normal UPDATE : updating the wrongly inherited column values to default values seems to work as expected.

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.