Marko Mäkelä added a comment - 2020-01-21 17:17 One explanation why the test crashes so rarely could be ‘stale’ pages left in the buffer pool after DISCARD TABLESPACE by MDEV-16283 .

Marko Mäkelä added a comment - 2020-03-20 10:20 I am taking over this, because the test innodb_zip.wl5522_debug_zip fails on 10.5 kvm-asan due to this every time. I was not able to repeat the failure on a clone of the VM image, but I was able to repeat a failure of innodb.innodb-wl5522-debug locally again.

Marko Mäkelä added a comment - 2020-03-20 11:10 It looks like ALTER TABLE…IMPORT TABLESPACE was always broken in this way. PageConverter::update_index_page() always failed to update all 3 copies of tablespace identifier in the index root pages. InnoDB generally ignored the wasted 4+4 bytes (which are not wasted only in B-tree root pages, but in each B-tree page header!) in BTR_SEG_TOP and BTR_SEG_LEAF . I was able to rather reliably repeat the error with the following test case: --source include/have_innodb.inc --source include/default_charset.inc --source include/have_sequence.inc   let MYSQLD_DATADIR =` SELECT @@datadir`;   # # Create a large table with delete marked records, disable purge during # the update so that we can test the IMPORT purge code. # CREATE TABLE t1 ( c1 SERIAL, c2 BIGINT , c3 VARCHAR (2048), c4 VARCHAR (2048), INDEX idx1(c2), INDEX idx2(c3(512)), INDEX idx3(c4(512))) Engine=InnoDB;   # Stop purge so that it doesn 't remove the delete marked entries. connect (purge_control,localhost,root); START TRANSACTION WITH CONSISTENT SNAPSHOT; connection default;   INSERT INTO t1 SELECT 1 + seq, 1 + (seq MOD 4), REPEAT(SUBSTR(' abcd ', 1 + (seq MOD 4), 1), 2048), REPEAT(SUBSTR(' abcd', 1 + (seq MOD 4), 1), 2048) FROM seq_0_to_127;   --disable_query_log UPDATE t1 SET c2 = c2 + c1; UPDATE t1 SET c2 = c2 + c1; UPDATE t1 SET c2 = c2 + c1; UPDATE t1 SET c2 = c2 + c1; UPDATE t1 SET c2 = c2 + c1; UPDATE t1 SET c2 = c2 + c1; UPDATE t1 SET c2 = c2 + c1; UPDATE t1 SET c2 = c2 + c1; UPDATE t1 SET c2 = c2 + c1; UPDATE t1 SET c2 = c2 + c1; UPDATE t1 SET c2 = c2 + c1; UPDATE t1 SET c2 = c2 + c1; UPDATE t1 SET c2 = c2 + c1; UPDATE t1 SET c3 = REPEAT( "c2" , 1024); UPDATE t1 SET c4 = REPEAT( "c4" , 1024); --enable_query_log   FLUSH TABLES t1 FOR EXPORT;   perl; do "$ENV{MTR_SUITE_DIR}/include/innodb-util.pl" ; ib_backup_tablespaces( "test" , "t1" ); EOF   UNLOCK TABLES;   # Enable normal operation connection purge_control; COMMIT ; disconnect purge_control; connection default ;   DROP TABLE t1;   CREATE TABLE t1 ( c1 SERIAL, c2 BIGINT , c3 VARCHAR (2048), c4 VARCHAR (2048), INDEX idx1(c2), INDEX idx2(c3(512)), INDEX idx3(c4(512))) Engine=InnoDB;   ALTER TABLE t1 DISCARD TABLESPACE;   perl; do "$ENV{MTR_SUITE_DIR}/include/innodb-util.pl" ; ib_restore_tablespaces( "test" , "t1" ); EOF   ALTER TABLE t1 IMPORT TABLESPACE; DROP TABLE t1; It would typically fail on the first or second round. It is unclear why this started to fail just now. Maybe MDEV-12353 was changing the I/O patterns enough. The consistency check was originally added along with the function btr_root_block_get() probably by me in Oracle’s InnoDB Plugin for MySQL 5.1 . With the following fix, the crash goes away: diff --git a/storage/innobase/row/row0import.cc b/storage/innobase/row/row0import.cc index 494a8b3b729..3c5e27f5b27 100644 --- a/storage/innobase/row/row0import.cc +++ b/storage/innobase/row/row0import.cc @@ -1913,6 +1913,23 @@ PageConverter::update_index_page( return(DB_SUCCESS); } + if (m_index && block->page.id.page_no() == m_index->m_page_no) { + byte *b = FIL_PAGE_DATA + PAGE_BTR_SEG_LEAF + FSEG_HDR_SPACE + + page; + mach_write_to_4(b, block->page.id.space()); + + memcpy(FIL_PAGE_DATA + PAGE_BTR_SEG_TOP + FSEG_HDR_SPACE + + page, b, 4); + if (UNIV_LIKELY_NULL(block->page.zip.data)) { + memcpy(&block->page.zip.data[FIL_PAGE_DATA + + PAGE_BTR_SEG_TOP + + FSEG_HDR_SPACE], b, 4); + memcpy(&block->page.zip.data[FIL_PAGE_DATA + + PAGE_BTR_SEG_LEAF + + FSEG_HDR_SPACE], b, 4); + } + } + #ifdef UNIV_ZIP_DEBUG ut_a(!block->page.zip.data || page_zip_validate(&block->page.zip, page, m_index->m_srv_index));

Marko Mäkelä added a comment - 2020-03-20 11:29 I pushed the fix to 10.5, to get a clean kvm-asan for the 10.5.2 release, without having to wait for several days for a merge from 10.2. While chasing this down, I worked on cleaning up the IMPORT tests. I plan to push that cleanup along with the fix to the 10.2 branch. I will close the bug once that is done.

Marko Mäkelä added a comment - 2020-03-20 14:46 I spent some additional effort to clean up the import/export tests , hoping to make them run slightly faster.