When I merged
MDEV-13328 from 10.1 to 10.2, the innodb.innodb test suddenly started to fail.
The culprit seemed to be a correct-looking conflict resolution in FlushObserver::flush(). As part of the test, innodb.innodb is executing the following:
This includes the following:
The duplicate key error triggers the following code at the end of row_merge_build_indexes():
This in turn causes a choice of a dangerous parameter:
The danger here is that m_space_id=0, the system tablespace. This is potentially discarding other writes to the InnoDB system tablespace, potentially corrupting the whole instance.
We must use the equivalent of BUF_REMOVE_FLUSH_WRITE for the system tablespace (and in MySQL 5.7, for any table that resides in a persistent shared tablespace). Failure to do so caused all sorts of trouble when running innodb.innodb after the merge, especially when using --innodb-buffer-pool-size=5m (the minimum).
Apparently the corruption was caused by the following sequence of events:
- Some pages in the system tablespace are modified.
- The ADD UNIQUE INDEX operation runs and fails, and finally removes system tablespace pages from the buf_pool->flush_list without writing to the file (see above).
- Some affected not-written-back pages are evicted from the buffer pool.
- A page is eventually needed and read back to the buffer pool, with too old contents, or with all-zero contents.
Various assertions failed due to a supposedly-initialized page being all-zero.
Note: For a failed ALTER TABLE in a tablename.ibd file, it is perfectly OK to discard the entries from the flush_list, and to subsequently mark the pages as freed, or to delete the file (if it was a table-rebuilding ALTER).