[MDEV-22456] Dropping the adaptive hash index may cause DDL to lock up InnoDB Created: 2020-05-04  Updated: 2023-12-28  Resolved: 2020-05-15

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 5.5, 10.0, 10.1, 10.2, 10.3, 10.4, 10.5
Fix Version/s: 10.5.4, 10.2.33, 10.3.24, 10.4.14

Type: Bug Priority: Major
Reporter: Marko Mäkelä Assignee: Marko Mäkelä
Resolution: Fixed Votes: 1
Labels: performance

Issue Links:
Blocks
is blocked by MDEV-22523 index->rtr_ssn.mutex is wasting memory Closed
Problem/Incident
causes MDEV-22611 Assertion btr_search_enabled failed d... Closed
causes MDEV-22790 Race between btr_page_mtr_lock() drop... Closed
causes MDEV-23233 Race condition for btr_search_drop_pa... Closed
causes MDEV-23332 Assertion failure in file innobase/bt... Closed
causes MDEV-23370 innodb_fts.innodb_fts_misc failed in ... Closed
causes MDEV-23452 Assertion `buf_page_get_io_fix(bpage)... Closed
causes MDEV-25527 ERROR: AddressSanitizer: attempting d... Closed
causes MDEV-26383 SUMMARY: AddressSanitizer: heap-use-a... Closed
causes MDEV-27700 SUMMARY: AddressSanitizer: heap-use-a... Closed
causes MDEV-29384 Hangs caused by innodb_adaptive_hash_... Closed
Relates
relates to MDEV-21936 Assertion `!btr_search_own_any(RW_LOC... Closed
relates to MDEV-23223 If you cancel CTAS Query on MariaDB S... Closed
relates to MDEV-23556 Truncate hangs InnoDB when innodb_saf... Closed
relates to MDEV-24188 Hang in buf_page_create() after reusi... Closed
relates to MDEV-24505 innodb.innodb-ucs2 test failed, asse... Closed
relates to MDEV-24863 InnoDB: Failing assertion: index == c... Closed
relates to MDEV-8069 DROP or rebuild of a large table may ... Closed
relates to MDEV-16283 ALTER TABLE...DISCARD TABLESPACE stil... Closed
relates to MDEV-26599 Assertion `!table->id' failed in bt... Open

 Description   

If the InnoDB buffer pool contains many pages for a table or index that is being dropped or rebuilt, and if many of such pages are pointed to by the adaptive hash index, dropping the adaptive hash index may consume a lot of time.

The time-consuming operation of dropping the adaptive hash index entries is being executed while the InnoDB data dictionary cache dict_sys is exclusively locked.

(Data file deletion also takes place while the dictionary cache is locked. That is being covered by MDEV-8069.)

It is not actually necessary to drop all entries from the adaptive hash index when a table or index is being dropped or rebuilt. We can let the LRU replacement policy of the buffer pool take care of this gradually. For this to work, we must detach the dict_table_t and dict_index_t objects from the main dict_sys cache, and once the last adaptive hash index entry for the detached table is removed (when the garbage page is evicted from the buffer pool) we can free the dict_table_t and dict_index_t object.

Related to this, in MDEV-16283, we made ALTER TABLE...DISCARD TABLESPACE skip both the buffer pool eviction and the drop of the adaptive hash index. We shifted the burden to ALTER TABLE...IMPORT TABLESPACE or DROP TABLE. We can remove the eviction from DROP TABLE. We must retain the eviction in the ALTER TABLE...IMPORT TABLESPACE code path, so that in case the discarded table is being re-imported with the same tablespace identifier, the fresh data from the imported tablespace will replace any stale pages in the buffer pool.



 Comments   
Comment by Marko Mäkelä [ 2020-05-11 ]

For deferred dropping of adaptive hash index entries pointing to indexed virtual columns, we must clone the dict_table_t object.

Comment by Marko Mäkelä [ 2020-05-14 ]

Please review bb-10.2-marko. My local tests were fine, but mleich reported some assertion failures for an earlier revision.

Comment by Matthias Leich [ 2020-05-15 ]

bb-10.2-marko commit d859d365f736185018febaeb58530f3e4d0855be
containing the code for MDEV-22456 performed well in RQG testing.
The earlier MDEV-22456 code related assertion failures did no more show up.

Generated at Thu Feb 08 09:14:52 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.