Details
-
Bug
-
Status: Closed (View Workflow)
-
Blocker
-
Resolution: Fixed
-
10.6.12, 10.6.13, 10.6.14, 10.7(EOL), 10.8(EOL), 10.9(EOL), 10.10(EOL), 10.11, 11.0(EOL), 11.1(EOL), 11.2(EOL)
Description
This was reproduced while trying to reproduce an older issue MDEV-30531.
Some InnoDB B-tree cursor refactoring in MDEV-30400 turns out to be unsafe, resulting in InnoDB tables being flagged as corrupted. This occurs also on PRIMARY KEY indexes (clustered indexes), not only on secondary index pages.
The root cause seems to be that some operations are accessing the buffer page frame contents while only holding a buffer-fix on the page, not a page latch. It could be the case that the page is being read into the buffer pool, or it is being decrypted or decompressed. In some core dumps of such failures (with additional instrumentation to essentially revert MDEV-13542), the corruption condition would no longer hold.
Attachments
Issue Links
- blocks
-
-
- Closed
-
- is caused by
-
-
- Closed
-
- relates to
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Open
-
-
-
- Closed
-
Activity
This bug affects at least long term releases versions 10.6.12 - 10.6.14 and 10.11.2-10.11.4
Potentially it could also affect 10.6.6-10.6.11.
Anyone using a short term release should ugrade to the next long term release or to the latest one in their serie.
Two runs of the RQG test battery on a RelWithDebInfo build and one run on a debug build of
origin/bb-10.6-primary-corruption b102872ad50cce5959ad95369740766d14e9e48c 2023-07-25T11:40:58+03:00
performed well. No new or unknown problems.
There could be an even older culprit to this than MDEV-30400. In MDEV-27058, I removed the function buf_wait_for_read(), which would keep acquiring and releasing a page latch as long as the page is read-fixed. This loop would probably prevent many of these issues when a page is only being buffer-fixed.
I do not think that it ever is a good idea to use buffer-fixing for the first-time lookup of a data page in a mini-transaction. If the page was not in the buffer pool and had to be loaded into it, the buffer-fixing could gain access to the page before the read request was completed and the page checksum was validated. Before MDEV-13542, this was not that much of an issue; we would crash on corruption anyway.
What my fix aims to do is to acquire proper page latches upfront. To avoid deadlocks when acquiring page latches in the wrong order (not from left to right), we can safely release a page latch for a short while while waiting for the left sibling page latch. A buffer-fix will prevent the current block from being evicted from the buffer pool.
This bug had been reproduced while trying to reproduce another issue
MDEV-30531. I am quoting the error log from this comment:2023-07-15 15:41:26 0 [Note] /test/MD220623-mariadb-11.1.2-linux-x86_64-opt/bin/mariadbd: ready for connections.Version: '11.1.2-MariaDB' socket: '/test/MD220623-mariadb-11.1.2-linux-x86_64-opt/socket.sock' port: 12801 MariaDB Server2023-07-15 15:41:38 1509 [Note] InnoDB: Number of transaction pools: 22023-07-15 16:19:46 72616 [ERROR] InnoDB: We detected index corruption in an InnoDB type table. You have to dump + drop + reimport the table or, in a case of widespread corruption, dump all InnoDB tables and recreate the whole tablespace. If the mariadbd server crashes after the startup or when you dump the tables. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.2023-07-15 16:19:46 72616 [ERROR] mariadbd: Index for table 't2' is corrupt; try to repair itThis bug was a race condition that would allow a being-read page to be accessed before it had been fully read or uncompressed. As a result, the table may be claimed to be corrupted, even though it is not.