[MDEV-25361] innochecksum must not report errors for freed pages Created: 2021-01-07  Updated: 2023-07-10  Resolved: 2021-07-26

Status: Closed
Project: MariaDB Server
Component/s: Scripts & Clients
Affects Version/s: 10.2, 10.3, 10.4, 10.5, 10.6
Fix Version/s: 10.2.40, 10.3.31, 10.4.21, 10.5.12, 10.6.4

Type: Bug Priority: Critical
Reporter: Chris Calender (Inactive) Assignee: Eugene Kosov (Inactive)
Resolution: Fixed Votes: 1
Labels: corruption

Issue Links:
Problem/Incident
causes MDEV-31641 innochecksum dies with Floating point... Closed
Relates
relates to MDEV-21109 Table corruption not detected with CH... Closed

 Description   

CHECK TABLE needs to be able to detect at least the same corruption as innochecksum. This is necessary because the server must be stopped for innochecksum whereas CHECK TABLE does not.

This is related to MDEV-21109, but distinct in that MDEV-21109 is specific to mariabackup (at least per the "Component" field whereas this was a feature request to enhance CHECK TABLE.

Edit: Marko Mäkelä thinks that it is more feasible to improve innochecksum so that it will ignore data pages that have been marked freed in the tablespace files.

The innochecksum tool is mostly operating on single pages only and can only find pages with invalid checksum. CHECK TABLE can find more, such as broken links between B+ tree pages, or some inconsistency between B+ tree indexes.

Freed pages should never be accessed by the server, be it by CHECK TABLE or elsewhere. We do not want CHECK TABLE to load every page of the data file into the buffer pool, because that could make CHECK TABLE extremely slow for tables that are stored in the InnoDB system tablespace. Furthermore, loading freed pages to the buffer pool could cause MariaDB tablespace encryption to mark corrupted pages as dirty. We should simply tolerate corrupted pages that are freed.

As a further enhancement to innochecksum, we could implement an option that would zero out pages that have been marked as freed (similar to what MariaDB 10.5 would do if innodb_immediate_scrub_data_uncompressed=ON). That would also make data files compress better.

Note: innochecksum already includes a call to xdes_is_free(), but it is currently only invoked for the -S and -D options that output data on page types.



 Comments   
Comment by Chris Calender (Inactive) [ 2021-05-20 ]

This needs to be non-blocking.

Comment by Marko Mäkelä [ 2021-06-28 ]

I think that we must identify the allocation bitmap pages based on the page number and page size.

Comment by Eugene Kosov (Inactive) [ 2021-06-28 ]

Page number is now used to find xdes bitmap page.

But I left the condition to check for the first page because innochecksum requires it anyways.

Comment by Eugene Kosov (Inactive) [ 2021-06-28 ]

The code is in https://github.com/MariaDB/server/tree/bb-10.2-MDEV-25361-innochecksum

Comment by Marko Mäkelä [ 2021-06-29 ]

The first page check appears to be broken in later versions of innochecksum, possibly due to innodb_checksum_algorithm=full_crc32 (MDEV-12026). In 10.2, the following works fine without your change, which insists that the first page have a specially initialized FIL_PAGE_TYPE. Such FIL_PAGE_TYPE value is not even guaranteed if the data file had been created by an old enough version of MySQL or MariaDB.

dd bs=16384 skip=3 count=1 if=var/install.db/ibdata1 of=page3.bin
../extra/innochecksum page3.bin

Comment by Eugene Kosov (Inactive) [ 2021-07-02 ]

You're right. I've removed the condition completely. https://github.com/MariaDB/server/commits/bb-10.2-MDEV-25361-innochecksum

Comment by Marko Mäkelä [ 2021-07-20 ]

Thank you, the fix looks correct to me now.

Generated at Thu Feb 08 09:37:07 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.