[MDEV-25361] innochecksum must not report errors for freed pages Created: 2021-01-07 Updated: 2023-07-10 Resolved: 2021-07-26 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Scripts & Clients |
| Affects Version/s: | 10.2, 10.3, 10.4, 10.5, 10.6 |
| Fix Version/s: | 10.2.40, 10.3.31, 10.4.21, 10.5.12, 10.6.4 |
| Type: | Bug | Priority: | Critical |
| Reporter: | Chris Calender (Inactive) | Assignee: | Eugene Kosov (Inactive) |
| Resolution: | Fixed | Votes: | 1 |
| Labels: | corruption | ||
| Issue Links: |
|
||||||||||||||||
| Description |
|
CHECK TABLE needs to be able to detect at least the same corruption as innochecksum. This is necessary because the server must be stopped for innochecksum whereas CHECK TABLE does not. This is related to Edit: Marko Mäkelä thinks that it is more feasible to improve innochecksum so that it will ignore data pages that have been marked freed in the tablespace files. The innochecksum tool is mostly operating on single pages only and can only find pages with invalid checksum. CHECK TABLE can find more, such as broken links between B+ tree pages, or some inconsistency between B+ tree indexes. Freed pages should never be accessed by the server, be it by CHECK TABLE or elsewhere. We do not want CHECK TABLE to load every page of the data file into the buffer pool, because that could make CHECK TABLE extremely slow for tables that are stored in the InnoDB system tablespace. Furthermore, loading freed pages to the buffer pool could cause MariaDB tablespace encryption to mark corrupted pages as dirty. We should simply tolerate corrupted pages that are freed. As a further enhancement to innochecksum, we could implement an option that would zero out pages that have been marked as freed (similar to what MariaDB 10.5 would do if innodb_immediate_scrub_data_uncompressed=ON). That would also make data files compress better. Note: innochecksum already includes a call to xdes_is_free(), but it is currently only invoked for the -S and -D options that output data on page types. |
| Comments |
| Comment by Chris Calender (Inactive) [ 2021-05-20 ] | ||
|
This needs to be non-blocking. | ||
| Comment by Marko Mäkelä [ 2021-06-28 ] | ||
|
I think that we must identify the allocation bitmap pages based on the page number and page size. | ||
| Comment by Eugene Kosov (Inactive) [ 2021-06-28 ] | ||
|
Page number is now used to find xdes bitmap page. But I left the condition to check for the first page because innochecksum requires it anyways. | ||
| Comment by Eugene Kosov (Inactive) [ 2021-06-28 ] | ||
|
The code is in https://github.com/MariaDB/server/tree/bb-10.2-MDEV-25361-innochecksum | ||
| Comment by Marko Mäkelä [ 2021-06-29 ] | ||
|
The first page check appears to be broken in later versions of innochecksum, possibly due to innodb_checksum_algorithm=full_crc32 (
| ||
| Comment by Eugene Kosov (Inactive) [ 2021-07-02 ] | ||
|
You're right. I've removed the condition completely. https://github.com/MariaDB/server/commits/bb-10.2-MDEV-25361-innochecksum | ||
| Comment by Marko Mäkelä [ 2021-07-20 ] | ||
|
Thank you, the fix looks correct to me now. |