[MDEV-32150] InnoDB reports corruption on 32-bit platforms with ibd files sizes > 4GB Created: 2023-09-12 Updated: 2023-09-12 Resolved: 2023-09-12 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Storage Engine - InnoDB |
| Affects Version/s: | 10.5.22, 10.11.5, 11.2.1 |
| Fix Version/s: | 10.5.23, 10.6.16, 10.10.7, 10.11.6, 11.0.4, 11.1.3, 11.2.2 |
| Type: | Bug | Priority: | Major |
| Reporter: | Danny Wood | Assignee: | Marko Mäkelä |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | regression | ||
| Environment: |
Linux 32-bit, reproducible on Debian 12 armhf and x86 releases. |
||
| Attachments: |
fix-innodb-32-bit-offset-truncation.patch
|
||||||||
| Issue Links: |
|
||||||||
| Description |
|
Mariadb reports a space/page mismatch and index corruption on 32-bit platforms when loading large InnoDB files. This started in the 10.5 series and is reproducible on each version I have tested since that series. I tracked the issue down to a 32-bit truncation when the file offset is calculated in storage/innobase/buf/buf0rea.cc. Casting this to os_offset_t solves the issue (patch attached). On 64-bit platforms this calculation would naturally occur in a 64-bit register so it is not reproducible there. |
| Comments |
| Comment by Marko Mäkelä [ 2023-09-12 ] |
|
Thank you for the report. We had something similar in |
| Comment by Danny Wood [ 2023-09-12 ] |
|
Thanks @Marko, I did have a thorough look through the bug database before I reported this. The issue I have found specifically affects 32-bit builds as the compiler will use 32-bit registers in the offset calculation as page_no() returns a 32-bit uint (it is 32-bit in the InnoDB file structure AFAIK), casting this to os_offset_t causes the compiler to do 64-bit maths as os_offset_t is typedef uint64_t. I first experienced this on my embedded linux platform (armhf) running mariadb 10.11.5 but was able to reproduce this on Debian armhf and x86 stock installs and the same database. With my patch my embedded platform loads my large database. I have tested the same database on Debian 12 64-bit (mariadb 10.11.3) and it works without issue. Do you want me to create an example database/table to test against? |
| Comment by Marko Mäkelä [ 2023-09-12 ] |
|
Luckily, this was the only call to fil_space_t::io() that would fail to use 64-bit arithmetics. |
| Comment by Danny Wood [ 2023-09-12 ] |
|
That's good! Thanks for the quick response on this. |
| Comment by Marko Mäkelä [ 2023-09-12 ] |
|
danwood76, sorry, I did not notice your update before closing this. I think that we’re good. In my patch, I used a C++11 style widening cast. The tablespace ID and page number are stored as 32-bit in each InnoDB data page; that is why they are being returned as 32-bit by the access methods. Also, the commonly used ulint (nowadays a typedef alias of size_t) would be 32 bits on 32-bit systems. |
| Comment by Danny Wood [ 2023-09-12 ] |
|
Hi @Marko, no problem. Thanks for the explanation. |