innodb_fast_shutdown=0 might corrupt the system tablespace on 32-bit systems

In MDEV-34529, a condition was added that is incorrect on 32-bit systems because a 64-bit value would be assigned to a 32-bit variable. The following patch highlights the problem and shows a possible fix:

diff --git a/storage/innobase/fsp/fsp0fsp.cc b/storage/innobase/fsp/fsp0fsp.cc

index 05de5823f25..66a01ffd69d 100644

--- a/storage/innobase/fsp/fsp0fsp.cc

+++ b/storage/innobase/fsp/fsp0fsp.cc

@@ -3645,14 +3645,15 @@ struct inode_info : private std::unordered_set<uint64_t>

       {

         const fseg_inode_t *inode=

           fsp_seg_inode_page_get_nth_inode(block->page.frame, i);

-        ulint seg_id= mach_read_from_8(FSEG_ID + inode);

         /* Consider TRX_SYS_FSEG_HEADER as used segment.

         While reinitializing the undo tablespace, InnoDB

         fail to reset the value of TRX_SYS_FSEG_HEADER

         in TRX_SYS page. so InnoDB shouldn't consider

         this segment as unused one */

-        if (seg_id == 0 || seg_id == 2)

+        switch (mach_read_from_8(FSEG_ID + inode)) {

+        case 0: case 2:

           continue;

+        }

 	uint16_t offset= uint16_t(inode - block->page.frame);

         if (offset < FIL_PAGE_DATA ||

             offset >= block->physical_size() - FIL_PAGE_DATA_END)

The problem is that the ulint type alias of size_t is insufficient for holding the 64-bit quantity.

This code can only be executed when the server is being shut down with innodb_fast_shutdown=0 or when the :autoshrink attribute is set in innodb_data_file_path, which specifies the names and sizes of the InnoDB system tablespace files.

As far as I can tell, the impact is that any file segment whose least significant 32 bits are 0 or 2 could be corrupted. This could include tables that are stored in the InnoDB system tablespace. Years ago, innodb_file_per_table=0 was the default. It could be possible that a rather recently added system table such as SYS_VIRTUAL could be affected when an old InnoDB installation already had enough many tables or indexes created (and possibly dropped) in the system tablespace.