Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-17958

Make bug-endian innodb_checksum_algorithm=crc32 optional

    XMLWordPrintable

    Details

      Description

      In MySQL 5.7, it was noticed that files are not portable between big-endian and little-endian systems (such as SPARC and x86), because the original implementation of innodb_checksum_algorithm=crc32 was not byte order agnostic.

      A byte order agnostic implementation of innodb_checksum_algorithm=crc32 was only added to MySQL 5.7, not backported to 5.6. Consequently, MariaDB Server versions 10.0 and 10.1 only contain the CRC-32C implementation that works incorrectly on big-endian architectures, and MariaDB Server 10.2.2 got the byte-order agnostic CRC-32C implementation from MySQL 5.7.

      MySQL 5.7 introduced a "legacy crc32" variant that is functionally equivalent to the big-endian version of the original crc32 implementation. Thanks to this variant, old data files can be transferred from big-endian systems to newer versions.

      Introducing new variants of checksum algorithms (without introducing new names for them) generally is a bad idea, because each checksum algorithm is like a lottery ticket. The more algorithms you try, the more likely it will be for the checksum to match on a corrupted page.

      So, essentially MySQL 5.7 weakened innodb_checksum_algorithm=crc32, and MariaDB 10.2.2 inherited this weakening.

      We elect to remove the bug-compatible variant of innodb_checksum_algorithm=crc32 as follows:

      1. We assume that most users are on little-endian.
      2. Let us make the bug-compatible variant only present on big-endian systems (#ifdef WORDS_BIGENDIAN).
      3. If someone is upgrading from MariaDB 10.0 or 10.1 or MySQL 5.6 to MariaDB 10.2 or later, they will stay on the same architecture.
      4. Completely remove the bug-compatible variant from MariaDB 10.4.
      5. If someone is switching from big-endian to little-endian, they can do it with logical dump, or they can use innochecksum to recompute the checksums.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                marko Marko Mäkelä
                Reporter:
                thiru Thirunarayanan Balathandayuthapani
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: