Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-17958

Make bug-endian innodb_checksum_algorithm=crc32 optional

Details

    Description

      In MySQL 5.7, it was noticed that files are not portable between big-endian and little-endian systems (such as SPARC and x86), because the original implementation of innodb_checksum_algorithm=crc32 was not byte order agnostic.

      A byte order agnostic implementation of innodb_checksum_algorithm=crc32 was only added to MySQL 5.7, not backported to 5.6. Consequently, MariaDB Server versions 10.0 and 10.1 only contain the CRC-32C implementation that works incorrectly on big-endian architectures, and MariaDB Server 10.2.2 got the byte-order agnostic CRC-32C implementation from MySQL 5.7.

      MySQL 5.7 introduced a "legacy crc32" variant that is functionally equivalent to the big-endian version of the original crc32 implementation. Thanks to this variant, old data files can be transferred from big-endian systems to newer versions.

      Introducing new variants of checksum algorithms (without introducing new names for them) generally is a bad idea, because each checksum algorithm is like a lottery ticket. The more algorithms you try, the more likely it will be for the checksum to match on a corrupted page.

      So, essentially MySQL 5.7 weakened innodb_checksum_algorithm=crc32, and MariaDB 10.2.2 inherited this weakening.

      We elect to remove the bug-compatible variant of innodb_checksum_algorithm=crc32 as follows:

      1. We assume that most users are on little-endian.
      2. Let us make the bug-compatible variant only present on big-endian systems (#ifdef WORDS_BIGENDIAN).
      3. If someone is upgrading from MariaDB 10.0 or 10.1 or MySQL 5.6 to MariaDB 10.2 or later, they will stay on the same architecture.
      4. Completely remove the bug-compatible variant from MariaDB 10.4.
      5. If someone is switching from big-endian to little-endian, they can do it with logical dump, or they can use innochecksum to recompute the checksums.

      Attachments

        Issue Links

          Activity

            Transition Time In Source Status Execution Times
            Thirunarayanan Balathandayuthapani made transition -
            Open In Progress
            1d 4h 49m 1
            Thirunarayanan Balathandayuthapani made transition -
            In Progress In Review
            7s 1
            Marko Mäkelä made transition -
            In Review Closed
            2d 47m 1

            People

              marko Marko Mäkelä
              thiru Thirunarayanan Balathandayuthapani
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.