Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-25105

Remove innodb_checksum_algorithm values none, innodb, strict_none, strict_innodb

Details

    Description

      Historically, InnoDB supported a buggy page checksum algorithm that did not compute a checksum over the full page. Later, well before MySQL 4.1 introduced .ibd files and the innodb_file_per_table option, the algorithm was corrected and the first 4 bytes of each page were redefined to be a checksum.

      The original checksum was so slow that an option to disable page checksum was introduced for benchmarketing purposes.

      The Intel Nehalem microarchitecture introduced the SSE4.2 instruction set extension, which includes instructions for faster computation of CRC-32C. In MySQL 5.6 (and MariaDB 10.0), innodb_checksum_algorithm=crc32 was implemented to make of that. As that option was changed to be the default in MySQL 5.7, a bug was found on big-endian platforms and some work-around code was added to weaken that checksum further. MariaDB disables that work-around by default since MDEV-17958.

      Later, SIMD-accelerated CRC-32C has been implemented in MariaDB for POWER and ARM and also for IA-32/AMD64, making use of the pclmul instruction (carry-less multiplication).

      Long story short, innodb_checksum_algorithm=crc32 is faster and more secure than the original checksum, called innodb_checksum_algorithm=innodb. It should have removed any need to configure innodb_checksum_algorithm=none.

      The setting innodb_checksum_algorithm=crc32 is also the default in MariaDB Server 10.2, 10.3, 10.4. In MariaDB 10.5, MDEV-19534 made innodb_checksum_algorithm=full_crc32 the default. It is even faster and more secure.

      The default settings do allow old data files to be read, no matter if a worse algorithm had been used. (The settings innodb_checksum_algorithm=strict_crc32 or innodb_checksum_algorithm=strict_full_crc32 would only allow CRC-32C checksums, which is why they are not the default.)

      A valid reason for using innodb_checksum_algorithm=innodb could have been the ability to downgrade to MariaDB 5.5, but that version has already reached its end of life. Also, if it is really needed, data files can be converted with the innochecksum utility.

      Because there is no good reason to allow data files to be written with insecure checksums, we will reject those option values:

      • innodb_checksum_algorithm=none
      • innodb_checksum_algorithm=innodb
      • innodb_checksum_algorithm=strict_none
      • innodb_checksum_algorithm=strict_innodb

      Furthermore, the following innochecksum options will be removed:

      • --strict-check or -C (we only support crc32), and strict check is the default unless --no-check or -n is specified
      • --write or -w (we only support crc32)

      Attachments

        Issue Links

          Activity

            ltning Eirik Øverby added a comment -

            Knowing this is a bit old, but there are very good reasons to disable checksumming entirely: ZFS.

            We run a multi-hundred TB Galera cluster with thousands of write operations per second. Doing checksumming twice (ZFS plus database) is a waste, and despite being fast, the crc32 CPU instructions are a finite resource. Double-checksumming hurts our performance significantly, even on pretty high-end hardware (nvme ssd, couple dozen ~3ghz EPYC3/4 CPU cores).

            Is there any other way to disable checksumming in maria/innodb in this day and age? We're stuck on MySQL 5.7 for a number of reasons, this being one of them.
            Note that disabling checksums in ZFS is not an option since that would also lose checksumming of data that is not checksummed by mariadb.

            ltning Eirik Øverby added a comment - Knowing this is a bit old, but there are very good reasons to disable checksumming entirely: ZFS. We run a multi-hundred TB Galera cluster with thousands of write operations per second. Doing checksumming twice (ZFS plus database) is a waste, and despite being fast, the crc32 CPU instructions are a finite resource. Double-checksumming hurts our performance significantly, even on pretty high-end hardware (nvme ssd, couple dozen ~3ghz EPYC3/4 CPU cores). Is there any other way to disable checksumming in maria/innodb in this day and age? We're stuck on MySQL 5.7 for a number of reasons, this being one of them. Note that disabling checksums in ZFS is not an option since that would also lose checksumming of data that is not checksummed by mariadb.

            People

              thiru Thirunarayanan Balathandayuthapani
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.