Historically, InnoDB supported a buggy page checksum algorithm that did not compute a checksum over the full page. Later, well before MySQL 4.1 introduced .ibd files and the innodb_file_per_table option, the algorithm was corrected and the first 4 bytes of each page were redefined to be a checksum.
The original checksum was so slow that an option to disable page checksum was introduced for benchmarketing purposes.
The Intel Nehalem microarchitecture introduced the SSE4.2 instruction set extension, which includes instructions for faster computation of CRC-32C. In MySQL 5.6 (and MariaDB 10.0), innodb_checksum_algorithm=crc32 was implemented to make of that. As that option was changed to be the default in MySQL 5.7, a bug was found on big-endian platforms and some work-around code was added to weaken that checksum further. MariaDB disables that work-around by default since MDEV-17958.
Later, SIMD-accelerated CRC-32C has been implemented in MariaDB for POWER and ARM and also for IA-32/AMD64, making use of the pclmul instruction (carry-less multiplication).
Long story short, innodb_checksum_algorithm=crc32 is faster and more secure than the original checksum, called innodb_checksum_algorithm=innodb. It should have removed any need to configure innodb_checksum_algorithm=none.
The setting innodb_checksum_algorithm=crc32 is also the default in MariaDB Server 10.2, 10.3, 10.4. In MariaDB 10.5, MDEV-19534 made innodb_checksum_algorithm=full_crc32 the default. It is even faster and more secure.
The default settings do allow old data files to be read, no matter if a worse algorithm had been used. (The settings innodb_checksum_algorithm=strict_crc32 or innodb_checksum_algorithm=strict_full_crc32 would only allow CRC-32C checksums, which is why they are not the default.)
A valid reason for using innodb_checksum_algorithm=innodb could have been the ability to downgrade to MariaDB 5.5, but that version has already reached its end of life. Also, if it is really needed, data files can be converted with the innochecksum utility.
Because there is no good reason to allow data files to be written with insecure checksums, we will reject those option values:
innodb_checksum_algorithm=none
innodb_checksum_algorithm=innodb
innodb_checksum_algorithm=strict_none
innodb_checksum_algorithm=strict_innodb
Furthermore, the following innochecksum options will be removed:
--strict-check or -C (we only support crc32), and strict check is the default unless --no-check or -n is specified
--write or -w (we only support crc32)
Attachments
Issue Links
blocks
MDEV-25106Issue deprecation warnings for innodb_checksum_algorithm values none,innodb,strict_none,strict_innodb
Closed
causes
MDEV-29619mariadb-backup option `innodb_checksum_algorithm` is documented in the wrong way
Closed
relates to
MDEV-27931A page in innodb_checksum_algorithm=innodb is wrongly claimed to be corrupted
Closed
MDEV-31473Wrong information about innodb_checksum_algorithm in the information_schema.SYSTEM_VARIABLES
Knowing this is a bit old, but there are very good reasons to disable checksumming entirely: ZFS.
We run a multi-hundred TB Galera cluster with thousands of write operations per second. Doing checksumming twice (ZFS plus database) is a waste, and despite being fast, the crc32 CPU instructions are a finite resource. Double-checksumming hurts our performance significantly, even on pretty high-end hardware (nvme ssd, couple dozen ~3ghz EPYC3/4 CPU cores).
Is there any other way to disable checksumming in maria/innodb in this day and age? We're stuck on MySQL 5.7 for a number of reasons, this being one of them.
Note that disabling checksums in ZFS is not an option since that would also lose checksumming of data that is not checksummed by mariadb.
Eirik Øverby
added a comment - Knowing this is a bit old, but there are very good reasons to disable checksumming entirely: ZFS.
We run a multi-hundred TB Galera cluster with thousands of write operations per second. Doing checksumming twice (ZFS plus database) is a waste, and despite being fast, the crc32 CPU instructions are a finite resource. Double-checksumming hurts our performance significantly, even on pretty high-end hardware (nvme ssd, couple dozen ~3ghz EPYC3/4 CPU cores).
Is there any other way to disable checksumming in maria/innodb in this day and age? We're stuck on MySQL 5.7 for a number of reasons, this being one of them.
Note that disabling checksums in ZFS is not an option since that would also lose checksumming of data that is not checksummed by mariadb.
People
Thirunarayanan Balathandayuthapani
Marko Mäkelä
Votes:
0Vote for this issue
Watchers:
4Start watching this issue
Dates
Created:
Updated:
Resolved:
Git Integration
Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.
{"report":{"fcp":1285.7999997138977,"ttfb":513.5999999046326,"pageVisibility":"visible","entityId":97781,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"7c904f50-b152-4274-9abc-75ae415e0482","navigationType":0,"readyForUser":1353.4000000953674,"redirectCount":0,"resourceLoadedEnd":1076.0999999046326,"resourceLoadedStart":518.7999997138977,"resourceTiming":[{"duration":234.90000009536743,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":518.7999997138977,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":518.7999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":753.6999998092651,"responseStart":0,"secureConnectionStart":0},{"duration":241.09999990463257,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":519.0999999046326,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":519.0999999046326,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":760.1999998092651,"responseStart":0,"secureConnectionStart":0},{"duration":305.7000002861023,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":519.2999997138977,"connectEnd":519.2999997138977,"connectStart":519.2999997138977,"domainLookupEnd":519.2999997138977,"domainLookupStart":519.2999997138977,"fetchStart":519.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":767.6999998092651,"responseEnd":825,"responseStart":782.2999997138977,"secureConnectionStart":519.2999997138977},{"duration":327.90000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":519.5,"connectEnd":519.5,"connectStart":519.5,"domainLookupEnd":519.5,"domainLookupStart":519.5,"fetchStart":519.5,"redirectEnd":0,"redirectStart":0,"requestStart":768.4000000953674,"responseEnd":847.4000000953674,"responseStart":782.9000000953674,"secureConnectionStart":519.5},{"duration":267.90000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":519.6999998092651,"connectEnd":519.6999998092651,"connectStart":519.6999998092651,"domainLookupEnd":519.6999998092651,"domainLookupStart":519.6999998092651,"fetchStart":519.6999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":770.0999999046326,"responseEnd":787.5999999046326,"responseStart":787,"secureConnectionStart":519.6999998092651},{"duration":274.8999996185303,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":519.9000000953674,"connectEnd":519.9000000953674,"connectStart":519.9000000953674,"domainLookupEnd":519.9000000953674,"domainLookupStart":519.9000000953674,"fetchStart":519.9000000953674,"redirectEnd":0,"redirectStart":0,"requestStart":771.2999997138977,"responseEnd":794.7999997138977,"responseStart":794.0999999046326,"secureConnectionStart":519.9000000953674},{"duration":275.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":520,"connectEnd":520,"connectStart":520,"domainLookupEnd":520,"domainLookupStart":520,"fetchStart":520,"redirectEnd":0,"redirectStart":0,"requestStart":772.4000000953674,"responseEnd":795.5,"responseStart":794.9000000953674,"secureConnectionStart":520},{"duration":251.59999990463257,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":520.1999998092651,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":520.1999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":771.7999997138977,"responseStart":0,"secureConnectionStart":0},{"duration":281.3999996185303,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":520.4000000953674,"connectEnd":520.4000000953674,"connectStart":520.4000000953674,"domainLookupEnd":520.4000000953674,"domainLookupStart":520.4000000953674,"fetchStart":520.4000000953674,"redirectEnd":0,"redirectStart":0,"requestStart":774.9000000953674,"responseEnd":801.7999997138977,"responseStart":801.1999998092651,"secureConnectionStart":520.4000000953674},{"duration":253.80000019073486,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":520.5999999046326,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":520.5999999046326,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":774.4000000953674,"responseStart":0,"secureConnectionStart":0},{"duration":270.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":520.6999998092651,"connectEnd":777.1999998092651,"connectStart":777.1999998092651,"domainLookupEnd":777.1999998092651,"domainLookupStart":777.1999998092651,"fetchStart":520.6999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":777.5999999046326,"responseEnd":791.1999998092651,"responseStart":789.7999997138977,"secureConnectionStart":777.1999998092651},{"duration":543.6000003814697,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":531.7999997138977,"connectEnd":531.7999997138977,"connectStart":531.7999997138977,"domainLookupEnd":531.7999997138977,"domainLookupStart":531.7999997138977,"fetchStart":531.7999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":985.5999999046326,"responseEnd":1075.4000000953674,"responseStart":1071.2999997138977,"secureConnectionStart":531.7999997138977},{"duration":544.1999998092651,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":531.9000000953674,"connectEnd":531.9000000953674,"connectStart":531.9000000953674,"domainLookupEnd":531.9000000953674,"domainLookupStart":531.9000000953674,"fetchStart":531.9000000953674,"redirectEnd":0,"redirectStart":0,"requestStart":987.4000000953674,"responseEnd":1076.0999999046326,"responseStart":1072.4000000953674,"secureConnectionStart":531.9000000953674},{"duration":208.09999990463257,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1037.6999998092651,"connectEnd":1037.6999998092651,"connectStart":1037.6999998092651,"domainLookupEnd":1037.6999998092651,"domainLookupStart":1037.6999998092651,"fetchStart":1037.6999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":1206.7999997138977,"responseEnd":1245.7999997138977,"responseStart":1245,"secureConnectionStart":1037.6999998092651},{"duration":197.19999980926514,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":1262,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1262,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1459.1999998092651,"responseStart":0,"secureConnectionStart":0}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":233,"responseStart":513,"responseEnd":531,"domLoading":517,"domInteractive":1413,"domContentLoadedEventStart":1413,"domContentLoadedEventEnd":1454,"domComplete":1826,"loadEventStart":1826,"loadEventEnd":1827,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1393.0999999046326},{"name":"bigPipe.sidebar-id.end","time":1394},{"name":"bigPipe.activity-panel-pipe-id.start","time":1394.0999999046326},{"name":"bigPipe.activity-panel-pipe-id.end","time":1395},{"name":"activityTabFullyLoaded","time":1472.1999998092651}],"measures":[],"correlationId":"21cf33c85bf031","effectiveType":"4g","downlink":10,"rtt":0,"serverDuration":149,"dbReadsTimeInMs":20,"dbConnsTimeInMs":30,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
Knowing this is a bit old, but there are very good reasons to disable checksumming entirely: ZFS.
We run a multi-hundred TB Galera cluster with thousands of write operations per second. Doing checksumming twice (ZFS plus database) is a waste, and despite being fast, the crc32 CPU instructions are a finite resource. Double-checksumming hurts our performance significantly, even on pretty high-end hardware (nvme ssd, couple dozen ~3ghz EPYC3/4 CPU cores).
Is there any other way to disable checksumming in maria/innodb in this day and age? We're stuck on MySQL 5.7 for a number of reasons, this being one of them.
Note that disabling checksums in ZFS is not an option since that would also lose checksumming of data that is not checksummed by mariadb.