The primary purpose of the parameter innodb_force_recovery is to allow data to be dumped from a corrupted database.
Ironically, the parameter may also be used to corrupt a database. Until MDEV-19514 in MariaDB Server 10.5, innodb_force_recovery=4 and larger settings could cause corruption of secondary indexes.
In all versions, the setting innodb_force_recovery=6 will cause the redo log to be ignored. This setting was somewhat redundant, because a similar effect could be achieved by replacing ib_logfile0 with a logically empty log at the desired log sequence number, or by deleting the ib_logfile0 altogether.
We must not allow InnoDB to start up when the log file is missing or physically empty, except when the option innodb_force_recovery=6 is specified, to ignore the redo log altogether and to start the database in read-only mode.
Previously, to allow InnoDB to automatically create a lost log file, a field FIL_PAGE_FILE_FLUSH_LSN was written to the first page of the system tablespace on shutdown, bypassing the redo log and the doublewrite buffer. If the server was killed in the middle of that write, the entire server could fail to start up, due to a corrupted first page of the system tablespace. Because we will no longer automatically create a new log file, we will not update that field in the system tablespace either.
Attachments
Issue Links
relates to
MDEV-27208Implement 2-ary CRC32() and the CRC32C() function
Closed
MDEV-27812Allow innodb_log_file_size to change without server restart
Closed
MDEV-32361mariadb-backup --move-back leaves out ib_logfile0
MDEV-14425 will make InnoDB refuse to start up without ib_logfile0 unless innodb_force_recovery=6 is specified. In preview-10.8-MDEV-14425-innodb there is a change that removes the updates of the field FIL_PAGE_FILE_FLUSH_LSN in the system tablespace, because due to MDEV-14425, that field is no longer being read.
Marko Mäkelä
added a comment - MDEV-14425 will make InnoDB refuse to start up without ib_logfile0 unless innodb_force_recovery=6 is specified. In preview-10.8- MDEV-14425 -innodb there is a change that removes the updates of the field FIL_PAGE_FILE_FLUSH_LSN in the system tablespace, because due to MDEV-14425 , that field is no longer being read.
which contains the patch for MDEV-27199 behaved well in RQG testing.
Bad effects observed exist in the main trees too and are known.
Matthias Leich
added a comment -
The tree
origin/bb-10.8-MDEV-14425 614e46b89ffe7357e5b72ea0d0fd3f490567a384 2022-01-13T20:32:56+02:00
which contains the patch for MDEV-27199 behaved well in RQG testing.
Bad effects observed exist in the main trees too and are known.
Thing with ib_logfile0 is that people may be used to removing it from offline backups. I certainly did that many times, and if I did, I expect many others do that too. That one can remove redo log after regular shutdown, is common knowledge. It should not take 20GB in the backup, when logically, it is empty. If there is not much effort to retain that feature, I'd retain it, but otherwise write a big Warning into the release notes.
Vladislav Vaintroub
added a comment - Thing with ib_logfile0 is that people may be used to removing it from offline backups. I certainly did that many times, and if I did, I expect many others do that too. That one can remove redo log after regular shutdown, is common knowledge. It should not take 20GB in the backup, when logically, it is empty. If there is not much effort to retain that feature, I'd retain it, but otherwise write a big Warning into the release notes.
wlad, before MDEV-14425, mariadb-backup --prepare created a zero-length ib_logfile0 as a dummy placeholder. In MDEV-14425, the size of that dummy file was increased to 12304 (0x3010) bytes. The code change attached to this MDEV-27199 will merely remove all updates of FIL_PAGE_FILE_FLUSH_LSN in the first page of the system tablespace.
If the server is started up with a zero-sized ib_logfile0, in MDEV-14425 we will assume that we are upgrading after a backup had been prepared. The start LSN will then be read from FIL_PAGE_FILE_FLUSH_LSN, and a new log file will be created starting from exactly that LSN.
If a user tries to be smart and creates a zero-sized ib_logfile0 without manually updating the FIL_PAGE_FILE_FLUSH_LSN in the system tablespace to a recent enough LSN, then scary error messages like "page LSN is in the future" may be issued. As always, if some log was discarded while some changes had already been written to data pages, all sort of corruption may occur.
If the database was initialized with a server that never updates the FIL_PAGE_FILE_FLUSH_LSN field, then any server startup attempts with a zero-size ib_logfile0 will be refused because of an invalid LSN. If that field was ever updated with a valid LSN by an older server, this safety mechanism cannot work, and the server may "rewind" to an earlier LSN.
I believe that any operations that would rebuild the redo log should be left to code that has been developed and tested. Users should not be encouraged to directly manipulate the log files themselves.
Marko Mäkelä
added a comment - wlad , before MDEV-14425 , mariadb-backup --prepare created a zero-length ib_logfile0 as a dummy placeholder. In MDEV-14425 , the size of that dummy file was increased to 12304 (0x3010) bytes. The code change attached to this MDEV-27199 will merely remove all updates of FIL_PAGE_FILE_FLUSH_LSN in the first page of the system tablespace.
If the server is started up with a zero-sized ib_logfile0 , in MDEV-14425 we will assume that we are upgrading after a backup had been prepared. The start LSN will then be read from FIL_PAGE_FILE_FLUSH_LSN , and a new log file will be created starting from exactly that LSN.
If a user tries to be smart and creates a zero-sized ib_logfile0 without manually updating the FIL_PAGE_FILE_FLUSH_LSN in the system tablespace to a recent enough LSN, then scary error messages like "page LSN is in the future" may be issued. As always, if some log was discarded while some changes had already been written to data pages, all sort of corruption may occur.
If the database was initialized with a server that never updates the FIL_PAGE_FILE_FLUSH_LSN field, then any server startup attempts with a zero-size ib_logfile0 will be refused because of an invalid LSN. If that field was ever updated with a valid LSN by an older server, this safety mechanism cannot work, and the server may "rewind" to an earlier LSN.
I believe that any operations that would rebuild the redo log should be left to code that has been developed and tested. Users should not be encouraged to directly manipulate the log files themselves.
Users who run performance tests (such as wlad) might want to initialize a server with test data, shut it down and then back up the files, to be restored before read/write performance test runs. That would save the trouble of loading data before each test run, and guarantee that each test is starting from the same state.
Such users could be accustomed to deleting the ib_logfile0 to save space in the backup. That used to be safe after a clean shutdown. Now that we no longer store the latest LSN anywhere else than the log file, the log file must not be deleted. To save space in the backup, the server could be restarted with a smaller innodb_log_file_size and then shut down. Alternatively, the database could be copied with mariadb-backup and prepared with mariadb-backup --prepare to create a minimal log file, and then backed up.
Marko Mäkelä
added a comment - Users who run performance tests (such as wlad ) might want to initialize a server with test data, shut it down and then back up the files, to be restored before read/write performance test runs. That would save the trouble of loading data before each test run, and guarantee that each test is starting from the same state.
Such users could be accustomed to deleting the ib_logfile0 to save space in the backup. That used to be safe after a clean shutdown. Now that we no longer store the latest LSN anywhere else than the log file, the log file must not be deleted. To save space in the backup, the server could be restarted with a smaller innodb_log_file_size and then shut down. Alternatively, the database could be copied with mariadb-backup and prepared with mariadb-backup --prepare to create a minimal log file, and then backed up.
MDEV-27812 would allow a DBA to shrink the log file before shutting down the server. The log could simply be resized to its minimal size:
SETGLOBAL innodb_log_file_size=4096*1024;
Marko Mäkelä
added a comment - MDEV-27812 would allow a DBA to shrink the log file before shutting down the server. The log could simply be resized to its minimal size:
SET GLOBAL innodb_log_file_size=4096*1024;
People
Marko Mäkelä
Marko Mäkelä
Votes:
0Vote for this issue
Watchers:
7Start watching this issue
Dates
Created:
Updated:
Resolved:
Git Integration
Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.
{"report":{"fcp":1750.2000002861023,"ttfb":357.40000009536743,"pageVisibility":"visible","entityId":105817,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":32,"apdex":0.5,"journeyId":"be606118-8018-412c-97e3-9d6502d54f11","navigationType":0,"readyForUser":1847,"redirectCount":0,"resourceLoadedEnd":2264.2000002861023,"resourceLoadedStart":364.80000019073486,"resourceTiming":[{"duration":763.1999998092651,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":364.80000019073486,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":364.80000019073486,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1128,"responseStart":0,"secureConnectionStart":0},{"duration":763.1999998092651,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":365.1000003814697,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":365.1000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1128.3000001907349,"responseStart":0,"secureConnectionStart":0},{"duration":779.1999998092651,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":365.30000019073486,"connectEnd":365.30000019073486,"connectStart":365.30000019073486,"domainLookupEnd":365.30000019073486,"domainLookupStart":365.30000019073486,"fetchStart":365.30000019073486,"redirectEnd":0,"redirectStart":0,"requestStart":365.30000019073486,"responseEnd":1144.5,"responseStart":1144.5,"secureConnectionStart":365.30000019073486},{"duration":934.8000001907349,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":365.5,"connectEnd":365.5,"connectStart":365.5,"domainLookupEnd":365.5,"domainLookupStart":365.5,"fetchStart":365.5,"redirectEnd":0,"redirectStart":0,"requestStart":365.5,"responseEnd":1300.3000001907349,"responseStart":1300.3000001907349,"secureConnectionStart":365.5},{"duration":937.6999998092651,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":365.6000003814697,"connectEnd":365.6000003814697,"connectStart":365.6000003814697,"domainLookupEnd":365.6000003814697,"domainLookupStart":365.6000003814697,"fetchStart":365.6000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":365.6000003814697,"responseEnd":1303.3000001907349,"responseStart":1303.3000001907349,"secureConnectionStart":365.6000003814697},{"duration":938.0999999046326,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":365.90000009536743,"connectEnd":365.90000009536743,"connectStart":365.90000009536743,"domainLookupEnd":365.90000009536743,"domainLookupStart":365.90000009536743,"fetchStart":365.90000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":365.90000009536743,"responseEnd":1304,"responseStart":1303.9000000953674,"secureConnectionStart":365.90000009536743},{"duration":938.4000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":366,"connectEnd":366,"connectStart":366,"domainLookupEnd":366,"domainLookupStart":366,"fetchStart":366,"redirectEnd":0,"redirectStart":0,"requestStart":366,"responseEnd":1304.4000000953674,"responseStart":1304.4000000953674,"secureConnectionStart":366},{"duration":973.2999997138977,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":366.1000003814697,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":366.1000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1339.4000000953674,"responseStart":0,"secureConnectionStart":0},{"duration":938.6999998092651,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":366.2000002861023,"connectEnd":366.2000002861023,"connectStart":366.2000002861023,"domainLookupEnd":366.2000002861023,"domainLookupStart":366.2000002861023,"fetchStart":366.2000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":366.2000002861023,"responseEnd":1304.9000000953674,"responseStart":1304.9000000953674,"secureConnectionStart":366.2000002861023},{"duration":973.2000002861023,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":366.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":366.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1339.7000002861023,"responseStart":0,"secureConnectionStart":0},{"duration":938.7999997138977,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":366.6000003814697,"connectEnd":366.6000003814697,"connectStart":366.6000003814697,"domainLookupEnd":366.6000003814697,"domainLookupStart":366.6000003814697,"fetchStart":366.6000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":366.6000003814697,"responseEnd":1305.4000000953674,"responseStart":1305.4000000953674,"secureConnectionStart":366.6000003814697},{"duration":1130.9000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":373.7000002861023,"connectEnd":373.7000002861023,"connectStart":373.7000002861023,"domainLookupEnd":373.7000002861023,"domainLookupStart":373.7000002861023,"fetchStart":373.7000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":373.7000002861023,"responseEnd":1504.6000003814697,"responseStart":1504.6000003814697,"secureConnectionStart":373.7000002861023},{"duration":1861.5999999046326,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":402.6000003814697,"connectEnd":402.6000003814697,"connectStart":402.6000003814697,"domainLookupEnd":402.6000003814697,"domainLookupStart":402.6000003814697,"fetchStart":402.6000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":402.6000003814697,"responseEnd":2264.2000002861023,"responseStart":2264.2000002861023,"secureConnectionStart":402.6000003814697},{"duration":138.90000009536743,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1366.2000002861023,"connectEnd":1366.2000002861023,"connectStart":1366.2000002861023,"domainLookupEnd":1366.2000002861023,"domainLookupStart":1366.2000002861023,"fetchStart":1366.2000002861023,"redirectEnd":0,"redirectStart":0,"requestStart":1366.2000002861023,"responseEnd":1505.1000003814697,"responseStart":1505.1000003814697,"secureConnectionStart":1366.2000002861023}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":185,"responseStart":358,"responseEnd":406,"domLoading":363,"domInteractive":2305,"domContentLoadedEventStart":2305,"domContentLoadedEventEnd":2366,"domComplete":3309,"loadEventStart":3309,"loadEventEnd":3309,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":2268.800000190735},{"name":"bigPipe.sidebar-id.end","time":2269.7000002861023},{"name":"bigPipe.activity-panel-pipe-id.start","time":2269.800000190735},{"name":"bigPipe.activity-panel-pipe-id.end","time":2272.2000002861023},{"name":"activityTabFullyLoaded","time":2383.4000000953674}],"measures":[],"correlationId":"d0ed199a6245ce","effectiveType":"4g","downlink":10,"rtt":0,"serverDuration":117,"dbReadsTimeInMs":17,"dbConnsTimeInMs":27,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
MDEV-14425will make InnoDB refuse to start up without ib_logfile0 unless innodb_force_recovery=6 is specified. In preview-10.8-MDEV-14425-innodb there is a change that removes the updates of the field FIL_PAGE_FILE_FLUSH_LSN in the system tablespace, because due toMDEV-14425, that field is no longer being read.