[MDEV-27199] Require ib_logfile0 to exist unless innodb_force_recovery=6 Created: 2021-12-08 Updated: 2023-10-06 Resolved: 2022-01-21 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Storage Engine - InnoDB |
| Fix Version/s: | 10.8.1 |
| Type: | Task | Priority: | Major |
| Reporter: | Marko Mäkelä | Assignee: | Marko Mäkelä |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | corruption, recovery | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Description |
|
The primary purpose of the parameter innodb_force_recovery is to allow data to be dumped from a corrupted database. Ironically, the parameter may also be used to corrupt a database. Until In all versions, the setting innodb_force_recovery=6 will cause the redo log to be ignored. This setting was somewhat redundant, because a similar effect could be achieved by replacing ib_logfile0 with a logically empty log at the desired log sequence number, or by deleting the ib_logfile0 altogether. We must not allow InnoDB to start up when the log file is missing or physically empty, except when the option innodb_force_recovery=6 is specified, to ignore the redo log altogether and to start the database in read-only mode. Previously, to allow InnoDB to automatically create a lost log file, a field FIL_PAGE_FILE_FLUSH_LSN was written to the first page of the system tablespace on shutdown, bypassing the redo log and the doublewrite buffer. If the server was killed in the middle of that write, the entire server could fail to start up, due to a corrupted first page of the system tablespace. Because we will no longer automatically create a new log file, we will not update that field in the system tablespace either. |
| Comments |
| Comment by Marko Mäkelä [ 2021-12-19 ] | ||||
|
| ||||
| Comment by Matthias Leich [ 2022-01-14 ] | ||||
|
| ||||
| Comment by Vladislav Vaintroub [ 2022-01-20 ] | ||||
|
Thing with ib_logfile0 is that people may be used to removing it from offline backups. I certainly did that many times, and if I did, I expect many others do that too. That one can remove redo log after regular shutdown, is common knowledge. It should not take 20GB in the backup, when logically, it is empty. If there is not much effort to retain that feature, I'd retain it, but otherwise write a big Warning into the release notes. | ||||
| Comment by Marko Mäkelä [ 2022-01-20 ] | ||||
|
wlad, before If the server is started up with a zero-sized ib_logfile0, in If a user tries to be smart and creates a zero-sized ib_logfile0 without manually updating the FIL_PAGE_FILE_FLUSH_LSN in the system tablespace to a recent enough LSN, then scary error messages like "page LSN is in the future" may be issued. As always, if some log was discarded while some changes had already been written to data pages, all sort of corruption may occur. If the database was initialized with a server that never updates the FIL_PAGE_FILE_FLUSH_LSN field, then any server startup attempts with a zero-size ib_logfile0 will be refused because of an invalid LSN. If that field was ever updated with a valid LSN by an older server, this safety mechanism cannot work, and the server may "rewind" to an earlier LSN. I believe that any operations that would rebuild the redo log should be left to code that has been developed and tested. Users should not be encouraged to directly manipulate the log files themselves. | ||||
| Comment by Roel Van de Paar [ 2022-01-21 ] | ||||
|
It would be great if the above was clarified in the documentation. | ||||
| Comment by Marko Mäkelä [ 2022-01-21 ] | ||||
|
greenman, can you please make sure that this as well as | ||||
| Comment by Marko Mäkelä [ 2022-01-26 ] | ||||
|
Users who run performance tests (such as wlad) might want to initialize a server with test data, shut it down and then back up the files, to be restored before read/write performance test runs. That would save the trouble of loading data before each test run, and guarantee that each test is starting from the same state. Such users could be accustomed to deleting the ib_logfile0 to save space in the backup. That used to be safe after a clean shutdown. Now that we no longer store the latest LSN anywhere else than the log file, the log file must not be deleted. To save space in the backup, the server could be restarted with a smaller innodb_log_file_size and then shut down. Alternatively, the database could be copied with mariadb-backup and prepared with mariadb-backup --prepare to create a minimal log file, and then backed up. | ||||
| Comment by Marko Mäkelä [ 2022-02-23 ] | ||||
|
|