Details
-
Task
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Won't Do
Description
InnoDB startup unnecessarily waits for recovered redo log records to be applied to the data files.
In fact, normally while the function trx_sys_init_at_db_start() is executing, the pages that it is requesting from the buffer pool will have any recovered redo log applied to them in the background.
Basically, we only need to remove or refactor some calls in the InnoDB server startup. Some of this was performed in MDEV-19514 and MDEV-21216.
The crash recovery would ‘complete’ at the time of the next redo log checkpoint is written.
We should rewrite or remove recv_recovery_from_checkpoint_finish() recv_sys.apply(true) so that it will not wait for any page flushing to complete (already done in MDEV-27022). While doing this, we must also remove the buf_pool_t::flush_rbt (removed in MDEV-23399) and use the normal flushing mechanism that strictly obeys the ARIES protocol for write-ahead logging (implemented in MDEV-24626).
Attachments
Issue Links
- blocks
-
MDEV-12700 Allow innodb_read_only startup without prior slow shutdown
-
- Closed
-
- includes
-
-
- Closed
-
- is blocked by
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
- relates to
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
Activity
| Field | Original Value | New Value |
|---|---|---|
| Link |
This issue relates to |
| Link |
This issue relates to |
| Link |
This issue is blocked by |
| Assignee | Marko Mäkelä [ marko ] | Thirunarayanan B [ thiru ] |
| Link |
This issue is blocked by |
| Link |
This issue relates to |
| NRE Projects | RM_105_CANDIDATE |
| Link |
This issue relates to |
| Link |
This issue relates to |
| Link |
This issue relates to |
| Link |
This issue relates to |
| Link |
This issue relates to |
| Link |
This issue is blocked by |
| Link |
This issue relates to |
| Fix Version/s | 10.5 [ 23123 ] | |
| Fix Version/s | 10.4 [ 22408 ] |
| Link |
This issue blocks |
| Link |
This issue is blocked by |
| Status | Open [ 1 ] | In Progress [ 3 ] |
| Link |
This issue is blocked by |
| Description |
InnoDB startup unnecessarily waits for recovered redo log records to be applied to the data files.
In fact, normally while the function trx_sys_init_at_db_start() is executing, the pages that it is requesting from the buffer pool will have any recovered redo log applied to them in the background. Basically, we only need to remove or refactor some calls in the InnoDB server startup. The crash recovery would ‘complete’ at the time of the next redo log checkpoint is written. One of the code pieces that must be removed is this one: {code:diff} commit 5103b38fc3cbc73d4a66a1b21740c4d7498f4a67 Author: Marko Mäkelä <marko.makela@mariadb.com> Date: Wed Nov 15 06:51:19 2017 +0200 Remove one more trace of innodb_file_format innobase_start_or_create_for_mysql(): Remove an unnecessary call that should have been removed as part of commit 0c92794db3026cda03218caf4918b996baab6ba6 when the dirty read of the innodb_file_format tag was removed. diff --git a/storage/innobase/srv/srv0start.cc b/storage/innobase/srv/srv0start.cc index 988ddf1a759..4f8823ef36c 100644 --- a/storage/innobase/srv/srv0start.cc +++ b/storage/innobase/srv/srv0start.cc @@ -2194,13 +2194,6 @@ innobase_start_or_create_for_mysql() return(srv_init_abort(err)); } } else { - /* Invalidate the buffer pool to ensure that we reread - the page that we read above, during recovery. - Note that this is not as heavy weight as it seems. At - this point there will be only ONE page in the buf_LRU - and there must be no page in the buf_flush list. */ - buf_pool_invalidate(); - /* Scan and locate truncate log files. Parsed located files and add table to truncate information to central vector for truncate fix-up action post recovery. */ {code} However, if we apply this patch, crash recovery will start failing. The reason could be that we are accessing the InnoDB system and undo tablespace files before opening the redo logs: {code:c} fil_open_log_and_system_tablespace_files(); ut_d(fil_space_get(0)->recv_size = srv_sys_space_size_debug); err = srv_undo_tablespaces_init(create_new_db); … err = recv_recovery_from_checkpoint_start(flushed_lsn); {code} If we first open the redo log files and then the persistent data files, then there should be no need to invalidate the buffer pool contents. Similarly, later in the startup, we should rewrite or remove recv_recovery_from_checkpoint_finish() so that it will not wait for any page flushing to complete. While doing this, we must also remove the buf_pool_t::flush_rbt and use the normal flushing mechanism that strictly obeys the ARIES protocol for write-ahead logging. |
InnoDB startup unnecessarily waits for recovered redo log records to be applied to the data files.
In fact, normally while the function {{trx_sys_init_at_db_start()}} is executing, the pages that it is requesting from the buffer pool will have any recovered redo log applied to them in the background. Basically, we only need to remove or refactor some calls in the InnoDB server startup. Some of this was performed in The crash recovery would ‘complete’ at the time of the next redo log checkpoint is written. We should rewrite or remove {{recv_recovery_from_checkpoint_finish()}} so that it will not wait for any page flushing to complete. While doing this, we must also remove the {{buf_pool_t::flush_rbt}} and use the normal flushing mechanism that strictly obeys the ARIES protocol for write-ahead logging. |
| Status | In Progress [ 3 ] | Stalled [ 10000 ] |
| Link |
This issue relates to |
| Due Date | 2020-03-31 |
| Fix Version/s | 10.6 [ 24028 ] | |
| Fix Version/s | 10.5 [ 23123 ] |
| Link |
This issue relates to |
| Due Date | 2020-03-31 |
| Rank | Ranked higher |
| Link |
This issue is blocked by |
| Link |
This issue is blocked by |
| Fix Version/s | N/A [ 14700 ] | |
| Fix Version/s | 10.6 [ 24028 ] |
| Fix Version/s | 10.7 [ 24805 ] | |
| Fix Version/s | N/A [ 14700 ] |
| Fix Version/s | 10.8 [ 26121 ] | |
| Fix Version/s | 10.7 [ 24805 ] |
| Assignee | Thirunarayanan Balathandayuthapani [ thiru ] | Eugene Kosov [ kevg ] |
| Priority | Major [ 3 ] | Critical [ 2 ] |
| Link |
This issue includes |
| Link |
This issue relates to |
| Attachment | Screenshot from 2021-11-26 10-08-41.png [ 60930 ] |
| Attachment | 1.svg [ 60931 ] |
| Workflow | MariaDB v3 [ 83949 ] | MariaDB v4 [ 131680 ] |
| Link |
This issue relates to |
| Fix Version/s | 10.9 [ 26905 ] | |
| Fix Version/s | 10.8 [ 26121 ] |
| Assignee | Eugene Kosov [ kevg ] | Vladislav Lesin [ vlad.lesin ] |
| Fix Version/s | 10.10 [ 27530 ] | |
| Fix Version/s | 10.9 [ 26905 ] |
| Priority | Critical [ 2 ] | Major [ 3 ] |
| Fix Version/s | 10.11 [ 27614 ] | |
| Fix Version/s | 10.10 [ 27530 ] |
| Fix Version/s | 10.12 [ 28320 ] | |
| Fix Version/s | 10.11 [ 27614 ] |
| Description |
InnoDB startup unnecessarily waits for recovered redo log records to be applied to the data files.
In fact, normally while the function {{trx_sys_init_at_db_start()}} is executing, the pages that it is requesting from the buffer pool will have any recovered redo log applied to them in the background. Basically, we only need to remove or refactor some calls in the InnoDB server startup. Some of this was performed in The crash recovery would ‘complete’ at the time of the next redo log checkpoint is written. We should rewrite or remove {{recv_recovery_from_checkpoint_finish()}} so that it will not wait for any page flushing to complete. While doing this, we must also remove the {{buf_pool_t::flush_rbt}} and use the normal flushing mechanism that strictly obeys the ARIES protocol for write-ahead logging. |
InnoDB startup unnecessarily waits for recovered redo log records to be applied to the data files. In fact, normally while the function {{trx_sys_init_at_db_start()}} is executing, the pages that it is requesting from the buffer pool will have any recovered redo log applied to them in the background. Basically, we only need to remove or refactor some calls in the InnoDB server startup. Some of this was performed in The crash recovery would ‘complete’ at the time of the next redo log checkpoint is written. We should rewrite or remove {{recv_recovery_from_checkpoint_finish()}} so that it will not wait for any page flushing to complete. While doing this, we must also remove the {{buf_pool_t::flush_rbt}} and use the normal flushing mechanism that strictly obeys the ARIES protocol for write\-ahead logging. |
| Link |
This issue relates to |
| Status | Stalled [ 10000 ] | In Progress [ 3 ] |
| Fix Version/s | 11.1 [ 28549 ] | |
| Fix Version/s | 11.0 [ 28320 ] |
| Link |
This issue is blocked by |
| Link |
This issue is blocked by |
| Description |
InnoDB startup unnecessarily waits for recovered redo log records to be applied to the data files. In fact, normally while the function {{trx_sys_init_at_db_start()}} is executing, the pages that it is requesting from the buffer pool will have any recovered redo log applied to them in the background. Basically, we only need to remove or refactor some calls in the InnoDB server startup. Some of this was performed in The crash recovery would ‘complete’ at the time of the next redo log checkpoint is written. We should rewrite or remove {{recv_recovery_from_checkpoint_finish()}} so that it will not wait for any page flushing to complete. While doing this, we must also remove the {{buf_pool_t::flush_rbt}} and use the normal flushing mechanism that strictly obeys the ARIES protocol for write\-ahead logging. |
InnoDB startup unnecessarily waits for recovered redo log records to be applied to the data files.
In fact, normally while the function {{trx_sys_init_at_db_start()}} is executing, the pages that it is requesting from the buffer pool will have any recovered redo log applied to them in the background. Basically, we only need to remove or refactor some calls in the InnoDB server startup. Some of this was performed in The crash recovery would ‘complete’ at the time of the next redo log checkpoint is written. We should rewrite or remove -{{recv_recovery_from_checkpoint_finish()}}- {{recv_sys.apply(true)}} so that -it will not wait for any page flushing to complete- (already done in |
| issue.field.resolutiondate | 2023-04-12 14:02:22.0 | 2023-04-12 14:02:22.471 |
| Fix Version/s | N/A [ 14700 ] | |
| Fix Version/s | 11.1 [ 28549 ] | |
| Assignee | Vladislav Lesin [ vlad.lesin ] | Marko Mäkelä [ marko ] |
| Resolution | Won't Do [ 10201 ] | |
| Status | In Progress [ 3 ] | Closed [ 6 ] |
| Link |
This issue relates to |