Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-32042

Special handling of crash recovery in buf_page_get_gen() may cause overhead

Details

    Description

      The fix of bug MDEV-21572 introduced a rarely holding condition in a commonly executed function buf_page_get_gen(), which is only needed while crash recovery is in progress.

      Before the last recovery batch is completed by invoking recv_sys.apply(true), we have the following accesses to the buffer pool:

      • Reading the TRX_SYS page (page 5 in the system tablespace) to determine the number of undo tablespaces.
      • Reading the DICT_HDR page (page 7 in the system tablespace) to initialize the maximum tablespace identifier, in dict_boot()
      • Before MDEV-29694 (MariaDB Server 11.0), reading at least 3 pages to initialize the change buffer.
      • Loading the definitions of some system tables. This can be deferred after recv_sys.apply(true) has been called.

      If we write a separate function that will be used for these 2 special cases of page access, we can simplify the function buf_page_get_gen(). Furthermore, the function buf_flush_LRU_list_batch() may be simplified so that it can enforce the innodb_io_capacity rate limit during recovery, because recv_sys_t::wait_for_pool() (MDEV-29911) would be able to ensure that enough buffer pool is available.

      Attachments

        Issue Links

          Activity

            marko Marko Mäkelä created issue -
            marko Marko Mäkelä made changes -
            Field Original Value New Value
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            marko Marko Mäkelä made changes -
            Description The fix of bug MDEV-21572 introduced a rarely holding condition the a commonly executed function {{buf_page_get_gen()}}, which is only needed while crash recovery is in progress.

            Before the last recovery batch is completed by invoking {{recv_sys.apply(true)}}, we have the following accesses to the buffer pool:
            * Reading the {{TRX_SYS}} page (page 5 in the system tablespace) to determine the number of undo tablespaces.
            * Reading the {{DICT_HDR}} page (page 7 in the system tablespace) to initialize the maximum tablespace identifier, in {{dict_boot()}}
            * Before MDEV-29694 (MariaDB Server 11.0), reading at least 3 pages to initialize the change buffer.
            * Loading the definitions of some system tables. This can be deferred after {{recv_sys.apply(true)}} has been called.

            If we write a separate function that will be used for these 2 special cases of page access, we can simplify the function {{buf_page_get_gen()}}. Furthermore, the function {{buf_flush_LRU_list_batch()}} may be simplified so that it can enforce the {{innodb_io_capacity}} rate limit during recovery, because {{recv_sys_t::wait_for_pool()}} (MDEV-29911) would be able to ensure that enough buffer pool is available.
            The fix of bug MDEV-21572 introduced a rarely holding condition in a commonly executed function {{buf_page_get_gen()}}, which is only needed while crash recovery is in progress.

            Before the last recovery batch is completed by invoking {{recv_sys.apply(true)}}, we have the following accesses to the buffer pool:
            * Reading the {{TRX_SYS}} page (page 5 in the system tablespace) to determine the number of undo tablespaces.
            * Reading the {{DICT_HDR}} page (page 7 in the system tablespace) to initialize the maximum tablespace identifier, in {{dict_boot()}}
            * Before MDEV-29694 (MariaDB Server 11.0), reading at least 3 pages to initialize the change buffer.
            * Loading the definitions of some system tables. This can be deferred after {{recv_sys.apply(true)}} has been called.

            If we write a separate function that will be used for these 2 special cases of page access, we can simplify the function {{buf_page_get_gen()}}. Furthermore, the function {{buf_flush_LRU_list_batch()}} may be simplified so that it can enforce the {{innodb_io_capacity}} rate limit during recovery, because {{recv_sys_t::wait_for_pool()}} (MDEV-29911) would be able to ensure that enough buffer pool is available.
            marko Marko Mäkelä made changes -
            Assignee Marko Mäkelä [ marko ] Thirunarayanan Balathandayuthapani [ thiru ]
            Status In Progress [ 3 ] In Review [ 10002 ]

            origin/11.0-MDEV-32042 e1fb69d6a535a6825a0f151b532ae0dc071b01bd 2023-12-01T11:42:00+02:00
            performed well in RQG testing.

            mleich Matthias Leich added a comment - origin/11.0- MDEV-32042 e1fb69d6a535a6825a0f151b532ae0dc071b01bd 2023-12-01T11:42:00+02:00 performed well in RQG testing.
            thiru Thirunarayanan Balathandayuthapani made changes -
            Assignee Thirunarayanan Balathandayuthapani [ thiru ] Marko Mäkelä [ marko ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            marko Marko Mäkelä made changes -
            issue.field.resolutiondate 2023-12-07 14:38:08.0 2023-12-07 14:38:07.75
            marko Marko Mäkelä made changes -
            Fix Version/s 11.0.5 [ 29520 ]
            Fix Version/s 11.1.4 [ 29024 ]
            Fix Version/s 11.2.3 [ 29521 ]
            Fix Version/s 11.0 [ 28320 ]
            Fix Version/s 11.1 [ 28549 ]
            Fix Version/s 11.2 [ 28603 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            marko Marko Mäkelä made changes -

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.