Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-26784

[Warning] InnoDB: Difficult to find free blocks in the buffer pool

Details

    Description

      The mariabackup --prepare shows this warning:

      2021-10-07  1:13:23 0 [Note] InnoDB: Starting a batch to recover 9962 pages from redo log.
      2021-10-07  1:13:28 0 [Warning] InnoDB: Difficult to find free blocks in the buffer pool (21 search iterations)! 21 failed attempts to flush a page! Consider increasing innodb_buffer_pool_size. Pending flushes (fsync) log: 0; buffer pool: 0. 582296 OS file reads, 504266 OS file writes, 2396 OS fsyncs.
      

      The error is similar to:
      https://jira.mariadb.org/browse/MDEV-20679
      https://jira.mariadb.org/browse/MDEV-19176

      But looks like they are already fixed in 10.5.1

      Attachments

        Issue Links

          Activity

            Issue present in 10.7 trunk

            10.7.1 5cc9cf9a05582307abefd8f3b57548945da86b1a (Debug)

            2021-10-09 13:18:55 0 [Warning] InnoDB: Difficult to find free blocks in the buffer pool (21 search iterations)! 21 failed attempts to flush a page! Consider increasing innodb_buffer_pool_size. Pending flushes (fsync) log: 0; buffer pool: 0. 149 OS file reads, 2 OS file writes, 2 OS fsyncs.
            

            Roel Roel Van de Paar added a comment - Issue present in 10.7 trunk 10.7.1 5cc9cf9a05582307abefd8f3b57548945da86b1a (Debug) 2021-10-09 13:18:55 0 [Warning] InnoDB: Difficult to find free blocks in the buffer pool (21 search iterations)! 21 failed attempts to flush a page! Consider increasing innodb_buffer_pool_size. Pending flushes (fsync) log: 0; buffer pool: 0. 149 OS file reads, 2 OS file writes, 2 OS fsyncs.

            I diagnosed this with thiru today. Similar to MDEV-27461, the buffer pool is filled with modified pages, which will not be written out because the page cleaner is in untimed sleep (MDEV-24278).

            We agreed on a fix that buf_flush_LRU_list_batch() should disregard the 256-page minimum length of buf_pool.LRU (BUF_LRU_MIN_LEN) during recovery, and simply initiate writes of all dirty pages whenever a page needs to be allocated. In the hang that thiru reproduced, the length of buf_pool.LRU was 215 pages.

            That fix is OK to push, along with the regression test case that reproduces the issue.

            marko Marko Mäkelä added a comment - I diagnosed this with thiru today. Similar to MDEV-27461 , the buffer pool is filled with modified pages, which will not be written out because the page cleaner is in untimed sleep ( MDEV-24278 ). We agreed on a fix that buf_flush_LRU_list_batch() should disregard the 256-page minimum length of buf_pool.LRU ( BUF_LRU_MIN_LEN ) during recovery, and simply initiate writes of all dirty pages whenever a page needs to be allocated. In the hang that thiru reproduced, the length of buf_pool.LRU was 215 pages. That fix is OK to push, along with the regression test case that reproduces the issue.

            The Innodb.recovery_memory test fails constantly in bb-11.0 with:

            CURRENT_TEST: innodb.recovery_memory
            mysqltest: In included file "./include/wait_until_connected_again.inc":
            included from ./include/start_mysqld.inc at line 49:
            included from ./include/restart_mysqld.inc at line 11:
            included from /home/my/maria-11.0-org/mysql-test/suite/innodb/t/recovery_memory.test at line 21:
            At line 40: Server failed to restart

            I assume most developers has not noticed this as this is marked as 'big'.

            monty Michael Widenius added a comment - The Innodb.recovery_memory test fails constantly in bb-11.0 with: CURRENT_TEST: innodb.recovery_memory mysqltest: In included file "./include/wait_until_connected_again.inc": included from ./include/start_mysqld.inc at line 49: included from ./include/restart_mysqld.inc at line 11: included from /home/my/maria-11.0-org/mysql-test/suite/innodb/t/recovery_memory.test at line 21: At line 40: Server failed to restart I assume most developers has not noticed this as this is marked as 'big'.

            monty, I think that your observation needs to be considered in the scope of the follow-up fix MDEV-30551 that thiru is currently working on.

            marko Marko Mäkelä added a comment - monty , I think that your observation needs to be considered in the scope of the follow-up fix MDEV-30551 that thiru is currently working on.

            monty Issue has been fixed as MDEV-30673

            thiru Thirunarayanan Balathandayuthapani added a comment - monty Issue has been fixed as MDEV-30673

            People

              thiru Thirunarayanan Balathandayuthapani
              niljoshi Nilnandan Joshi
              Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.