Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-41718

innodb.innodb_buffer_pool_resize_pressure fails with "innodb_buffer_pool_size change is already in progress" when the memory pressure listener is active

    XMLWordPrintable

Details

    Description

      How it fails

      innodb.innodb_buffer_pool_resize_pressure w35 [ fail ]
              Test ended at 2026-10-22 11:53:21
       
      CURRENT_TEST: innodb.innodb_buffer_pool_resize_pressure
      mysqltest: At line 17: query 'SET GLOBAL innodb_buffer_pool_size=@save_size' failed: ER_WRONG_USAGE (1221): innodb_buffer_pool_size change is already in progress
      

      Line 17 is the last line of mysql-test/suite/innodb/t/innodb_buffer_pool_resize_pressure.test. Before that, the search at line 14 also misses its pattern:

      SET STATEMENT debug_dbug='+d,trigger_garbage_collection,buf_shrink_fail' FOR
      SET GLOBAL innodb_buffer_pool_size=25165824;
      NOT FOUND /\[Note\] InnoDB: Memory pressure event failed to shrink innodb_buffer_pool_size=(16|24)m/ in mysqld.1.err
      SET GLOBAL innodb_buffer_pool_size_auto_min=@save_min;
      SET GLOBAL innodb_buffer_pool_size=@save_size;
      

      Server error log for the test:

      2026-10-22 11:53:19 0 [Note] InnoDB: Initialized memory pressure event listener
      ...
      2026-10-22 11:53:20 4 [Note] InnoDB: Memory pressure event disregarded; innodb_buffer_pool_size=24m, innodb_buffer_pool_size_auto_min=8m
      2026-10-22 11:53:20 0 [Note] Got signal 15 to shutdown server
      2026-10-22 11:53:20 0 [Note] /home/midenok/src/mariadb/10.6b/build/sql/mariadbd (initiated by: unknown): Normal shutdown
      2026-10-22 11:53:20 0 [Note] InnoDB: FTS optimize thread exiting.
      2026-10-22 11:53:20 0 [Note] InnoDB: Starting shutdown...
      2026-10-22 11:53:21 0 [Note] InnoDB: Memory pressure event shrunk innodb_buffer_pool_size=16m (1010 pages) from 24m (1515 pages)
      

      The SQL thread (thread 4) logs "disregarded". The successful shrink to 16m comes from the background listener thread (thread 0) and completes only during shutdown, about 1 second after the trigger. No "failed to shrink" message is logged at all.

      The test was added in MDEV-40950 (commit 4df4e197ca7).

      Analysis

      The test:

      --source include/have_innodb.inc
      --source include/not_embedded.inc
      --source include/have_debug.inc
       
      SET @save_min=@@GLOBAL.innodb_buffer_pool_size_auto_min;
      SET @save_size=@@GLOBAL.innodb_buffer_pool_size;
       
      SET GLOBAL innodb_buffer_pool_size_auto_min=8388608;
      SET GLOBAL innodb_buffer_pool_size=25165824;
      SET STATEMENT debug_dbug='+d,trigger_garbage_collection,buf_shrink_fail' FOR
      SET GLOBAL innodb_buffer_pool_size=25165824;
      let SEARCH_FILE = $MYSQLTEST_VARDIR/log/mysqld.1.err;
      let SEARCH_PATTERN=\[Note\] InnoDB: Memory pressure event failed to shrink innodb_buffer_pool_size=(16|24)m;
      --source include/search_pattern_in_file.inc
       
      SET GLOBAL innodb_buffer_pool_size_auto_min=@save_min;
      SET GLOBAL innodb_buffer_pool_size=@save_size;
      

      With trigger_garbage_collection, buf_pool_t::resize() starts two garbage_collect() calls (storage/innobase/buf/buf0buf.cc):

      #ifdef __linux__
        DBUG_EXECUTE_IF("trigger_garbage_collection",
                        mem_pressure_obj.trigger_collection(););     // line 1938
      #endif
       
        if (size == old_size)
        {
          mysql_mutex_unlock(&mutex);
          DBUG_EXECUTE_IF("trigger_garbage_collection",              // line 1945
                          std::this_thread::sleep_for(std::chrono::milliseconds(50));
                          garbage_collect(););
          return;
        }
      

      1. trigger_collection() (line 919) writes to the eventfd. If the listener is initialized, mem_pressure::pressure_routine() wakes up and calls buf_pool.garbage_collect() (line 1003). This thread has its own DBUG state and does not see the session debug_dbug, so buf_shrink_fail does not apply to it and its shrink succeeds.
      2. After a 50ms sleep the SQL thread calls garbage_collect() itself. buf_shrink_fail applies here, so buf_pool_t::shrink() returns SHRINK_ABORT (line 1657) and "Memory pressure event failed to shrink" is logged (line 1117). This is the call the test checks.

      The test expects one of two outcomes:

      • the listener is not initialized (m_event_fd < 0), so only the SQL thread runs garbage_collect() and logs "failed to shrink innodb_buffer_pool_size=24m";
      • the background shrink completes within 50ms, so the SQL thread then shrinks from 16m and logs "failed to shrink innodb_buffer_pool_size=16m".

      Neither holds when the listener is active and the host is busy. The background garbage_collect() sets first_to_withdraw and size_in_bytes_requested, then waits for the page cleaner in my_cond_wait(&done_flush_list, ...) (line 1058). In the failing run this took about 1 second. The SQL thread's garbage_collect() sees the shrink in progress at line 1034:

        if (old_size < min_size + reduce_size ||
            first_to_withdraw || old_size != size_in_bytes_requested)
        {
          mysql_mutex_unlock(&mutex);
          sql_print_information("InnoDB: Memory pressure event disregarded;"
      

      and returns without trying to shrink. As a result:

      • no "failed to shrink" message is logged, so the search at line 14 of the test reports NOT FOUND;
      • the background shrink is still running when the test executes SET GLOBAL innodb_buffer_pool_size=@save_size, and resize() rejects it at line 1926 with ER_WRONG_USAGE "innodb_buffer_pool_size change is already in progress" (line 1930).

      Fix options

      1. Do not wake the listener thread when buf_shrink_fail is set, so only the SQL thread runs garbage_collect() and the outcome is deterministic. MDEV-40950 is about handling SHRINK_ABORT; the eventfd path is not needed to test it, and it is still covered by innodb.mem_pressure:

        #ifdef __linux__
          DBUG_EXECUTE_IF("trigger_garbage_collection",
                          if (!DBUG_IF("buf_shrink_fail"))
                            mem_pressure_obj.trigger_collection(););
        #endif
        

      2. Test only: set buf_shrink_fail globally (SET GLOBAL debug_dbug) so that the listener thread also aborts its shrink. The test must then wait (SEARCH_WAIT=FOUND) for both garbage_collect() calls to finish, each logging either "disregarded" or "failed to shrink", before it restores innodb_buffer_pool_size.
      3. Minimal test change: before the final SET GLOBAL innodb_buffer_pool_size=@save_size, wait for the background shrink to complete, as innodb.mem_pressure does with wait_condition. This fixes only the "already in progress" error. The "failed to shrink" pattern can still be NOT FOUND, so this option is not enough on its own.

      Option 1 is the preferred one.

      Attachments

        Activity

          People

            thiru Thirunarayanan Balathandayuthapani
            midenok Aleksey Midenkov
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.