Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-23807

Assertion n_pending_flushes failed in fil_node_t::prepare_to_close_or_detach()

Details

    Description

      In a 10.5-based branch, we got an assertion failure on the shutdown of mariabackup --prepare:

      2020-09-18  6:37:20 0 [Note] InnoDB: Initializing buffer pool, total size = 104857600, chunk size = 104857600
      2020-09-18  6:37:20 0 [Note] InnoDB: Completed initialization of buffer pool
      2020-09-18  6:37:21 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
      2020-09-18  6:37:21 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=3423646
      2020-09-18  6:37:21 0 [Note] InnoDB: Starting final batch to recover 44 pages from redo log.
      2020-09-18  6:37:22 0 [Note] InnoDB: Last binlog file './mysql-bin.000001', position 970415
      [00] 2020-09-18 06:37:23 Last binlog file ./mysql-bin.000001, position 970415
      2020-09-18 06:37:23 0x2bf14d750800  InnoDB: Assertion failure in file /home/mleich/Server/bb-10.5-MDEV-23399B/storage/innobase/fil/fil0fil.cc line 508
      InnoDB: Failing assertion: n_pending_flushes == 0
      

      I believe that this failure is possible also during server shutdown.

      The fix should be to check for pending flushes while closing files:

      diff --git a/storage/innobase/fil/fil0fil.cc b/storage/innobase/fil/fil0fil.cc
      index f549938bfc2..84425e0d1b6 100644
      --- a/storage/innobase/fil/fil0fil.cc
      +++ b/storage/innobase/fil/fil0fil.cc
      @@ -1601,7 +1601,8 @@ void fil_close_all_files()
       				if (!node->is_open()) {
       					goto next;
       				}
      -				if (!node->n_pending) {
      +				if (!node->n_pending
      +				    && !node->n_pending_flushes) {
       					node->close();
       					goto next;
       				}
      @@ -1609,7 +1610,9 @@ void fil_close_all_files()
       
       			ib::error() << "File '" << node->name
       				    << "' has " << node->n_pending
      -				    << " operations";
      +				    << " operations and "
      +				    << node->n_pending_flushes
      +				    << " flushes";
       		}
       
       		space = UT_LIST_GET_NEXT(space_list, space);
      

      Attachments

        Issue Links

          Activity

            marko Marko Mäkelä created issue -
            marko Marko Mäkelä made changes -
            Field Original Value New Value
            Status Open [ 1 ] In Progress [ 3 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.3 [ 22126 ]
            Fix Version/s 10.4 [ 22408 ]
            Affects Version/s 10.5.4 [ 24264 ]
            Affects Version/s 10.2 [ 14601 ]
            Affects Version/s 10.3 [ 22126 ]
            Affects Version/s 10.4 [ 22408 ]
            Affects Version/s 10.5 [ 23123 ]

            On closer inspection, it looks like this may have been caused by MDEV-15053, where we removed some tablespace lookups. The code indicated in the Description was introduced in MDEV-15053.

            marko Marko Mäkelä added a comment - On closer inspection, it looks like this may have been caused by MDEV-15053 , where we removed some tablespace lookups. The code indicated in the Description was introduced in MDEV-15053 .
            marko Marko Mäkelä made changes -
            issue.field.resolutiondate 2020-09-25 15:03:31.0 2020-09-25 15:03:31.878
            marko Marko Mäkelä made changes -
            Fix Version/s 10.5.6 [ 24508 ]
            Fix Version/s 10.5 [ 23123 ]
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Closed [ 6 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Fix Version/s 10.5.7 [ 25019 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Fix Version/s 10.5.6 [ 24508 ]
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 113806 ] MariaDB v4 [ 158397 ]

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.