Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-30829

InnoDB: Cannot close file <tablename>.ibd because of pending fsync

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Incomplete
    • 10.5.19
    • N/A
    • OTHER
    • Ubuntu 20 LTS x86_64 with 64GB Physical RAM

    Description

      Dear Engineer
      Since we switched one of the replication slave machines to 10.5.19, we are observing the following strange messages in our MariaDB error log file.

      2023-01-01 21:20:13 449813 [Note] InnoDB: Cannot close file ./cvt/specificroutes008.ibd because of pending fsync
      		 
      2023-01-01 21:30:13 449813 [Note] InnoDB: Cannot close file ./cvt/specificroutes008.ibd because of pending fsync
      		 
      2023-01-01 22:00:13 449813 [Note] InnoDB: Cannot close file ./cvt/specificroutes008.ibd because of pending fsync
      		 
      2023-01-01 22:10:14 449813 [Note] InnoDB: Cannot close file ./cvt/specificroutes008.ibd because of pending fsync
      		 
      2023-01-01 23:00:14 449813 [Note] InnoDB: Cannot close file ./cvt/specificroutes008.ibd because of pending fsync
      		 
      2023-01-02  0:50:02 449813 [Note] InnoDB: Cannot close file ./cvt/specificroutes008.ibd because of pending fsync
      		 
      2023-01-02  1:10:02 449813 [Note] InnoDB: Cannot close file ./cvt/specificroutes008.ibd because of pending fsync
      		 
      2023-01-02  2:10:03 449813 [Note] InnoDB: Cannot close file ./cvt/specificroutes008.ibd because of pending fsync
      		 
      2023-01-02  5:20:01 469053 [Note] InnoDB: Cannot close file ./cvt/log01.ibd because of pending fsync
      		 
      2023-01-02  5:20:06 469053 [Note] InnoDB: Cannot close file ./cvt/log01.ibd because of pending fsync
      		 
      2023-01-02  5:20:11 469053 [Note] InnoDB: Cannot close file ./cvt/log01.ibd because of pending fsync
      		 
      2023-01-02  5:40:07 469671 [Note] InnoDB: Cannot close file ./cvt/log01.ibd because of pending fsync
      		 
      2023-01-02  5:50:07 469671 [Note] InnoDB: Cannot close file ./cvt/log01.ibd because of pending fsync
      		 
      2023-01-02  6:10:07 469671 [Note] InnoDB: Cannot close file ./cvt/log01.ibd because of pending fsync
      		 
      2023-01-02  6:20:07 469671 [Note] InnoDB: Cannot close file ./cvt/log01.ibd because of pending fsync
      		 
      2023-01-02  7:00:08 469671 [Note] InnoDB: Cannot close file ./cvt/log01.ibd because of pending fsync
      		 
      2023-01-02  7:10:08 469671 [Note] InnoDB: Cannot close file ./cvt/log01.ibd because of pending fsync
      		 
      2023-01-02  7:30:09 469671 [Note] InnoDB: Cannot close file ./cvt/log01.ibd because of pending fsync
      		 
      2023-01-02  9:00:10 469671 [Note] InnoDB: Cannot close file ./cvt/log01.ibd because of pending fsync
      		 
      2023-01-02  9:10:10 469671 [Note] InnoDB: Cannot close file ./cvt/log01.ibd because of pending fsync
      		 
      2023-01-02 10:50:12 469671 [Note] InnoDB: Cannot close file ./cvt/log01.ibd because of pending fsync
      		 
      2023-01-02 11:00:12 469671 [Note] InnoDB: Cannot close file ./cvt/log01.ibd because of pending fsync
      		 
      2023-01-02 11:20:12 469671 [Note] InnoDB: Cannot close file ./cvt/log01.ibd because of pending fsync
      		 
      2023-01-06  4:02:37 140075 [Note] InnoDB: Cannot close file ./cvt/standardroutes.ibd because of pending fsync
      		 
      2023-01-06  4:10:09 140075 [Note] InnoDB: Cannot close file ./cvt/standardroutes.ibd because of pending fsync
      		 
      2023-01-06  4:20:09 140075 [Note] InnoDB: Cannot close file ./cvt/standardroutes.ibd because of pending fsync
      		 
      2023-01-06  4:30:09 140075 [Note] InnoDB: Cannot close file ./cvt/standardroutes.ibd because of pending fsync
      		 
      2023-01-06  4:40:10 140075 [Note] InnoDB: Cannot close file ./cvt/standardroutes.ibd because of pending fsync
      		 
      2023-01-06  4:50:10 140075 [Note] InnoDB: Cannot close file ./cvt/standardroutes.ibd because of pending fsync
      		 
      2023-01-06  5:00:10 140075 [Note] InnoDB: Cannot close file ./cvt/standardroutes.ibd because of pending fsync
      		 
      2023-01-06  5:10:10 140075 [Note] InnoDB: Cannot close file ./cvt/standardroutes.ibd because of pending fsync
      		 
      2023-01-06  5:20:10 140075 [Note] InnoDB: Cannot close file ./cvt/standardroutes.ibd because of pending fsync
      		 
      2023-01-06  5:50:11 140075 [Note] InnoDB: Cannot close file ./cvt/standardroutes.ibd because of pending fsync
      		 
      2023-01-06  6:10:11 140075 [Note] InnoDB: Cannot close file ./cvt/standardroutes.ibd because of pending fsync
      		 
      2023-01-06  9:20:14 140075 [Note] InnoDB: Cannot close file ./cvt/standardroutes.ibd because of pending fsync
      		 
      2023-01-08  4:10:01 262765 [Note] InnoDB: Cannot close file ./cvt/activesessionlog.ibd because of pending fsync
      		 
      2023-01-08  4:20:01 262765 [Note] InnoDB: Cannot close file ./cvt/activesessionlog.ibd because of pending fsync
      		 
      2023-01-08  4:50:02 262765 [Note] InnoDB: Cannot close file ./cvt/activesessionlog.ibd because of pending fsync
      		 
      2023-01-08  5:00:02 262765 [Note] InnoDB: Cannot close file ./cvt/activesessionlog.ibd because of pending fsync
      		 
      2023-01-08  5:20:02 262765 [Note] InnoDB: Cannot close file ./cvt/activesessionlog.ibd because of pending fsync
      		 
      2023-01-08  5:50:03 262765 [Note] InnoDB: Cannot close file ./cvt/activesessionlog.ibd because of pending fsync
      		 
      2023-01-08  6:00:03 262765 [Note] InnoDB: Cannot close file ./cvt/activesessionlog.ibd because of pending fsync
      		 
      2023-01-08  6:10:03 262765 [Note] InnoDB: Cannot close file ./cvt/activesessionlog.ibd because of pending fsync
      		 
      2023-01-08  9:30:06 262765 [Note] InnoDB: Cannot close file ./cvt/activesessionlog.ibd because of pending fsync
      		 
      2023-01-08 13:20:11 262765 [Note] InnoDB: Cannot close file ./cvt/activesessionlog.ibd because of pending fsync
      		 
      2023-01-08 13:30:11 262765 [Note] InnoDB: Cannot close file ./cvt/activesessionlog.ibd because of pending fsync
      		 
      2023-01-08 15:50:14 262765 [Note] InnoDB: Cannot close file ./cvt/activesessionlog.ibd because of pending fsync
      		 
      2023-01-08 18:10:01 262765 [Note] InnoDB: Cannot close file ./cvt/activesessionlog.ibd because of pending fsync
      		 
      2023-01-09  2:40:10 262765 [Note] InnoDB: Cannot close file ./cvt/activesessionlog.ibd because of pending fsync
      		 
      2023-01-09  2:50:10 262765 [Note] InnoDB: Cannot close file ./cvt/activesessionlog.ibd because of pending fsync
      		 
      2023-02-04  4:10:02 1295717 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-04  4:20:02 1295717 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-04  4:30:02 1295717 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-04  4:40:02 1295717 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-04  4:50:03 1295717 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-04  6:10:04 1296239 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-04  6:20:04 1296239 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-04  6:52:17 1297925 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-05 13:20:05 29055 [Note] InnoDB: Cannot close file ./cvt/transactivitysc027.ibd because of pending fsync
      		 
      2023-02-05 13:30:05 29055 [Note] InnoDB: Cannot close file ./cvt/transactivitysc027.ibd because of pending fsync
      		 
      2023-02-05 14:10:06 29055 [Note] InnoDB: Cannot close file ./cvt/transactivitysc027.ibd because of pending fsync
      		 
      2023-02-05 20:10:12 29055 [Note] InnoDB: Cannot close file ./cvt/transactivitysc027.ibd because of pending fsync
      		 
      2023-02-05 23:10:00 29055 [Note] InnoDB: Cannot close file ./cvt/transactivitysc027.ibd because of pending fsync
      		 
      2023-02-11  5:30:12 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11  5:40:13 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11  5:50:13 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11  6:00:13 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11  6:10:13 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11  6:30:13 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11  7:00:14 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11  7:10:14 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11  7:50:00 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11  8:10:00 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11 10:30:02 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11 10:40:02 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11 14:10:06 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11 14:20:06 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11 15:00:07 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11 15:10:07 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11 15:40:08 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11 18:10:10 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11 18:20:10 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11 20:20:12 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11 20:50:13 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11 21:20:14 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-11 22:20:00 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-12  0:10:01 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-12  1:10:02 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-12  1:20:02 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-12  2:10:03 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-12  2:40:04 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-12  3:20:04 297501 [Note] InnoDB: Cannot close file ./cvt/deliveryaddress.ibd because of pending fsync
      		 
      2023-02-27  4:02:28 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  4:02:40 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  4:02:46 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  4:02:57 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  4:03:03 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  4:03:08 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  4:05:47 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  4:06:21 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  4:40:11 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  4:50:11 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  5:00:11 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  5:30:12 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  5:40:12 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  5:50:12 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  6:00:12 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  6:10:12 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  6:40:13 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-02-27  6:50:13 1134307 [Note] InnoDB: Cannot close file ./cvt/transactivity005.ibd because of pending fsync
      		 
      2023-03-06  4:02:33 1568412 [Note] InnoDB: Cannot close file ./cvt/specificproduct.ibd because of pending fsync
      		 
      2023-03-06  4:10:09 1568412 [Note] InnoDB: Cannot close file ./cvt/specificproduct.ibd because of pending fsync
      		 
      2023-03-06  4:20:09 1568412 [Note] InnoDB: Cannot close file ./cvt/specificproduct.ibd because of pending fsync
      		 
      2023-03-06  5:00:10 1568412 [Note] InnoDB: Cannot close file ./cvt/specificproduct.ibd because of pending fsync
      		 
      2023-03-06  5:50:11 1568412 [Note] InnoDB: Cannot close file ./cvt/specificproduct.ibd because of pending fsync
      		 
      2023-03-06  6:00:11 1568412 [Note] InnoDB: Cannot close file ./cvt/specificproduct.ibd because of pending fsync
      		 
      2023-03-06  6:10:11 1568412 [Note] InnoDB: Cannot close file ./cvt/specificproduct.ibd because of pending fsync
      		 
      2023-03-06  9:40:14 1568412 [Note] InnoDB: Cannot close file ./cvt/specificproduct.ibd because of pending fsync
      		 
      2023-03-06  9:50:00 1568412 [Note] InnoDB: Cannot close file ./cvt/specificproduct.ibd because of pending fsync
      		 
      2023-03-06 11:10:01 1568412 [Note] InnoDB: Cannot close file ./cvt/specificproduct.ibd because of pending fsync
      		 
      2023-03-06 15:10:06 1568412 [Note] InnoDB: Cannot close file ./cvt/specificproduct.ibd because of pending fsync

      As you can see every time the table name (IBD) is getting changed?
      We are not sure "Is it a bug or some kind of config error or hardware-related issue or what?"
      Even do not know if it is affecting the replication quality or not.
      One thing we can say is that it is only pointing to an InnoDB table while we have lots of MyISAM tables too.

      Below I am placing the current configuration file too

      [client]
      		 
      port=3306
      		 
      socket=/run/mysqld/mysqld.sock
      		 
       
      		 
      [mysqld-safe]
      		 
      socket=/run/mysqld/mysqld.sock
      		 
      nice=0
      		 
      log-error=/var/log/mysqld.log
      		 
      pid-file=/data/mysqld.pid
      		 
       
      		 
      [mysqld]
      		 
      user=mysql
      		 
      socket=/run/mysqld/mysqld.sock
      		 
      bind-address=0.0.0.0
      		 
      port=3306
      		 
      basedir=/usr
      		 
      datadir=/data
      		 
      pid-file=/data/mysqld.pid
      		 
      log-error=/var/log/mysqld.log
      		 
      performance-schema=0
      		 
      sql-mode=""
      		 
      log-warnings=0
      		 
      #slow-query-log = 1
      		 
      #long-query-time = 3
      		 
      #slow-query_log-file = /var/log/mysqld-slow.log
      		 
      #long-query-time=1
      		 
      max-connections=100
      		 
      tmpdir=/tmp/
      		 
      bulk-insert-buffer-size=256M
      		 
      innodb-file-per-table = 1
      		 
      innodb-data-home-dir=/data
      		 
      innodb-data-file-path=ibdata1:1G:autoextend
      		 
      innodb-buffer-pool-size=10G
      		 
      innodb-log-file-size=1G
      		 
      innodb-log-buffer-size=1G
      		 
      innodb_stats_persistent=0
      		 
      preload-buffer-size=1M
      		 
      sort-buffer-size=1M
      		 
      myisam-sort-buffer-size=1M
      		 
      key-buffer-size=4G
      		 
      thread-stack=8M
      		 
      bulk-insert-buffer-size=2M
      		 
      innodb-flush-log-at-trx-commit=2
      		 
      table-cache=512
      		 
      query-cache-size=96M
      		 
      query-cache-type=1
      		 
      query-cache-limit=1M
      		 
      join-buffer-size=1M
      		 
      tmp-table-size=64M
      		 
      max-heap-table-size=64M
      		 
      read-buffer-size=2M
      		 
      read-rnd-buffer-size=2M
      		 
      bulk-insert-buffer-size=2M
      		 
      myisam-sort-buffer-size=2M
      		 
      myisam-max-sort-file-size=2M
      		 
      myisam-repair-threads=1
      		 
      #log-bin-trust-function-creators=1
      		 
      interactive-timeout=5000
      		 
      wait-timeout=5000
      		 
      log-bin = /backup/mysqlbinlog/teak15-bin
      		 
      log-bin-index = /backup/mysqlbinlog/teak15-bin-index
      		 
      expire-logs-days = 30
      		 
      server-id = 15
      		 
      innodb-fast-shutdown=0
      		 
      innodb-force-recovery=0
      		 
      max-allowed-packet=256M
      		 
      binlog-format = STATEMENT
      		 
      log-slave-updates=1
      		 
      binlog-ignore-db=information_schema
      		 
      binlog-ignore-db=mysql
      		 
      binlog-ignore-db=performance_schema
      		 
      binlog-ignore-db=test
      		 
      replicate-ignore-db=information_schema
      		 
      replicate-ignore-db=mysql
      		 
      replicate-ignore-db=performance_schema
      		 
      replicate-ignore-db=test
      		 
      default-tmp-storage-engine=MyISAM
      		 
      optimizer-switch='rowid_filter=off'
      		 
       
      		 
      [mysqldump]
      		 
      quick
      		 
      quote-names
      		 
      max-allowed-packet=256M

      please help us
      thanks
      Devang

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-25215 Excessive logging "Cannot close file ./test/FTS_xxx.ibd because of n pending operations and pending fsync

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-30775 Performance regression in fil_space_t::try_to_close() introduced in MDEV-23855

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-31049 fil_delete_tablespace() returns wrong file handle if tablespace was closed by parallel thread

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-31347 fil_ibd_create() may hijack the file handle of an old file

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              marko Marko Mäkelä
              devang_modi Devang
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.