Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-41152

Recovery may fail if a file covered by FILE_CREATE is missing

    XMLWordPrintable

Details

    • Can result in data loss
    • Crash recovery or applying a backup that involves InnoDB data file creation might fail in rare cases.

    Description

      saahil reported a recovery failure while testing MDEV-14992. I can reproduce the failure with a copy of the data set, data.tar.xz, on a branch that supports innodb_log_archive (MDEV-37949):

      tar xJf data.tar.xz
      mariadbd --innodb-page-size=8k --innodb-log-recovery-start=6699128 --datadir="$PWD"/data
      

      The log can be converted to the 10.11 format by following steps similar to MDEV-40728. First, create a log file header by executing the following SQL:

      mysql-test/main/log.test

      set @first_lsn=x'0000000000003000';
      set @checkpoint_lsn=x'0000000000652fa4';
      set @end_lsn=x'0000000000663878';
      set @header=concat('Phys',x'00000000',@first_lsn,repeat(x'00',492));
      set @header=concat(@header,unhex(hex(crc32c(@header))),repeat(x'00',3584));
      set @checkpoint=concat(@checkpoint_lsn,@end_lsn,repeat(x'00',44));
      set @checkpoint=concat(@checkpoint,unhex(hex(crc32c(@checkpoint))),
                             repeat(x'00',8128));
      select concat(@header,@checkpoint) into dumpfile 'header';
      

      Then, convert the log file to the innodb_log_archive=OFF format:

      mysql-test/mtr main.log
      tar xJf data.tar.gz -C ..
      mv ../data/ib_0000000000003000.log ../data/ib_logfile0
      dd conv=notrunc of=../data/ib_logfile0 if=mysql-test/var/mysqld.1/data/test/header
      

      Now an older server can be started on the data directory:

      sql/mariadbd --innodb-page-size=8k --datadir="$(pwd)/../data"
      

      Recovery will report the following error:

      10.11 d5813edd08b70302f3a009c7d2904e16996f6545

      2026-09-16 15:54:01 0 [Note] InnoDB: Some operating system error numbers are described at https://mariadb.com/docs/server/reference/error-codes/operating-system-error-codes
      2026-09-16 15:54:01 0 [ERROR] InnoDB: Cannot open datafile for read-only: './cool_down/t4.ibd' OS error: 71
      2026-09-16 15:54:01 0 [ERROR] InnoDB: Could not find a valid tablespace file for cool_down/t4. Please refer to https://mariadb.com/docs/server/server-usage/storage-engines/innodb/innodb-troubleshooting/innodb-data-dictionary-troubleshooting for how to resolve the issue.
      2026-09-16 15:54:01 0 [Warning] InnoDB: Ignoring tablespace for cool_down/t4 because it could not be opened.
      

      With my fix applied, the recovery will succeed, and CHECK TABLE will not report any errors on InnoDB tables:

      10.11 d5813edd08b70302f3a009c7d2904e16996f6545, patched

      2026-09-16 15:55:29 0 [Note] InnoDB: Initialized memory pressure event listener
      2026-09-16 15:55:29 0 [Note] InnoDB: Completed initialization of buffer pool
      2026-09-16 15:55:29 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=6631332
      2026-09-16 15:55:29 0 [Note] InnoDB: End of log at LSN=12122573
      2026-09-16 15:55:29 0 [ERROR] InnoDB: The change buffer is corrupted or has been removed on upgrade to MariaDB 11.0 or later
      2026-09-16 15:55:29 0 [Note] InnoDB: To recover: 384 pages
      2026-09-16 15:55:30 0 [Note] InnoDB: Last binlog file './mysql-bin.000001', position 3067629
      2026-09-16 15:55:30 0 [Note] InnoDB: Resizing redo log from 11.562MiB to 8.000GiB; LSN=12122573
      

      Attachments

        1. data-rename.tar.xz
          3.53 MB
          Marko Mäkelä
        2. data.tar.xz
          1.13 MB
          Marko Mäkelä

        Issue Links

          Activity

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Marko Mäkelä Marko Mäkelä
              Thirunarayanan Balathandayuthapani Thirunarayanan Balathandayuthapani
              Saahil Alam Saahil Alam
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 0d
                  0d
                  Remaining:
                  Remaining Estimate - 0d
                  0d
                  Logged:
                  Time Spent - 3d 1h
                  3d 1h

                  Git Integration

                    Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.