Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-41152

Recovery may fail if a file covered by FILE_CREATE is missing

    XMLWordPrintable

Details

    • Can result in data loss

    Description

      saahil reported a recovery failure while testing MDEV-14992. I can reproduce the failure with a copy of the data set, data.tar.xz, on a branch that supports innodb_log_archive (MDEV-37949):

      tar xJf data.tar.xz
      		 
      mariadbd --innodb-page-size=8k --innodb-log-recovery-start=6699128 --datadir="$PWD"/data

      The log can be converted to the 10.11 format by following steps similar to MDEV-40728. First, create a log file header by executing the following SQL:

      mysql-test/main/log.test

      		 
      set @first_lsn=x'0000000000003000';
      		 
      set @checkpoint_lsn=x'0000000000652fa4';
      		 
      set @end_lsn=x'0000000000663878';
      		 
      set @header=concat('Phys',x'00000000',@first_lsn,repeat(x'00',492));
      		 
      set @header=concat(@header,unhex(hex(crc32c(@header))),repeat(x'00',3584));
      		 
      set @checkpoint=concat(@checkpoint_lsn,@end_lsn,repeat(x'00',44));
      		 
      set @checkpoint=concat(@checkpoint,unhex(hex(crc32c(@checkpoint))),
      		 
                             repeat(x'00',8128));
      		 
      select concat(@header,@checkpoint) into dumpfile 'header';

      Then, convert the log file to the innodb_log_archive=OFF format:

      mysql-test/mtr main.log
      		 
      tar xJf data.tar.gz -C ..
      		 
      mv ../data/ib_0000000000003000.log ../data/ib_logfile0
      		 
      dd conv=notrunc of=../data/ib_logfile0 if=mysql-test/var/mysqld.1/data/test/header

      Now an older server can be started on the data directory:

      sql/mariadbd --innodb-page-size=8k --datadir="$(pwd)/../data"

      Recovery will report the following error:

      10.11 d5813edd08b70302f3a009c7d2904e16996f6545

      		 
      2026-09-16 15:54:01 0 [Note] InnoDB: Some operating system error numbers are described at https://mariadb.com/docs/server/reference/error-codes/operating-system-error-codes
      		 
      2026-09-16 15:54:01 0 [ERROR] InnoDB: Cannot open datafile for read-only: './cool_down/t4.ibd' OS error: 71
      		 
      2026-09-16 15:54:01 0 [ERROR] InnoDB: Could not find a valid tablespace file for cool_down/t4. Please refer to https://mariadb.com/docs/server/server-usage/storage-engines/innodb/innodb-troubleshooting/innodb-data-dictionary-troubleshooting for how to resolve the issue.
      		 
      2026-09-16 15:54:01 0 [Warning] InnoDB: Ignoring tablespace for cool_down/t4 because it could not be opened.

      With my fix applied, the recovery will succeed, and CHECK TABLE will not report any errors on InnoDB tables:

      10.11 d5813edd08b70302f3a009c7d2904e16996f6545, patched

      		 
      2026-09-16 15:55:29 0 [Note] InnoDB: Initialized memory pressure event listener
      		 
      2026-09-16 15:55:29 0 [Note] InnoDB: Completed initialization of buffer pool
      		 
      2026-09-16 15:55:29 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=6631332
      		 
      2026-09-16 15:55:29 0 [Note] InnoDB: End of log at LSN=12122573
      		 
      2026-09-16 15:55:29 0 [ERROR] InnoDB: The change buffer is corrupted or has been removed on upgrade to MariaDB 11.0 or later
      		 
      2026-09-16 15:55:29 0 [Note] InnoDB: To recover: 384 pages
      		 
      2026-09-16 15:55:30 0 [Note] InnoDB: Last binlog file './mysql-bin.000001', position 3067629
      		 
      2026-09-16 15:55:30 0 [Note] InnoDB: Resizing redo log from 11.562MiB to 8.000GiB; LSN=12122573

      Attachments

        1. data.tar.xz
          1.13 MB
          Marko Mäkelä
        2. data-rename.tar.xz
          3.53 MB
          Marko Mäkelä

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-41166 mariadb-backup --backup --innodb-log-checkpoint-now may copy too much log

          • Major - Major loss of function.
          • In Review

          Bug - A problem which impairs or prevents the functions of the product. MDEV-41187 deferred_spaces is redundant

          • Major - Major loss of function.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. MDEV-14992 BACKUP SERVER to mounted file system

          • Critical - Crashes, loss of data, severe memory leak.
          • In Progress

          New Feature - A new feature of the product, which has yet to be developed. MDEV-37949 Implement innodb_log_archive

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-40728 Recovery wrongly fails if FILE_CREATE is followed by FILE_RENAME

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              thiru Thirunarayanan Balathandayuthapani
              marko Marko Mäkelä
              Marko Mäkelä Marko Mäkelä
              Thirunarayanan Balathandayuthapani Thirunarayanan Balathandayuthapani
              Saahil Alam Saahil Alam
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - 0d
                  0d
                  Remaining:
                  Remaining Estimate - 0d
                  0d
                  Logged:
                  Time Spent - 2d 6.25h
                  2d 6.25h

                  Git Integration

                    Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.