[MDEV-18128] Simplify .ibd file creation Created: 2019-01-03  Updated: 2019-08-19  Resolved: 2019-08-16

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.2, 10.3, 10.4
Fix Version/s: 10.2.27, 10.3.18, 10.4.8

Type: Bug Priority: Major
Reporter: Marko Mäkelä Assignee: Thirunarayanan Balathandayuthapani
Resolution: Fixed Votes: 1
Labels: backup, encryption

Issue Links:
Duplicate
is duplicated by MDEV-19348 MariaBackup prepare fails with InnoDB... Closed
Relates
relates to MDEV-18129 Backup fails for encrypted tables: ma... Closed
relates to MDEV-18415 mariabackup.mdev-14447 test case fail... Closed
relates to MDEV-12112 corruption in encrypted table may be ... Closed
relates to MDEV-14398 When innodb_encryption_rotate_key_age... Closed
relates to MDEV-15671 innodb.recovery_shutdown failed in bu... Closed

 Description   

When InnoDB creates an .ibd file, it first writes a dummy page 0 with no information about encryption. This may prevent Mariabackup 10.2.20 or 10.2.21 from copying encrypted data files that were craeted shortly before backup started.

Maybe we should create a 0-byte .ibd file first, and then let page flushing write the first pages? Alternatively, preallocate a zero-filled file and then let page flushing do its job.

A valid page 0 must be written to the created files before writing any other pages. Otherwise, crash recovery cannot work.

 Comments   
Comment by Marko Mäkelä [ 2019-01-04 ]

It looks like crash recovery is broken because of this. Here is a test that initializes the database and kills&restarts it soon after that. It can occasionally fail:

10.2 23e4446adc0de8b6f763f746fd20f0e546a40129

 
innodb.group_commit_crash_no_optimize_thread 'innodb' w3 [ fail ]  Found warnings/errors in server log file!
 
       Test ended at 2019-01-03 16:54:22
 
line
 
2019-01-03 16:54:21 3069097920 [ERROR] InnoDB: Checksum mismatch in datafile: ./ibdata1, Space ID:0, Flags: 0. Please refer to http://dev.mysql.com/doc/refman/5.7/en/innodb-troubleshooting-datadict.html for how to resolve the issue.
 
^ Found warnings in /mnt/buildbot/build/mariadb-10.2.22/mysql-test/var/3/log/mysqld.1.err

Perhaps, in order to simplify things, we can keep the crash-unsafe creation of the InnoDB system tablespace that only happens when the database is initialized, and only fix the .ibd file creation. After all, I would at some point (MDEV-11633) make the InnoDB system tablespace optional (split its contents into more manageable pieces).

Comment by Marko Mäkelä [ 2019-08-16 ]

This looks OK. Unfortunately, we were unable to reproduce failures with mariabackup --backup --skip-encrypted-backup with or without the patch. The purpose of that option was to work around this bug. I think that we should remove that option in 10.5.

Comment by Marko Mäkelä [ 2019-08-16 ]

The commit message accidentally omitted the bug title.

Generated at Thu Feb 08 08:41:42 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.