[MDEV-24300]  InnoDB: Corrupted page of datafile './ibdata1' could not be found in the doublewrite buffer. Created: 2020-11-27  Updated: 2021-03-16  Resolved: 2021-03-16

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.2, 10.3, 10.4
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Alice Sherepa Assignee: Thirunarayanan Balathandayuthapani
Resolution: Not a Bug Votes: 0
Labels: None


 Description   

repeatable on 10.2-10.4, but not on 10.5

./mtr encryption.tempfiles,cbc,innodb,none  encryption.innodb-checksum-algorithm  --mysqld=--innodb-thread-concurrency=32

encryption.tempfiles 'cbc,innodb,none'   [ pass ]    986
 
MTR's internal check of the test case 'encryption.tempfiles' failed.
This means that the test case does not preserve the state that existed
before the test case was executed.  Most likely the test case did not
do a proper clean-up. It could also be caused by the previous test run
by this thread, if the server wasn't restarted.
This is the diff of the states of the servers before and after the
test case was executed:
mysqltest: Logging to '/d1/git/10.4/mysql-test/var/tmp/check-mysqld_1.log'.
mysqltest: Results saved in '/d1/git/10.4/mysql-test/var/tmp/check-mysqld_1.result'.
mysqltest: Connecting to server localhost:16000 (socket /d1/git/10.4/mysql-test/var/tmp/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...
mysqltest: ... Connected.
mysqltest: Start processing test commands from './include/check-testcase.test' ...
mysqltest: ... Done processing test commands.
--- /d1/git/10.4/mysql-test/var/tmp/check-mysqld_1.result	2020-11-27 15:38:06.776000000 +0000
+++ /d1/git/10.4/mysql-test/var/tmp/check-mysqld_1.reject	2020-11-27 15:38:08.060000000 +0000
@@ -362,7 +362,7 @@
 INNODB_TABLE_LOCKS	ON
 INNODB_TEMP_DATA_FILE_PATH	ibtmp1:12M:autoextend
 INNODB_THREAD_CONCURRENCY	32
-INNODB_THREAD_SLEEP_DELAY	10000
+INNODB_THREAD_SLEEP_DELAY	4
 INNODB_TMPDIR	
 INNODB_TRX_PURGE_VIEW_UPDATE_ONLY_DEBUG	OFF
 INNODB_TRX_RSEG_N_SLOTS_DEBUG	0
 
mysqltest: Result length mismatch
 
not ok
 
encryption.innodb-checksum-algorithm '32k,cbc,innodb' [ fail ]
        Test ended at 2020-11-27 15:38:13
 
CURRENT_TEST: encryption.innodb-checksum-algorithm
mysqltest: At line 4: query 'SET @saved_file_per_table = @@global.innodb_file_per_table' failed: 1193: Unknown system variable 'innodb_file_per_table'
 
The result from queries just before the failure was:
SET @saved_file_per_table = @@global.innodb_file_per_table;

2020-11-27 15:38:11 0 [ERROR] InnoDB: Corrupted page [page id: space=0, page number=0] of datafile './ibdata1' could not be found in the doublewrite buffer.
2020-11-27 15:38:11 0 [ERROR] InnoDB: Plugin initialization aborted at srv0start.cc[1605] with error Data structure corruption
2020-11-27 15:38:12 0 [Note] InnoDB: Starting shutdown...
2020-11-27 15:38:12 0 [ERROR] Plugin 'InnoDB' init function returned error.
2020-11-27 15:38:12 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.



 Comments   
Comment by Thirunarayanan Balathandayuthapani [ 2021-03-16 ]

I think bootstrap failure happend when you run with --innodb_thread_concurrency=32.
The following error is:

2021-03-16 13:53:07 140318084118336 [ERROR] InnoDB: Cannot continue operation. ib_logfiles are too small for innodb_thread_concurrency=32. The combined size of ib_logfiles should be bigger than 200 kB * innodb_thread_concurrency. Please refer to https://mariadb.com/kb/en/library/innodb-system-variables/
2021-03-16 13:53:07 140318084118336 [ERROR] InnoDB: Database creation was aborted at srv0start.cc[1939] with error Generic error. You may need to delete the ibdata1 file before trying to start up again.
2021-03-16 13:53:07 140318084118336 [Note] InnoDB: Starting shutdown...
2021-03-16 13:53:08 140318084118336 [ERROR] Plugin 'InnoDB' init function returned error.
2021-03-16 13:53:08 140318084118336 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.

Comment by Thirunarayanan Balathandayuthapani [ 2021-03-16 ]

./mtr encryption.innodb-checksum-algorithm  --mysqld=--innodb-thread-concurrency=32

Above command is enough to repeat the failure. Problem happens even during bootstrap
itself due to insufficient innodb_log_file_size for the given innodb_thread_concurrency.

Error is:

2021-03-16 13:53:07 140318084118336 [ERROR] InnoDB: Cannot continue operation. ib_logfiles are too small for innodb_thread_concurrency=32. The combined size of ib_logfiles should be bigger than 200 kB * innodb_thread_concurrency. Please refer to https://mariadb.com/kb/en/library/innodb-system-variables/
2021-03-16 13:53:07 140318084118336 [ERROR] InnoDB: Database creation was aborted at srv0start.cc[1939] with error Generic error. You may need to delete the ibdata1 file before trying to start up again.
2021-03-16 13:53:07 140318084118336 [Note] InnoDB: Starting shutdown...
2021-03-16 13:53:08 140318084118336 [ERROR] Plugin 'InnoDB' init function returned error.
2021-03-16 13:53:08 140318084118336 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.

So ibdata1 file is not flushed due to this failure. While starting the server, InnoDB gives the error
about ibdata1 page0 is zero filled. I don't think this is a issue.

Generated at Thu Feb 08 09:28:56 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.