[MDEV-30722] sporadic failure spider/bugfix.insert_select Created: 2023-02-24  Updated: 2023-11-24

Status: Open
Project: MariaDB Server
Component/s: Storage Engine - Spider
Affects Version/s: 10.6
Fix Version/s: 10.6

Type: Bug Priority: Major
Reporter: Yuchen Pei Assignee: Yuchen Pei
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Blocks
is blocked by MDEV-22979 "mysqld --bootstrap" / mysql_install_... Closed
is blocked by MDEV-29870 Backport fixes to spider init bugs to... Closed
Relates
relates to MDBF-535 Document / improve buildbot setup for... Closed

 Description   

Reported by marko.

https://buildbot.mariadb.org/#/builders/236/builds/8787/steps/7/logs/stdio
commit 96133583cc54f0da7524e4f657daf412e9a50209

spider/bugfix.insert_select              w3 [ fail ]
 
        Test ended at 2023-02-24 08:46:42
 
CURRENT_TEST: spider/bugfix.insert_select
 
Failed to start mysqld.2.1
 
mysqltest failed but provided no output
 
 - saving '/home/buildbot/amd64-ubuntu-2004-fulltest/build/mysql-test/var/3/log/spider/bugfix.insert_select/' to '/home/buildbot/amd64-ubuntu-2004-fulltest/build/mysql-test/var/log/bugfix.insert_select/'
 
Retrying test spider/bugfix.insert_select, attempt(2/3)...
 
worker[3] > Restart  - not started
 
worker[3] > Restart  - not started
 
***Warnings generated in error logs during shutdown after running tests: spider/bugfix.insert_select
 
2023-02-24  8:46:41 0 [Warning] InnoDB: Are you sure you are using the right ib_logfile0 to start up the database? Log sequence number in the ib_logfile0 is 42778, less than the log sequence number in the first system tablespace file header, 43654.
 
2023-02-24  8:46:41 0 [ERROR] InnoDB: Page [page id: space=0, page number=11] log sequence number 43412 is in the future! Current system log sequence number 42802.
 
2023-02-24  8:46:41 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
 
2023-02-24  8:46:41 0 [ERROR] InnoDB: Page [page id: space=0, page number=8] log sequence number 43466 is in the future! Current system log sequence number 42802.
 
2023-02-24  8:46:41 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
 
2023-02-24  8:46:41 0 [ERROR] InnoDB: Page [page id: space=0, page number=10] log sequence number 43574 is in the future! Current system log sequence number 42802.
 
2023-02-24  8:46:41 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
 
2023-02-24  8:46:41 0 [ERROR] InnoDB: Page [page id: space=0, page number=12] log sequence number 43300 is in the future! Current system log sequence number 42802.
 
2023-02-24  8:46:41 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
 
2023-02-24  8:46:41 0 [ERROR] InnoDB: Page [page id: space=0, page number=9] log sequence number 43459 is in the future! Current system log sequence number 42802.
 
2023-02-24  8:46:41 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
 
2023-02-24  8:46:41 0 [ERROR] InnoDB: Page [page id: space=1, page number=3] log sequence number 43452 is in the future! Current system log sequence number 42802.
 
2023-02-24  8:46:41 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
 
2023-02-24  8:46:41 0 [ERROR] InnoDB: Page [page id: space=2, page number=3] log sequence number 43546 is in the future! Current system log sequence number 42802.
 
2023-02-24  8:46:41 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
 
2023-02-24  8:46:41 0 [ERROR] mariadbd: Aria recovery failed. Please run aria_chk -r on all Aria tables (*.MAI) and delete all aria_log.######## files
 
2023-02-24  8:46:41 0 [ERROR] Plugin 'Aria' init function returned error.
 
2023-02-24  8:46:41 0 [ERROR] Plugin 'Aria' registration as a STORAGE ENGINE failed.
 
2023-02-24  8:46:41 0 [ERROR] Could not open mysql.plugin table: "Unknown storage engine 'Aria'". Some plugins may be not loaded
 
2023-02-24  8:46:41 0 [ERROR] Failed to initialize plugins.
 
2023-02-24  8:46:41 0 [ERROR] Aborting

marko said it is similar to MDEV-28976.

 Comments   
Comment by Yuchen Pei [ 2023-07-05 ]

Another instance on 10.4 https://buildbot.mariadb.net/buildbot/builders/kvm-rpm-centos74-amd64/builds/34151.

spider/bugfix.delete_with_float_column_mariadb w4 [ fail ]  Found warnings/errors in server log file!
 
        Test ended at 2023-06-28 18:54:09
 
line
 
2023-06-28 18:54:07 2 [ERROR] InnoDB: Page [page id: space=1, page number=3] log sequence number 68125 is in the future! Current system log sequence number 67906.
 
2023-06-28 18:54:07 2 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.

Given the case in the description exhibits an spider initialisation issue due to dependency on aria which will be fixed in MDEV-22979. I'll only come back to this ticket after the init bugs are fixed.

Comment by Yuchen Pei [ 2023-11-24 ]

Another instance, this time
spider/bugfix.slave_transaction_retry_errors_5digit
in amd64-opensuse-15 at 10.4 d96324905f5d770b7b422e04a48ab404a1e68cfd:

https://buildbot.mariadb.org/#/builders/159/builds/25871

spider/bugfix.slave_transaction_retry_errors_5digit w1 [ fail ]  Found warnings/errors in server log file!
 
        Test ended at 2023-11-24 03:42:48
 
line
 
2023-11-24  3:42:45 0 [ERROR] InnoDB: Page [page id: space=0, page number=11] log sequence number 67810 is in the future! Current system log sequence number 67789.
 
2023-11-24  3:42:45 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
 
2023-11-24  3:42:45 0 [ERROR] InnoDB: Page [page id: space=0, page number=8] log sequence number 68168 is in the future! Current system log sequence number 67789.
 
2023-11-24  3:42:45 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
 
2023-11-24  3:42:45 0 [ERROR] InnoDB: Page [page id: space=0, page number=10] log sequence number 68179 is in the future! Current system log sequence number 67789.
 
2023-11-24  3:42:45 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
 
2023-11-24  3:42:45 0 [ERROR] InnoDB: Page [page id: space=0, page number=12] log sequence number 68184 is in the future! Current system log sequence number 67789.
 
2023-11-24  3:42:45 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
 
2023-11-24  3:42:45 0 [ERROR] InnoDB: Page [page id: space=0, page number=307] log sequence number 68174 is in the future! Current system log sequence number 67789.
 
2023-11-24  3:42:45 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
 
2023-11-24  3:42:45 0 [ERROR] InnoDB: Page [page id: space=0, page number=308] log sequence number 67795 is in the future! Current system log sequence number 67789.
 
2023-11-24  3:42:45 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
 
2023-11-24  3:42:45 1 [ERROR] InnoDB: Page [page id: space=0, page number=9] log sequence number 68163 is in the future! Current system log sequence number 67789.
 
2023-11-24  3:42:45 1 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
 
2023-11-24  3:42:45 1 [ERROR] InnoDB: Page [page id: space=2, page number=3] log sequence number 68194 is in the future! Current system log sequence number 67789.
 
2023-11-24  3:42:45 1 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
 
2023-11-24  3:42:45 1 [ERROR] InnoDB: Page [page id: space=1, page number=3] log sequence number 68158 is in the future! Current system log sequence number 67789.
 
2023-11-24  3:42:45 1 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
 
^ Found warnings in /home/buildbot/amd64-opensuse-15/build/mysql-test/var/1/log/mysqld.2.1.err
 
ok

Generated at Thu Feb 08 10:18:24 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.