[MDEV-26640] InnoDB: Assertion failure in file /wrkdirs/usr/ports/databases/mariadb105-server/work/mariadb-10.5.12/storage/innobase/btr/btr0pcur.cc line 520 Created: 2021-09-19  Updated: 2021-09-21  Resolved: 2021-09-21

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.5.12
Fix Version/s: 10.2.41, 10.3.32, 10.4.22, 10.5.13, 10.6.5

Type: Bug Priority: Critical
Reporter: Masachika ISHIZUKA Assignee: Marko Mäkelä
Resolution: Duplicate Votes: 0
Labels: corruption, need_feedback
Environment:

FreeBSD 13.0-RELEASE-p4
mariadb 10.5.12 (mariadb105-server-10.5.12_1)


Issue Links:
Relates
relates to MDEV-26537 InnoDB corrupts files due to incorrec... Closed

 Description   

I cannot access table disk.
mysqldump was stopped intermidiate of dumpping table disk.
table disk has about 1500 records.

> desc disk;
+---------+---------------------+------+-----+---------+-------+
| Field   | Type                | Null | Key | Default | Extra |
+---------+---------------------+------+-----+---------+-------+
| label   | varchar(10)         | NO   | PRI | NULL    |       |
| size    | bigint(20) unsigned | NO   |     | NULL    |       |
| re      | tinyint(3) unsigned | NO   |     | NULL    |       |
| unavail | bigint(20) unsigned | YES  |     | NULL    |       |
+---------+---------------------+------+-----+---------+-------+
4 rows in set (0.01 sec)
> select count(*) from disk;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id:    4
Current database: bdr
 
ERROR 2013 (HY000): Lost connection to MySQL server during query

% cat /var/log/mysql/mysqld.err
..[snip]..
2021-09-19 13:25:28 0 [Note] /usr/local/libexec/mariadbd: ready for connections.
Version: '10.5.12-MariaDB-log'  socket: '/var/run/mysql/mysql.sock'  port: 3306  FreeBSD Ports
2021-09-19 13:25:28 0 [Note] InnoDB: Buffer pool(s) load completed at 210919 13:25:28
2021-09-19 13:25:32 0x81f27bf00  InnoDB: Assertion failure in file /wrkdirs/usr/ports/databases/mariadb105-server/work/mariadb-10.5.12/storage/innobase/btr/btr0pcur.cc line 520
InnoDB: Failing assertion: page_is_comp(next_page) == page_is_comp(page)
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
InnoDB: about forcing recovery.
210919 13:25:32 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 10.5.12-MariaDB-log
key_buffer_size=134217728
read_buffer_size=131072
max_used_connections=1
max_threads=153
thread_count=2
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467792 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x81fc0fb98
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7fffded02f38 thread_stack 0x49000
0x130cd6c <my_print_stacktrace+0x3c> at /usr/local/libexec/mariadbd
0xc6e6af <handle_fatal_signal+0x28f> at /usr/local/libexec/mariadbd
0x801904e00 <_pthread_sigmask+0x530> at /lib/libthr.so.3
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x81fc64d30): select count(*) from disk
 
Connection ID (thread ID): 4
Status: NOT_KILLED
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off
 
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
information that should help you find out what is causing the crash.
Core pattern: %N.core
2021-09-19 13:25:33 0 [Note] InnoDB: Uses event mutexes
2021-09-19 13:25:33 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-09-19 13:25:33 0 [Note] InnoDB: Number of pools: 1
2021-09-19 13:25:33 0 [Note] InnoDB: Using generic crc32 instructions
2021-09-19 13:25:33 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
2021-09-19 13:25:33 0 [Note] InnoDB: Completed initialization of buffer pool
2021-09-19 13:25:33 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=17580653,17580653
2021-09-19 13:25:33 0 [Note] InnoDB: Last binlog file '/var/db/mysql/log/mysql-bin.000011', position 171168
2021-09-19 13:25:33 0 [Note] InnoDB: 128 rollback segments are active.
2021-09-19 13:25:33 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2021-09-19 13:25:33 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2021-09-19 13:25:33 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2021-09-19 13:25:33 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2021-09-19 13:25:33 0 [Note] InnoDB: 10.5.12 started; log sequence number 17580737; transaction id 23194
2021-09-19 13:25:33 0 [Note] InnoDB: Loading buffer pool(s) from /var/db/mysql/ib_buffer_pool
2021-09-19 13:25:33 0 [Note] Plugin 'FEEDBACK' is disabled.
2021-09-19 13:25:33 0 [Note] Recovering after a crash using /var/db/mysql/log/mysql-bin
2021-09-19 13:25:33 0 [Note] Starting crash recovery...
2021-09-19 13:25:33 0 [Note] Crash recovery finished.
2021-09-19 13:25:33 0 [Note] Server socket created on IP: '0.0.0.0'.
2021-09-19 13:25:33 0 [Note] Reading of all Master_info entries succeeded
2021-09-19 13:25:33 0 [Note] Added new Master_info '' to hash table
2021-09-19 13:25:33 0 [Note] /usr/local/libexec/mariadbd: ready for connections.
Version: '10.5.12-MariaDB-log'  socket: '/var/run/mysql/mysql.sock'  port: 3306  FreeBSD Ports
2021-09-19 13:25:33 0 [Note] InnoDB: Buffer pool(s) load completed at 210919 13:25:33



 Comments   
Comment by Marko Mäkelä [ 2021-09-20 ]

If I have understood correctly, the FreeBSD package mariadb105-server-10.5.12_1 differs from an earlier 10.5.12 package in that it should include a fix of the corruption bug MDEV-26537.

Due to bug MDEV-26537, any InnoDB data files that were extended or created could become corrupted.

The corruption would not be magically fixed by switching to a release that does not include the erroneous code.

Did you use the FreeBSD mariadb-105-server-10.5.12 package earlier? It may have corrupted your data.

Comment by Masachika ISHIZUKA [ 2021-09-21 ]

Thank you, Marko.

I updated to 10.5.12_1 (with MDEV-26537 patch) on Sep 16, and the database corruption was occured before Sep 16.
So, that might be happened by MDEV-26537.
Thank you very much.

Comment by Marko Mäkelä [ 2021-09-21 ]

Thank you, ish. I will close this as a duplicate of MDEV-26537.

Once again, I am sorry for introducing this regression and causing data corruption. The documentation of st_blksize was confusing or unclear to me. No problems were observed on Linux. MariaDB itself does not build any executables for FreeBSD.

Comment by Masachika ISHIZUKA [ 2021-09-21 ]

I'm sorry for missing to find MDEV-26537.
On my environment, FreeBSD zfs might yield this bug.
Thank you for detailed explanation.

Generated at Thu Feb 08 09:46:50 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.