[MDEV-29485] MariaDB crashes with InnoDB: Assertion failure in file /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.6.8/storage/innobase/row/row0ins.cc line 2132 Created: 2022-09-07  Updated: 2024-01-16  Resolved: 2024-01-16

Status: Closed
Project: MariaDB Server
Component/s: Data Manipulation - Insert
Affects Version/s: 10.6.8
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: b2b-support-abd-others Assignee: Marko Mäkelä
Resolution: Incomplete Votes: 0
Labels: Crash
Environment:

VM with Red Hat Enterprise Linux 8.6 (Ootpa)


Attachments: Text File Bug report.log     Text File check table crash.log    

 Description   

Mariadb crashed reporting:

InnoDB: Assertion failure in file /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.6.8/storage/innobase/row/row0ins.cc line 2132
InnoDB: Failing assertion: cmp < 0



 Comments   
Comment by Marko Mäkelä [ 2022-09-07 ]

The assertion fails in row_ins_scan_sec_index_for_duplicate() because some records are in wrong order in an index. This crash was not fixed in MDEV-13542.

What does the following command report?

CHECK TABLE `files`;

Comment by b2b-support-abd-others [ 2022-09-07 ]

The check table caused a crash:

2022-09-07 14:22:52 1606324 [ERROR] InnoDB: Records in wrong order
2022-09-07 14:22:52 1606324 [ERROR] InnoDB: Corruption of an index tree: table `betdocgen`.`files` index `files_project_id_branch_path_unique`, father ptr page no 640, child page no 184
PHYSICAL RECORD: n_fields 4; compact format; info bits 0
2022-09-07 14:22:52 1606324 [ERROR] [FATAL] InnoDB: You should dump + drop + reimport the table to fix the corruption. If the crash happens at database startup. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery. Then dump + drop + reimport.
220907 14:22:52 [ERROR] mysqld got signal 6 ;

See attached log

Comment by Marko Mäkelä [ 2022-09-07 ]

Can you post also the output of the following:

SHOW CREATE TABLE `files`;

You may redact some of it, but please include at least the definitions of the PRIMARY KEY, the secondary index, and the columns that they depend on. You may also obfuscate column names. Make sure that the resulting CREATE TABLE statement can be executed.

Comment by b2b-support-abd-others [ 2022-09-07 ]

CREATE TABLE `files` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`project_id` int(11) NOT NULL,
`path` varchar(255) NOT NULL,
`description` varchar(255) NOT NULL,
`branch` varchar(255) NOT NULL,
`type` varchar(255) NOT NULL,
`parent` varchar(255) NOT NULL,
`priority` int(11) NOT NULL,
`confidentiality` int(11) NOT NULL,
`integrity` int(11) NOT NULL,
`availability` int(11) NOT NULL,
`accountability` int(11) NOT NULL,
`name` varchar(255) NOT NULL,
`hash` varchar(255) NOT NULL,
`blob_id` varchar(255) NOT NULL,
`commit_id` varchar(255) NOT NULL,
`last_commit_id` varchar(255) NOT NULL,
`created_at` timestamp NULL DEFAULT NULL,
`updated_at` timestamp NULL DEFAULT NULL,
`version` varchar(255) NOT NULL,
`owner` varchar(255) NOT NULL,
`overall` varchar(255) NOT NULL,
`active` tinyint(1) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `files_project_id_branch_path_unique` (`project_id`,`branch`,`path`),
KEY `files_project_id_path_index` (`project_id`,`path`)
) ENGINE=InnoDB AUTO_INCREMENT=15574 DEFAULT CHARSET=utf8mb4

Comment by Marko Mäkelä [ 2022-09-08 ]

Thank you. The affected index is a unique index that includes two VARCHAR(255) columns that are in the utf8mb4 encoding. I was suspecting that this would cause hidden indexed virtual columns to be created by MDEV-371, but that is not the case.

Can you provide SQL statements for reproducing this crash? This could be a bug in the collation implementation, only reproducible with some specific data. All other columns than id, project_id, branch, path should be irrelevant and can be omitted from the SQL.

Comment by b2b-support-abd-others [ 2022-09-20 ]

Hello Marko,

We had to implement the following steps:
Run a check table on all databases and all tables.
On each table found to have the problem, dropped the indexes and recreated them. Two tables had duplicates that had to be solved.

We exported the databases from a 10.1.23 with utf8, imported them in the new 10.6.8 and converted all databases, tables and columns to utf8mb4.
We encountered some conversion problems during the conversion, corrected the issues and restarted the conversion. The four tables causing the crash problems did not show any issues during conversion

Comment by Marko Mäkelä [ 2023-12-14 ]

A possible source of the corruption could be the InnoDB change buffer, for example, due to MDEV-24449, MDEV-24709, or MDEV-32132. It was disabled by default in MDEV-27734.

Has MariaDB Server 10.6 worked well for you? We had some regressions there, such as MDEV-30531.

Generated at Thu Feb 08 10:08:57 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.