[MDEV-28450] 0x7f5a60da6700 InnoDB: Assertion failure rem0rec.cc line 580 Created: 2022-05-02  Updated: 2022-05-06  Resolved: 2022-05-06

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.2.10
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Luke Assignee: Unassigned
Resolution: Cannot Reproduce Votes: 0
Labels: None
Environment:

CentOS release 6.10 (Final)


 Description   

We use MariaDB 10.2.10 in our QA and Production environments.
And I find one of our QA instances crashes frequently and the log error saying:

2022-05-02 01:31:23 0x7f5a60da6700  InnoDB: Assertion failure in file /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.2.10/storage/innobase/rem/rem0rec.cc line 580
 
InnoDB: We intentionally generate a memory trap.
 
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
 
InnoDB: If you get repeated assertion failures or crashes, even
 
InnoDB: immediately after the mysqld startup, there may be
 
InnoDB: corruption in the InnoDB tablespace. Please refer to
 
InnoDB: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html
 
InnoDB: about forcing recovery.
 
220502 01:31:32 mysqld_safe Number of processes running now: 0
 
220502 01:31:32 mysqld_safe mysqld restarted
 
2022-05-02  1:31:32 140437669955584 [Warning] option 'binlog_cache_size': unsigned value 0 adjusted to 4096
 
2022-05-02  1:31:32 140437669955584 [Note] /usr/sbin/mysqld (mysqld 10.2.10-MariaDB-log) starting as process 43423 ...

The only indicative info is:

0x7f5a60da6700  InnoDB: Assertion failure in file /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.2.10/storage/innobase/rem/rem0rec.cc line 580

And I also find that each time I run this query, the bug is triggered:

SELECT `lpbu_client_id`,`lpbu_pca_id`,`publisher_object_id`,`landing_page`,`mobile_landing_page` FROM (SELECT       act.client_id AS lpbu_client_id,       pca.client_account_id AS lpbu_pca_id,       lpbu.linked_vo_id AS publisher_object_id,       GROUP_CONCAT(CASE lpbu.is_mobile WHEN 0 THEN lpbu.lp_base_url ELSE null END SEPARATOR " ; ") as landing_page,       GROUP_CONCAT(CASE lpbu.is_mobile WHEN 1 THEN lpbu.lp_base_url ELSE null END SEPARATOR " ; ") as mobile_landing_page        FROM keyword_instances leaf       INNER JOIN lp_base_urls lpbu ON (lpbu.vo_type = 1 AND lpbu.linked_vo_id = leaf.id)       LEFT JOIN publisher_groups grop USING (publisher_group_id)       LEFT JOIN publisher_campaigns cpgn USING(publisher_campaign_id)       LEFT JOIN publisher_client_accounts pca USING(client_account_id)       LEFT JOIN publisher_accounts act USING(publisher_account_id)       WHERE pca.client_account_id = 123456 AND pca.publisher_client_account_operation_status NOT IN ('UNLINK','UNLINKED')       GROUP BY leaf.id       ) as tbl;
 
ERROR 2013 (HY000): Lost connection to MySQL server during query

I search. the internet and find there are some same cases but none of them are solved. Even in the MariaDB jira system, I also find a matched one: https://jira.mariadb.org/browse/MDEV-11131
But it's still open.

If you need other info, feel free to contact me. Thanks.

BTW: the error log says it generated a memory trap, but I don't know where it is. Is it useful for you to analysis this bug? Thanks.

 Comments   
Comment by Daniel Black [ 2022-05-02 ]

Hi lujinke,

Thanks for the bug report. The line of the fault indicates some form of index corruption. I don't think I can do anything with this. The cause may have been anytime in the past. The 10.2 release series has a long series of updates in the last 4.5 years, and is about to get its last release this year and is pretty much finalized.

If you are able to take a mysqldump of the tables in this query and import them into the latest 10.3 series and retry the query. If there's still a crash, a full stack trace might form the beginning of something we can actually fix.

I can only recommend recreating the indexes on the tables of this query on your QA server might mitigate this problem until what condition corrupted the index arises again.

Note, we've stopped releasing on Centos6 as its end of life.

Comment by Luke [ 2022-05-06 ]

@Daniel Black Thanks for your quick response, I try to recreate this table(rebuild index can not be achieved as it happened to be the primary key corruption), and now it's OK now.

Comment by Daniel Black [ 2022-05-06 ]

Glad to hear. Happy to take bug reports in future.

Generated at Thu Feb 08 10:00:50 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.