[MDEV-11131] InnoDB: Assertion failure - rem0rec.cc line 580 Created: 2016-10-25  Updated: 2023-04-12  Resolved: 2023-04-11

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.1.14
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Brendon Abbott Assignee: Jan Lindström (Inactive)
Resolution: Won't Fix Votes: 1
Labels: None
Environment:

Ubuntu 14.04. 2x20 core processors.
3 node Galera cluster. All read and write operations directed at one node.

Attachments: File mysql-error.log     HTML File mysql-syslog    
Issue Links:
Relates
relates to MDEV-26893 innodb assertion on startup - rem0rec... Closed

 Description   

We received the below error on our production server, which caused the database server to crash. Despite being a 3 node cluster, we direct all read and write operations to one node, using the other 2 nodes as hot spares. Upon failure, all access redirected to one of the other nodes in the cluster with no issue.

I have attached the syslog containing all the events that occured afterwards in case they are useful. Of note, it appears that the first restart of the server, had trouble synchronising because of an unexpected primary key. I suspect our on-call engineers then deleted the DB, forcing an SST. I have trimmed the log where it is copying and moving several files.

Oct 21 15:51:58 MEDX-C1 mysqld: 2016-10-21 15:51:58 7f3905a40b00  InnoDB: Assertion failure in thread 139882884500224 in file rem0rec.cc line 580



 Comments   
Comment by Brendon Abbott [ 2016-10-25 ]

I had a hunt around and couldn't find any ticket with this specific error. However, I did find the following with Google.

https://bugs.launchpad.net/percona-xtradb-cluster/+bug/1589919

Comment by Brendon Abbott [ 2016-10-25 ]

I should add that we are currently working with our engineering team to upgrade this production system to the latest stable release in the 10.1 series.

Comment by Elena Stepanova [ 2016-10-25 ]

nirbhay_c, please take a look. The stack trace in the attached log is non-distinctive, but those in Percona's LP hint that the crash happens somewhere in wsrep_rec_get_foreign_key, so it might be related to the cluster after all.

Comment by Simon Ruggier [ 2017-11-27 ]

I've observed this in production with MariaDB 10.0.24 on Debian Jessie, see the attachment I added for the information that was dumped at the time of the crash.

Comment by Jan Lindström [ 2023-04-11 ]

10.1 is EOL.

Generated at Thu Feb 08 07:47:33 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.