Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-11131

InnoDB: Assertion failure - rem0rec.cc line 580

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Won't Fix
    • 10.1.14
    • N/A
    • Storage Engine - InnoDB
    • None
    • Ubuntu 14.04. 2x20 core processors.
      3 node Galera cluster. All read and write operations directed at one node.

    Description

      We received the below error on our production server, which caused the database server to crash. Despite being a 3 node cluster, we direct all read and write operations to one node, using the other 2 nodes as hot spares. Upon failure, all access redirected to one of the other nodes in the cluster with no issue.

      I have attached the syslog containing all the events that occured afterwards in case they are useful. Of note, it appears that the first restart of the server, had trouble synchronising because of an unexpected primary key. I suspect our on-call engineers then deleted the DB, forcing an SST. I have trimmed the log where it is copying and moving several files.

      Oct 21 15:51:58 MEDX-C1 mysqld: 2016-10-21 15:51:58 7f3905a40b00  InnoDB: Assertion failure in thread 139882884500224 in file rem0rec.cc line 580

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-26893 innodb assertion on startup - rem0rec:877

          • Major - Major loss of function.
          • Closed
          links to

          Web Link Bug#1589919 server crash with InnoDB: Assertion failure in thread ... in file rem0rec.cc line 580

          Activity

            Ascending order - Click to sort in descending order
            brendon Brendon Abbott added a comment -

            I had a hunt around and couldn't find any ticket with this specific error. However, I did find the following with Google.

            https://bugs.launchpad.net/percona-xtradb-cluster/+bug/1589919

            brendon Brendon Abbott added a comment - I had a hunt around and couldn't find any ticket with this specific error. However, I did find the following with Google. https://bugs.launchpad.net/percona-xtradb-cluster/+bug/1589919
            brendon Brendon Abbott added a comment -

            I should add that we are currently working with our engineering team to upgrade this production system to the latest stable release in the 10.1 series.

            brendon Brendon Abbott added a comment - I should add that we are currently working with our engineering team to upgrade this production system to the latest stable release in the 10.1 series.
            elenst Elena Stepanova added a comment -

            nirbhay_c, please take a look. The stack trace in the attached log is non-distinctive, but those in Percona's LP hint that the crash happens somewhere in wsrep_rec_get_foreign_key, so it might be related to the cluster after all.

            elenst Elena Stepanova added a comment - nirbhay_c , please take a look. The stack trace in the attached log is non-distinctive, but those in Percona's LP hint that the crash happens somewhere in wsrep_rec_get_foreign_key , so it might be related to the cluster after all.
            sruggier Simon Ruggier added a comment -

            I've observed this in production with MariaDB 10.0.24 on Debian Jessie, see the attachment I added for the information that was dumped at the time of the crash.

            sruggier Simon Ruggier added a comment - I've observed this in production with MariaDB 10.0.24 on Debian Jessie, see the attachment I added for the information that was dumped at the time of the crash.
            janlindstrom Jan Lindström added a comment -

            10.1 is EOL.

            janlindstrom Jan Lindström added a comment - 10.1 is EOL.

            People

              jplindst Jan Lindström (Inactive)
              brendon Brendon Abbott
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.