Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-20413

Mariadb Galera Cluster Crash and won't restart

Details

    Description

      Our Galera Cluster crash (3 nodes successively) with the following error on the first node

      All nodes are unable to restart.

      We tried to bootstrap the cluster whith mysqld --wsrep-new-cluster without success

      We also tried to bootstrap the cluster whith mysqld --wsrep-new-cluster --innodb-force-recovery=from 1 to 4 without success

      on one node we tried --innodb-force-recovery=6 without success too

      Attachments

        1. node_innodb-force-recovery-6.txt
          12 kB
        2. node_wsrep-new-cluster.txt
          9 kB
        3. node1_crash.txt
          9 kB
        4. node2_crash.txt
          17 kB
        5. node3_crash.txt
          12 kB

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-19743 Crash while reorganizing an index page

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-19783 Random crashes and corrupt data in INSTANT-added columns

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-19916 Corruption after instant ADD/DROP and shrinking the index tree

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            vialink_syst Patrice CARMANT added a comment -

            We tried to update one node to 10.4.7 but galera won't start anymore

            vialink_syst Patrice CARMANT added a comment - We tried to update one node to 10.4.7 but galera won't start anymore
            elenst Elena Stepanova added a comment - 

            ERROR] [FATAL] InnoDB: Rec offset 99, cur1 offset 10363, cur2 offset 16179

            elenst Elena Stepanova added a comment - ERROR] [FATAL] InnoDB: Rec offset 99, cur1 offset 10363, cur2 offset 16179
            marko Marko Mäkelä added a comment -

            vialink_syst, did you upgrade to MariaDB 10.3.17 from some earlier 10.3 version?

            If yes, the table could have been corrupted while running an earlier version. I suspect that you could have been the victim of MDEV-19916 (corruption of the table after using instant ADD COLUMN).

            marko Marko Mäkelä added a comment - vialink_syst , did you upgrade to MariaDB 10.3.17 from some earlier 10.3 version? If yes, the table could have been corrupted while running an earlier version. I suspect that you could have been the victim of MDEV-19916 (corruption of the table after using instant ADD COLUMN ).

            People

              marko Marko Mäkelä
              vialink_syst Patrice CARMANT
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.