Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-24229

During Galera IST: Assertion failure in file lock0lock.cc at line 687

Details

    • Bug
    • Status: Closed (View Workflow)
    • Blocker
    • Resolution: Duplicate
    • 10.4.17
    • N/A
    • Galera
    • None

    Description

      After a manual SST with mariabackup the IST fails with a assertion failure at /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.4.17/storage/innobase/lock/lock0lock.cc line 687.

      We tried the manual SST + IST twice and the WSREP Error before the assertion concerned both times the same table and index with the conflicting lock and the same insert (with different values).

      Attachments

        Issue Links

          includes

          Bug - A problem which impairs or prevents the functions of the product. MDEV-23851 Galera assertion at lock0lock.cc line 655 because of BF-BF lock wait

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24406 Galera crashes when deleting records

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24437 Galera 4 read node crashes after DML statement from writer node

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-23851 Galera assertion at lock0lock.cc line 655 because of BF-BF lock wait

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24989 Galera assertion at lock0lock.cc line 655

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            View 7 older comments
            smiba Bart Smienk added a comment - - edited

            Same issue over here, currently downgrading to 10.3.25 (oldest available in the repos) from 10.3.27 at the moment to see if that resolves our issues.

            Upgraded a few weeks ago from 10.3.18 which did not present any issues, but the secondary cluster node now crashes multiple times per week. We're running a master/master setup (with garb as witness)

            Our cluster successfully does SST or IST, but just crashes at random when some locks conflict. This happened every now and then already in the past but never resulted in a crash of the cluster

            smiba Bart Smienk added a comment - - edited Same issue over here, currently downgrading to 10.3.25 (oldest available in the repos) from 10.3.27 at the moment to see if that resolves our issues. Upgraded a few weeks ago from 10.3.18 which did not present any issues, but the secondary cluster node now crashes multiple times per week. We're running a master/master setup (with garb as witness) Our cluster successfully does SST or IST, but just crashes at random when some locks conflict. This happened every now and then already in the past but never resulted in a crash of the cluster
            seppo Seppo Jaakola added a comment -

            Submitted two pull requests to fix this issue and MDEV-23851, which appear to be duplicates. Reason for the crashes is recently added too strict assertion, which fires in safe execution. This is fixed by the pull request: https://github.com/MariaDB/server/pull/1723.

            There is also a real bug resulting from a more rare scenario, where unique secondary key value is modified in certain way (often caused by REPLACE statement usage), resulting in secondary index gap locking and there happens simultaneous INSERT replicated from other node and inserting into the locked gap. This issue has a fix in pull request: https://github.com/MariaDB/server/pull/1724

            seppo Seppo Jaakola added a comment - Submitted two pull requests to fix this issue and MDEV-23851 , which appear to be duplicates. Reason for the crashes is recently added too strict assertion, which fires in safe execution. This is fixed by the pull request: https://github.com/MariaDB/server/pull/1723 . There is also a real bug resulting from a more rare scenario, where unique secondary key value is modified in certain way (often caused by REPLACE statement usage), resulting in secondary index gap locking and there happens simultaneous INSERT replicated from other node and inserting into the locked gap. This issue has a fix in pull request: https://github.com/MariaDB/server/pull/1724
            seppo Seppo Jaakola added a comment -

            two pull requests submitted for this:
            https://github.com/MariaDB/server/pull/1723
            https://github.com/MariaDB/server/pull/1724

            seppo Seppo Jaakola added a comment - two pull requests submitted for this: https://github.com/MariaDB/server/pull/1723 https://github.com/MariaDB/server/pull/1724
            smiba Bart Smienk added a comment -

            Thanks for your work Seppo, much appreciated.
            I'll keep an eye on future releases to see if your fix has been included, so we can update the cluster again to the latest release.

            smiba Bart Smienk added a comment - Thanks for your work Seppo, much appreciated. I'll keep an eye on future releases to see if your fix has been included, so we can update the cluster again to the latest release.
            jplindst Jan Lindström (Inactive) added a comment -

            Duplicate of MDEV-23851

            jplindst Jan Lindström (Inactive) added a comment - Duplicate of MDEV-23851

            People

              jplindst Jan Lindström (Inactive)
              glalex Alexander Glaser
              Votes:
              14 Vote for this issue
              Watchers:
              22 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.