Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-35294

MariaDB Galera cluster encountered severe inconsistency issues when handling data synchronization

Details

    • Bug
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • 10.6.19
    • 10.6
    • wsrep
    • Ubuntu 22.04

    Description

      The MariaDB Galera cluster was experiencing severe errors and inconsistencies when processing DML operations and consistency checks. The following is a detailed analysis:
      1. Record not found error
      2024-10-27 6:33:04 2 [ERROR] mariadbd: Can't find record in 'test8'
      2024-10-27 6:33:04 2 [Warning] WSREP: Ignoring error 'Can't find record in 'test8'' on Delete_rows_v1 event. Error_code: 1032
      2. Duplicate entry error
      2024-10-27 6:33:22 2 [ERROR] Slave SQL: Could not execute Update_rows_v1 event on table mytest1.test8; Duplicate entry '75.41' for key 'column3', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 204, Internal MariaDB error code: 1062
      2024-10-27 6:33:22 2 [Warning] WSREP: Event 3 Update_rows_v1 apply failed: 121, seqno 39957
      3. WSREP consistency voting and inconsistency detection
      2024-10-27 6:33:22 0 [Note] WSREP: Member 0(node0) initiates vote on a9e687f9-9421-11ef-8f3c-322ad9279b98:39957,e813fdab298b3dec: Duplicate entry '75.41' for key 'column3', Error_code: 1062;
      2024-10-27 6:33:22 0 [Note] WSREP: Votes over a9e687f9-9421-11ef-8f3c-322ad9279b98:39957:
      e813fdab298b3dec: 1/4
      Waiting for more votes.
      2024-10-27 6:33:22 0 [Note] WSREP: Member 2(node2) responds to vote on a9e687f9-9421-11ef-8f3c-322ad9279b98:39957,0000000000000000: Success
      2024-10-27 6:33:22 0 [Note] WSREP: Votes over a9e687f9-9421-11ef-8f3c-322ad9279b98:39957:
      0000000000000000: 1/4
      e813fdab298b3dec: 1/4
      Waiting for more votes.
      2024-10-27 6:33:22 0 [Note] WSREP: Member 3(node4) responds to vote on a9e687f9-9421-11ef-8f3c-322ad9279b98:39957,0000000000000000: Success
      2024-10-27 6:33:22 0 [Note] WSREP: Votes over a9e687f9-9421-11ef-8f3c-322ad9279b98:39957:
      0000000000000000: 2/4
      e813fdab298b3dec: 1/4
      Winner: 0000000000000000
      2024-10-27 6:33:22 2 [ERROR] WSREP: Inconsistency detected: Inconsistent by consensus on a9e687f9-9421-11ef-8f3c-322ad9279b98:39957
      at ./galera/src/replicator_smm.cpp:process_apply_error():1343
      2024-10-27 6:33:22 2 [Note] WSREP: Closing send monitor...
      2024-10-27 6:33:22 2 [Note] WSREP: Closed send monitor.
      2024-10-27 6:33:22 2 [Note] WSREP: gcomm: terminating thread
      2024-10-27 6:33:22 2 [Note] WSREP: gcomm: joining thread
      2024-10-27 6:33:22 2 [Note] WSREP: gcomm: closing backend

      Attachments

        1. maradb_0.7z
          913 kB
        2. maradb_1.7z
          613 kB
        3. maradb_2.7z
          709 kB
        4. maradb_3.7z
          651 kB
        5. maradb_4.7z
          716 kB

        Activity

          ralf.gebhardt Ralf Gebhardt added a comment -

          janlindstrom, please check if you see something critical in this bug report

          ralf.gebhardt Ralf Gebhardt added a comment - janlindstrom , please check if you see something critical in this bug report

          CunDi Fang Can you provide more error log and show create table test8. If you can reproduce this and MDEV-35293 please provide steps to reproduce.

          janlindstrom Jan Lindström added a comment - CunDi Fang Can you provide more error log and show create table test8. If you can reproduce this and MDEV-35293 please provide steps to reproduce.
          CunDi Fang CunDi Fang added a comment -

          maradb_0.7z maradb_1.7z maradb_2.7z maradb_3.7z maradb_4.7z
          Okay, sorry it's a little late, there's been some delays looking for these files.I found the error while running on a 5 node mariadb cluster, this is the log file and all the files inside the data folder at that time.

          CunDi Fang CunDi Fang added a comment - maradb_0.7z maradb_1.7z maradb_2.7z maradb_3.7z maradb_4.7z Okay, sorry it's a little late, there's been some delays looking for these files.I found the error while running on a 5 node mariadb cluster, this is the log file and all the files inside the data folder at that time.

          People

            janlindstrom Jan Lindström
            CunDi Fang CunDi Fang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.