Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-33514

Getting "WSREP: RecordSet checksum does not match when" error when starting 2nd node in galera while the only one is running

    XMLWordPrintable

Details

    • Bug
    • Status: In Testing (View Workflow)
    • Critical
    • Resolution: Unresolved
    • 10.11
    • 10.11
    • Galera
    • The user is using the updated mariadb-backup from bb-10.11-monty

    Description

      This is reported by a MariaDB user:

      We have highly loaded stand with MariaDB 10.11.7 in configuration 3 multi-master Galera nodes.
      We was checking how data synchronization is working if we stop 2/3 nodes (only 1st node is online), then cleaned-up /var/lib/mysql/ in 2nd and 3rd nodes to provoke SST, then started 2nd node.
      After the Joining node (2nd) completes SST it fails to get synchronized with Donor (1st online node) with IST.

      From the log:

      2024-02-20 17:50:40 3 [Note] WSREP: Recovered view from SST:
      id: dc32fd00-903d-11ee-85ad-5bead3d487be:6109657999
      status: primary
      protocol_version: 4
      capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
      final: no
      own_index: 0
      members(2):
      0: 3a44102f-cfff-11ee-a7ca-1e0888e4626a, xxx-vm02
      1: 748569a1-cf3e-11ee-9cff-b36126e8e923, xxx-vm01
      ....
      Lots of rows:
      2024-02-20 17:50:40 46 [Note] WSREP: Starting applier thread 46
      ...
      2024-02-20 17:50:40 0 [Note] InnoDB: Rolled back recovered transaction 138599956425
      2024-02-20 17:50:40 38 [Note] WSREP: Starting applier thread 38
      2024-02-20 17:50:40 65 [Note] WSREP: Starting applier thread 65
      2024-02-20 17:50:40 69 [Note] WSREP: Starting applier thread 69
      2024-02-20 17:50:40 0 [ERROR] WSREP: RecordSet checksum does not match:
      computed: 4a533acb bf185da8 24de7515 d69efc6f
      found: c3dc9002 058f7e28 4bf63484 b3a4a111: 22 (Invalid argument)
      at /home/buildbot/buildbot/build/galerautils/src/gu_rset.cpp:checksum():554
      2024-02-20 17:50:40 0 [ERROR] WSREP: Writeset deserialization failed: Writeset checksum failed: 22 (Invalid argument)
      at /home/buildbot/buildbot/build/galera/src/write_set_ng.hpp:checksum_fin():865
      at /home/buildbot/buildbot/build/galera/src/trx_handle.hpp:unserialize():541
      WS flags: 0
      Trx proto: 5
      Trx source: 00000000-0000-0000-0000-000000000000
      Trx conn_id: 18446744073709551615
      Trx trx_id: 18446744073709551615
      Trx last_seen: -1
      2024-02-20 17:50:40 0 [ERROR] WSREP: got exception while reading IST stream: Writeset checksum failed: 22 (Invalid argument)

      Important note: This issue doesn't happen on the same environment if we stop application load. It happens only under load

      Attachments

        Issue Links

          Activity

            People

              Yurchenko Alexey
              monty Michael Widenius
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.