Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-27524

Incorrect binlogs after Galera SST using rsync and mariabackup

    XMLWordPrintable

    Details

      Description

      In many scenarios, nodes crash after SST if the donor node uses multiple binlogs files and then SST is performed using rsync or mariabackup. Especially often the failure occurs after accessing binlogs from the joiner side. These failures are due to the fact that currently neither SST with rsync nor SST with mariabackup transfers all binlogs (not just the latest one), and the binlogs handling code in SST scripts is very simplistic and cannot correctly work in a number of configurations, for example, in configurations including absolute paths to binlogs in an index file, etc. An example of such a failure due to incorrect transmission of binlogs after SST:

      021-12-28  1:48:20 45 [Warning] Access denied for user 'testuser'@'127.0.0.1' (using password: YES)
      2021-12-28  2:11:01 0 [Note] InnoDB: Buffer pool(s) load completed at 211228  2:11:01
      2021-12-28  3:31:32 0 [Note] WSREP: Member 1.0 (ddmsmariadb03) desyncs itself from group
      2021-12-28  3:31:32 0 [Note] WSREP: Member 1.0 (ddmsmariadb03) resyncs itself to group.
      2021-12-28  3:31:32 0 [Note] WSREP: Member 1.0 (ddmsmariadb03) synced with group.
      2021-12-28  9:11:13 10 [ERROR] Error in Log_event::read_log_event(): 'Sanity check failed', data_len: 0, event_type: 0
      2021-12-28  9:11:13 10 [ERROR] WSREP: applier could not read binlog event, seqno: 196201, len: 18446744072749699206
      2021-12-28  9:11:13 0 [Note] WSREP: Member 1(ddmsmariadb03) initiates vote on 68ed27ac-671d-11ec-bed4-924ee3878f93:196201,8c41510586499f3b: 
      2021-12-28  9:11:13 0 [Note] WSREP: Votes over 68ed27ac-671d-11ec-bed4-924ee3878f93:196201:
         0000000000000000:   1/3
         8c41510586499f3b:   1/3
      Waiting for more votes.
      2021-12-28  9:11:13 5 [Note] WSREP: Got vote request for seqno 68ed27ac-671d-11ec-bed4-924ee3878f93:196201
      2021-12-28  9:11:13 0 [Note] WSREP: Member 0(ddmsmariadb01) responds to vote on 68ed27ac-671d-11ec-bed4-924ee3878f93:196201,0000000000000000: Success
      2021-12-28  9:11:13 0 [Warning] WSREP: Received bogus VOTE message: 196201.0, from node 52e83762-673c-11ec-b091-279e19995029, expected > 196203. Ignoring.
      2021-12-28  9:11:13 0 [Note] WSREP: Votes over 68ed27ac-671d-11ec-bed4-924ee3878f93:196201:
         0000000000000000:   1/3
         8c41510586499f3b:   1/3
      Waiting for more votes.
      2021-12-28  9:11:13 0 [Note] WSREP: Member 2(ddmsmariadb02) initiates vote on 68ed27ac-671d-11ec-bed4-924ee3878f93:196201,8c41510586499f3b: 
      2021-12-28  9:11:13 0 [Note] WSREP: Votes over 68ed27ac-671d-11ec-bed4-924ee3878f93:196201:
         0000000000000000:   1/3
         8c41510586499f3b:   2/3
      Winner: 8c41510586499f3b
      2021-12-28  9:11:13 10 [ERROR] WSREP: Failed to apply write set: gtid: 68ed27ac-671d-11ec-bed4-924ee3878f93:196201 server_id: 52e83762-673c-11ec-b091-279e19995029 client_id: 5863 trx_id: 6901439 flags: 3 (start_transaction | commit)
      2021-12-28  9:11:13 10 [Note] WSREP: Closing send monitor...
      2021-12-28  9:11:13 10 [Note] WSREP: Closed send monitor.
      2021-12-28  9:11:13 10 [Note] WSREP: gcomm: terminating thread
      2021-12-28  9:11:13 10 [Note] WSREP: gcomm: joining thread
      2021-12-28  9:11:13 10 [Note] WSREP: gcomm: closing backend
      2021-12-28  9:11:15 10 [Note] WSREP: view(view_id(NON_PRIM,52e83762-b091,3) memb {
      	6e8d8876-803a,0
      } joined {
      } left {
      } partitioned {
      	52e83762-b091,0
      	5ef9bf5e-bd37,0
      })
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              sysprg Julius Goryavsky
              Reporter:
              sysprg Julius Goryavsky
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.