Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-29785

Duplicate gtid event in binlog

Details

    • Bug
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • 10.4.26
    • 10.5
    • Galera, Replication
    • None
    • RHEL7 derivate, 3 node galera cluster

    Description

      As the GTIDs drift during upgrade process of a galera node (mysql_upgrade --skip-write-binlog --system-tables) and to resynchonize the nodes I ran:

      SET SESSION gtid_seq_no=<a nice number above the current gtid>;
      		 
      INSERT INTO hurfdurf (a) VALUES (1);

      With 10.4.26 the gtid set event was recorded twice in the binlog causing binlog replication to stop. This is a new bug introduced after 10.4.21 as previous upgrades using the same method worked fine. 10.5.17 with set session wsrep_gtid_seq_no is not affected.

      Workaround:
      1. stop slave, switch to file+position after the duplicate event and start slave
      2. flush binlog at source as you can't continue from a file which has the duplicate event even if you're past the duplicate event itself.
      3. stop slave, flip back to gtid based replication, start slave

      Attachments

        Activity

          Ascending order - Click to sort in descending order
          ospi Jesse created issue -
          alice Alice Sherepa made changes -
          Field Original Value New Value
          Assignee Angelique Sklavounos [ JIRAUSER50741 ]
          angelique.sklavounos Angelique Sklavounos (Inactive) added a comment -

          Hi ospi

          Thank you for the report. I will try to reproduce, but could I please get more details on the replication topology of the setup? Also, what is show slave status output when replication stops?

          Thank you.

          angelique.sklavounos Angelique Sklavounos (Inactive) added a comment - Hi ospi Thank you for the report. I will try to reproduce, but could I please get more details on the replication topology of the setup? Also, what is show slave status output when replication stops? Thank you.
          angelique.sklavounos Angelique Sklavounos (Inactive) made changes -
          Status Open [ 1 ] Needs Feedback [ 10501 ]
          ospi Jesse added a comment -

          Actually this isn't galera related. More likely wsrep_gtid_mode related bug

          Steps to reproduce with podman
          /opt/MDEV-29785/server.cnf contains the following:

          [server]
          		 
          log-bin=1
          		 
          wsrep_gtid_mode=1
          		 
          server-id=777

          Podman steps

          podman run -dt --rm -v /opt/MDEV-29785:/etc/mysql/conf.d --name mariadb-master -e MYSQL_ROOT_PASSWORD=root mariadb:10.4
          		 
          podman run -dt --rm --name mariadb-slave -e MYSQL_ROOT_PASSWORD=root mariadb:10.4
          		 
           
          		 
          # Start slave
          		 
          change_master="
          		 
          CHANGE MASTER TO MASTER_HOST=\"$(podman inspect mariadb-master -f '{{ .NetworkSettings.IPAddress }}')\",
          		 
          MASTER_USER=\"root\",
          		 
          MASTER_PASSWORD=\"root\",
          		 
          MASTER_USE_GTID=slave_pos;
          		 
          START SLAVE;"
          		 
          podman exec -it mariadb-slave mysql -uroot -proot -e "$change_master"
          		 
           
          		 
          # Do something on master
          		 
          podman exec -it mariadb-master mysql -uroot -proot -e "create database derp"
          		 
          # verify replication is OK
          		 
          podman exec -it mariadb-slave mysql -uroot -proot -e "show databases"
          		 
           
          		 
          # This works : bump gtid seq no and create a database
          		 
          podman exec -it mariadb-master mysql -uroot -proot -e "SET SESSION gtid_seq_no=1000; CREATE DATABASE derp_also;"
          		 
           
          		 
          # This works : bump gtid seq no and create a table
          		 
          podman exec -it mariadb-master mysql -uroot -proot -e "SET SESSION gtid_seq_no=2000; CREATE TABLE derp.duh (id int unsigned auto_increment primary key);"
          		 
           
          		 
          # This doesn't work : bump gtid and INSERT
          		 
          podman exec -it mariadb-master mysql -uroot -proot -e "SET SESSION gtid_seq_no=3000; INSERT INTO derp.duh (id) VALUES (null);"
          		 
           
          		 
          # Slave stoppped
          		 
          podman exec -it mariadb-slave mysql -uroot -proot -e "show slave status \G"
          		 
          # Binlogs show duplicate event for GTID 0-777-3000 trans
          		 
          podman exec -it mariadb-master /bin/bash -c "/usr/bin/mysqlbinlog /var/lib/mysql/\$(tail -n1 /var/lib/mysql/1.index)"

          ospi Jesse added a comment - Actually this isn't galera related. More likely wsrep_gtid_mode related bug Steps to reproduce with podman /opt/ MDEV-29785 /server.cnf contains the following: [server] log-bin=1 wsrep_gtid_mode=1 server-id=777 Podman steps podman run -dt --rm -v /opt/MDEV-29785:/etc/mysql/conf.d --name mariadb-master -e MYSQL_ROOT_PASSWORD=root mariadb:10.4 podman run -dt --rm --name mariadb-slave -e MYSQL_ROOT_PASSWORD=root mariadb:10.4   # Start slave change_master=" CHANGE MASTER TO MASTER_HOST=\"$(podman inspect mariadb-master -f '{{ .NetworkSettings.IPAddress }}')\", MASTER_USER=\"root\", MASTER_PASSWORD=\"root\", MASTER_USE_GTID=slave_pos; START SLAVE;" podman exec -it mariadb-slave mysql -uroot -proot -e "$change_master"   # Do something on master podman exec -it mariadb-master mysql -uroot -proot -e "create database derp" # verify replication is OK podman exec -it mariadb-slave mysql -uroot -proot -e "show databases"   # This works : bump gtid seq no and create a database podman exec -it mariadb-master mysql -uroot -proot -e "SET SESSION gtid_seq_no=1000; CREATE DATABASE derp_also;"   # This works : bump gtid seq no and create a table podman exec -it mariadb-master mysql -uroot -proot -e "SET SESSION gtid_seq_no=2000; CREATE TABLE derp.duh (id int unsigned auto_increment primary key);"   # This doesn't work : bump gtid and INSERT podman exec -it mariadb-master mysql -uroot -proot -e "SET SESSION gtid_seq_no=3000; INSERT INTO derp.duh (id) VALUES (null);"   # Slave stoppped podman exec -it mariadb-slave mysql -uroot -proot -e "show slave status \G" # Binlogs show duplicate event for GTID 0-777-3000 trans podman exec -it mariadb-master /bin/bash -c "/usr/bin/mysqlbinlog /var/lib/mysql/\$(tail -n1 /var/lib/mysql/1.index)"
          angelique.sklavounos Angelique Sklavounos (Inactive) made changes -
          Status Needs Feedback [ 10501 ] Open [ 1 ]
          bnestere Brandon Nesterenko made changes -
          Assignee Angelique Sklavounos [ JIRAUSER50741 ] Brandon Nesterenko [ JIRAUSER48702 ]
          serg Sergei Golubchik made changes -
          Fix Version/s 10.5 [ 23123 ]

          People

            bnestere Brandon Nesterenko
            ospi Jesse
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.