Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-8458

Galera Cluster replication stream doesn't pass along MariaDB's GTID

Details

    Description

      If a MariaDB Galera Cluster 10.0 cluster is configured as a slave of MariaDB 10.0 using GTID, GTIDs from the master do not propagate to the WSREP replication stream. This makes it difficult to use master_use_gtid=current_pos if you ever need to switch slaves, since the non-slave nodes in the cluster won't be aware of the master's GTIDs.

      Let's say we have a replication topology like this:

      | MariaDB 10.0 | --- master_use_gtid=current_pos ---> | MariaDB Galera Cluster 10.0 (galera1) | <--- WSREP ---> | MariaDB Galera Cluster 10.0 (galera2) |

      On the master:

      MariaDB [tmp]> show global variables like 'server_id';
      +---------------+-------+
      | Variable_name | Value |
      +---------------+-------+
      | server_id     | 2     |
      +---------------+-------+
      1 row in set (0.00 sec)
       
      MariaDB [tmp]> show global variables like 'gtid_domain_id';
      +----------------+-------+
      | Variable_name  | Value |
      +----------------+-------+
      | gtid_domain_id | 22    |
      +----------------+-------+
      1 row in set (0.00 sec)
       
      MariaDB [tmp]> show global variables like 'gtid_current_pos';
      +------------------+-------+
      | Variable_name    | Value |
      +------------------+-------+
      | gtid_current_pos |       |
      +------------------+-------+
      1 row in set (0.00 sec)

      On galera1:

      MariaDB [(none)]> show global variables like 'server_id';
      +---------------+-------+
      | Variable_name | Value |
      +---------------+-------+
      | server_id     | 1     |
      +---------------+-------+
      1 row in set (0.00 sec)
       
      MariaDB [(none)]> show global variables like 'gtid_domain_id';
      +----------------+-------+
      | Variable_name  | Value |
      +----------------+-------+
      | gtid_domain_id | 21    |
      +----------------+-------+
      1 row in set (0.00 sec)
       
      MariaDB [(none)]> show global variables like 'gtid_current_pos';
      +------------------+--------+
      | Variable_name    | Value  |
      +------------------+--------+
      | gtid_current_pos | 21-1-3 |
      +------------------+--------+
      1 row in set (0.00 sec)
       
      MariaDB [(none)]> show slave status\G
      *************************** 1. row ***************************
                     Slave_IO_State: Waiting for master to send event
                        Master_Host: 192.168.1.33
                        Master_User: repl
                        Master_Port: 3306
                      Connect_Retry: 60
                    Master_Log_File: mariadb-bin.000001
                Read_Master_Log_Pos: 314
                     Relay_Log_File: localhost-relay-bin.000002
                      Relay_Log_Pos: 603
              Relay_Master_Log_File: mariadb-bin.000001
                   Slave_IO_Running: Yes
                  Slave_SQL_Running: Yes
                    Replicate_Do_DB: 
                Replicate_Ignore_DB: 
                 Replicate_Do_Table: 
             Replicate_Ignore_Table: 
            Replicate_Wild_Do_Table: 
        Replicate_Wild_Ignore_Table: 
                         Last_Errno: 0
                         Last_Error: 
                       Skip_Counter: 0
                Exec_Master_Log_Pos: 314
                    Relay_Log_Space: 904
                    Until_Condition: None
                     Until_Log_File: 
                      Until_Log_Pos: 0
                 Master_SSL_Allowed: No
                 Master_SSL_CA_File: 
                 Master_SSL_CA_Path: 
                    Master_SSL_Cert: 
                  Master_SSL_Cipher: 
                     Master_SSL_Key: 
              Seconds_Behind_Master: 0
      Master_SSL_Verify_Server_Cert: No
                      Last_IO_Errno: 0
                      Last_IO_Error: 
                     Last_SQL_Errno: 0
                     Last_SQL_Error: 
        Replicate_Ignore_Server_Ids: 
                   Master_Server_Id: 2
                     Master_SSL_Crl: 
                 Master_SSL_Crlpath: 
                         Using_Gtid: Current_Pos
                        Gtid_IO_Pos: 21-1-3
      1 row in set (0.00 sec)

      On galera2:

      MariaDB [(none)]> show global variables like 'server_id';
      +---------------+-------+
      | Variable_name | Value |
      +---------------+-------+
      | server_id     | 1     |
      +---------------+-------+
      1 row in set (0.00 sec)
       
      MariaDB [(none)]> show global variables like 'gtid_domain_id';
      +----------------+-------+
      | Variable_name  | Value |
      +----------------+-------+
      | gtid_domain_id | 21    |
      +----------------+-------+
      1 row in set (0.00 sec)
       
      MariaDB [(none)]> show global variables like 'gtid_current_pos';
      +------------------+---------+
      | Variable_name    | Value   |
      +------------------+---------+
      | gtid_current_pos | 21-1-10 |
      +------------------+---------+
      1 row in set (0.00 sec)

      Now let's insert some data on the master to see how it affects gtid_current_pos:

      MariaDB [tmp]> INSERT INTO tmp.test_table VALUES (1, 'str');
      Query OK, 1 row affected (0.00 sec)
       
      MariaDB [tmp]> show global variables like 'gtid_current_pos';
      +------------------+--------+
      | Variable_name    | Value  |
      +------------------+--------+
      | gtid_current_pos | 22-2-1 |
      +------------------+--------+

      On galera1:

      MariaDB [(none)]> show global variables like 'gtid_current_pos';
      +------------------+---------------+
      | Variable_name    | Value         |
      +------------------+---------------+
      | gtid_current_pos | 22-2-1,21-1-3 |
      +------------------+---------------+
      1 row in set (0.00 sec)
       
      MariaDB [(none)]> show slave status\G
      *************************** 1. row ***************************
                     Slave_IO_State: Waiting for master to send event
                        Master_Host: 192.168.1.33
                        Master_User: repl
                        Master_Port: 3306
                      Connect_Retry: 60
                    Master_Log_File: mariadb-bin.000001
                Read_Master_Log_Pos: 485
                     Relay_Log_File: localhost-relay-bin.000002
                      Relay_Log_Pos: 774
              Relay_Master_Log_File: mariadb-bin.000001
                   Slave_IO_Running: Yes
                  Slave_SQL_Running: Yes
                    Replicate_Do_DB: 
                Replicate_Ignore_DB: 
                 Replicate_Do_Table: 
             Replicate_Ignore_Table: 
            Replicate_Wild_Do_Table: 
        Replicate_Wild_Ignore_Table: 
                         Last_Errno: 0
                         Last_Error: 
                       Skip_Counter: 0
                Exec_Master_Log_Pos: 485
                    Relay_Log_Space: 1075
                    Until_Condition: None
                     Until_Log_File: 
                      Until_Log_Pos: 0
                 Master_SSL_Allowed: No
                 Master_SSL_CA_File: 
                 Master_SSL_CA_Path: 
                    Master_SSL_Cert: 
                  Master_SSL_Cipher: 
                     Master_SSL_Key: 
              Seconds_Behind_Master: 0
      Master_SSL_Verify_Server_Cert: No
                      Last_IO_Errno: 0
                      Last_IO_Error: 
                     Last_SQL_Errno: 0
                     Last_SQL_Error: 
        Replicate_Ignore_Server_Ids: 
                   Master_Server_Id: 2
                     Master_SSL_Crl: 
                 Master_SSL_Crlpath: 
                         Using_Gtid: Current_Pos
                        Gtid_IO_Pos: 21-1-3,22-2-1
      1 row in set (0.00 sec)

      On galera2 the value is now empty:

      MariaDB [(none)]> show global variables like 'gtid_current_pos';
      +------------------+-------+
      | Variable_name    | Value |
      +------------------+-------+
      | gtid_current_pos |       |
      +------------------+-------+
      1 row in set (0.00 sec)

      Attachments

        Issue Links

          Activity

            Same issue with:

            binlog_format=ROW
            slave_parallel_threads=0
            wsrep_slave_threads=2

            and:

            binlog_format=ROW
            slave_parallel_threads=0
            wsrep_slave_threads=1

            I was able to see the GTID propagate to galera2 yesterday with the second set of values, but I haven't been able to make that happen again today. There might be some other interaction going on here.

            GeoffMontee Geoff Montee (Inactive) added a comment - Same issue with: binlog_format=ROW slave_parallel_threads=0 wsrep_slave_threads=2 and: binlog_format=ROW slave_parallel_threads=0 wsrep_slave_threads=1 I was able to see the GTID propagate to galera2 yesterday with the second set of values, but I haven't been able to make that happen again today. There might be some other interaction going on here.

            This problem might be unrelated to the replication thread.

            Leaving out the master server completely, I inserted a value directly into galera1. Galera2 assigned the transaction a different GTID.

            galera1:

            MariaDB [(none)]> INSERT INTO tmp.test_table VALUES (12, 'str');
            Query OK, 1 row affected (0.01 sec)
             
            MariaDB [(none)]> show global variables like 'gtid_current_pos';
            +------------------+----------------+
            | Variable_name    | Value          |
            +------------------+----------------+
            | gtid_current_pos | 22-2-11,21-1-1 |
            +------------------+----------------+
            1 row in set (0.01 sec)

            galera2:

            MariaDB [(none)]> show global variables like 'gtid_current_pos';
            +------------------+--------+
            | Variable_name    | Value  |
            +------------------+--------+
            | gtid_current_pos | 21-1-2 |
            +------------------+--------+
            1 row in set (0.00 sec)

            galera1 binlog:

            #150714 12:08:59 server id 1  end_log_pos 520   GTID 21-1-1
            /*!100001 SET @@session.gtid_domain_id=21*//*!*/;
            /*!100001 SET @@session.server_id=1*//*!*/;
            /*!100001 SET @@session.gtid_seq_no=1*//*!*/;
            BEGIN
            /*!*/;
            # at 520
            # at 571
            #150714 12:08:59 server id 1  end_log_pos 571   Table_map: `tmp`.`test_table` mapped to number 70
            #150714 12:08:59 server id 1  end_log_pos 609   Write_rows: table id 70 flags: STMT_END_F
            ### INSERT INTO `tmp`.`test_table`
            ### SET
            ###   @1=12
            ###   @2='str'
            # at 609
            #150714 12:08:59 server id 1  end_log_pos 636   Xid = 31
            COMMIT/*!*/;

            galera2 binlog:

            #150714 12:08:59 server id 1  end_log_pos 506   GTID 21-1-2
            /*!100001 SET @@session.server_id=1*//*!*/;
            /*!100001 SET @@session.gtid_seq_no=2*//*!*/;
            BEGIN
            /*!*/;
            # at 506
            # at 557
            #150714 12:08:59 server id 1  end_log_pos 557   Table_map: `tmp`.`test_table` mapped to number 70
            #150714 12:08:59 server id 1  end_log_pos 595   Write_rows: table id 70 flags: STMT_END_F
            ### INSERT INTO `tmp`.`test_table`
            ### SET
            ###   @1=12
            ###   @2='str'
            # at 595
            #150714 12:08:59 server id 1  end_log_pos 622   Xid = 31
            COMMIT/*!*/;

            GeoffMontee Geoff Montee (Inactive) added a comment - This problem might be unrelated to the replication thread. Leaving out the master server completely, I inserted a value directly into galera1. Galera2 assigned the transaction a different GTID. galera1: MariaDB [(none)]> INSERT INTO tmp.test_table VALUES (12, 'str'); Query OK, 1 row affected (0.01 sec)   MariaDB [(none)]> show global variables like 'gtid_current_pos'; +------------------+----------------+ | Variable_name | Value | +------------------+----------------+ | gtid_current_pos | 22-2-11,21-1-1 | +------------------+----------------+ 1 row in set (0.01 sec) galera2: MariaDB [(none)]> show global variables like 'gtid_current_pos'; +------------------+--------+ | Variable_name | Value | +------------------+--------+ | gtid_current_pos | 21-1-2 | +------------------+--------+ 1 row in set (0.00 sec) galera1 binlog: #150714 12:08:59 server id 1 end_log_pos 520 GTID 21-1-1 /*!100001 SET @@session.gtid_domain_id=21*//*!*/; /*!100001 SET @@session.server_id=1*//*!*/; /*!100001 SET @@session.gtid_seq_no=1*//*!*/; BEGIN /*!*/; # at 520 # at 571 #150714 12:08:59 server id 1 end_log_pos 571 Table_map: `tmp`.`test_table` mapped to number 70 #150714 12:08:59 server id 1 end_log_pos 609 Write_rows: table id 70 flags: STMT_END_F ### INSERT INTO `tmp`.`test_table` ### SET ### @1=12 ### @2='str' # at 609 #150714 12:08:59 server id 1 end_log_pos 636 Xid = 31 COMMIT/*!*/; galera2 binlog: #150714 12:08:59 server id 1 end_log_pos 506 GTID 21-1-2 /*!100001 SET @@session.server_id=1*//*!*/; /*!100001 SET @@session.gtid_seq_no=2*//*!*/; BEGIN /*!*/; # at 506 # at 557 #150714 12:08:59 server id 1 end_log_pos 557 Table_map: `tmp`.`test_table` mapped to number 70 #150714 12:08:59 server id 1 end_log_pos 595 Write_rows: table id 70 flags: STMT_END_F ### INSERT INTO `tmp`.`test_table` ### SET ### @1=12 ### @2='str' # at 595 #150714 12:08:59 server id 1 end_log_pos 622 Xid = 31 COMMIT/*!*/;

            Users run into this problem all the time. Should wsrep_gtid_mode (MDEV-6594) be backported to MariaDB Galera Cluster 10.0?

            https://mariadb.com/kb/en/mariadb/galera-cluster-system-variables/#wsrep_gtid_mode

            Or should this issue just be closed as Won't Fix for 10.0?

            GeoffMontee Geoff Montee (Inactive) added a comment - Users run into this problem all the time. Should wsrep_gtid_mode ( MDEV-6594 ) be backported to MariaDB Galera Cluster 10.0? https://mariadb.com/kb/en/mariadb/galera-cluster-system-variables/#wsrep_gtid_mode Or should this issue just be closed as Won't Fix for 10.0?

            Hi GeoffMontee!

            I think this can be done without wsrep_gtid_mode (some modification of 10715 patch). But there are some precaution that user should tale to avoid double apply of events on galera node
            , I mean use of Ignore_slave_ids is required , for this patch to work.

            sachin.setiya.007 Sachin Setiya (Inactive) added a comment - Hi GeoffMontee ! I think this can be done without wsrep_gtid_mode (some modification of 10715 patch). But there are some precaution that user should tale to avoid double apply of events on galera node , I mean use of Ignore_slave_ids is required , for this patch to work.

            10715 patch in 10.1.31 will fix this. Wont fix for 10.0-galera.

            sachin.setiya.007 Sachin Setiya (Inactive) added a comment - 10715 patch in 10.1.31 will fix this. Wont fix for 10.0-galera.

            People

              sachin.setiya.007 Sachin Setiya (Inactive)
              GeoffMontee Geoff Montee (Inactive)
              Votes:
              7 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.