Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-9113

Endless loop in multi-source replication with do_domain_ids

    XMLWordPrintable

Details

    Description

      Inspired by bug reports MDEV-9107, MDEV-9108.

      Setup:
      We have three servers, S1, S2, S3. Each server has a unique server ID, and unique GTID domain ID – 1,2,3 correspondingly.

      Replication configuration is this:
      S1 <----> S3 ----> S2 ----> S1

      Every slave connection is configured the same way, e.g.

        do_domain_ids=(<GTID domain ID of the master it is connected to>),
        master_host='127.0.0.1', 
        master_port=<Port of the master it is connected to>,
        master_user='root',
        master_use_gtid=slave_pos;

      We configure all servers, start all slaves, create an event on S2, and check binary logs after a few moments.
      It turns out that in S2 binary log it is written as expected, once, with GTID 2-2-1;
      however, in both S1 and S3 binary logs, it keeps being written endlessly, over and over, with GTIDs 3-2-N, where N keeps increasing. Note 3 instead of 2 in GTID!

      MTR test cnf file t3.cnf

      !include suite/rpl/my.cnf
       
      [mysqld.1]
      log-bin=mysql-bin
      server-id=1
      log-slave-updates
      gtid-domain-id=1
      gtid-ignore-duplicates
       
      [mysqld.2]
      log-bin=mysql-bin
      server-id=2
      log-slave-updates
      gtid-domain-id=2
      gtid-ignore-duplicates
       
      [mysqld.3]
      log-bin=mysql-bin
      server-id=3
      log-slave-updates
      gtid-domain-id=3
      gtid-ignore-duplicates
       
      [ENV]
      SERVER_MYPORT_1= @mysqld.1.port
      SERVER_MYPORT_2= @mysqld.2.port
      SERVER_MYPORT_3= @mysqld.3.port

      MTR test case t3.test

      --enable_connect_log
       
      --connect (server_1,127.0.0.1,root,,test,$SERVER_MYPORT_1)
       
      eval change master 'm2' to 
        do_domain_ids=(2),
        master_host='127.0.0.1', 
        master_port=$SERVER_MYPORT_2,
        master_user='root',
        master_use_gtid=slave_pos;
       
      eval change master 'm3' to 
        do_domain_ids=(3),
        master_host='127.0.0.1', 
        master_port=$SERVER_MYPORT_3,
        master_user='root',
        master_use_gtid=slave_pos;
       
      start all slaves;
       
      --connect (server_2,127.0.0.1,root,,test,$SERVER_MYPORT_2)
       
      eval change master 'm3' to 
        do_domain_ids=(3),
        master_host='127.0.0.1', 
        master_port=$SERVER_MYPORT_3,
        master_user='root',
        master_use_gtid=slave_pos;
       
      start all slaves;
       
      --connect (server_3,127.0.0.1,root,,test,$SERVER_MYPORT_3)
       
      eval change master 'm1' to 
        do_domain_ids=(1),
        master_host='127.0.0.1', 
        master_port=$SERVER_MYPORT_1,
        master_user='root',
        master_use_gtid=slave_pos;
       
      start all slaves;
       
      --connection server_2
       
      create table t1 (i int);
       
      --connection server_1
      show binlog events;
       
      --connection server_2
      show binlog events;
       
      --connection server_3
      show binlog events;
       
      sleep 2;
       
      --connection server_1
      show binlog events;
       
      --connection server_2
      show binlog events;
       
      --connection server_3
      show binlog events;

      Output

      connection server_1;
      show binlog events;
      Log_name	Pos	Event_type	Server_id	End_log_pos	Info
      mysql-bin.000001	4	Format_desc	1	249	Server ver: 10.1.8-MariaDB-log, Binlog ver: 4
      mysql-bin.000001	249	Gtid_list	1	274	[]
      mysql-bin.000001	274	Binlog_checkpoint	1	313	mysql-bin.000001
      mysql-bin.000001	313	Gtid	2	351	GTID 2-2-1
      mysql-bin.000001	351	Query	2	437	use `test`; create table t1 (i int)
      mysql-bin.000001	437	Gtid	2	475	GTID 3-2-1
      mysql-bin.000001	475	Query	2	561	use `test`; create table t1 (i int)
      mysql-bin.000001	561	Gtid	2	599	GTID 3-2-2
      mysql-bin.000001	599	Query	2	685	use `test`; create table t1 (i int)
      ......
      ......
      mysql-bin.000001	137829	Gtid	2	137867	GTID 3-2-1109
      mysql-bin.000001	137867	Query	2	137953	use `test`; create table t1 (i int)
      mysql-bin.000001	137953	Gtid	2	137991	GTID 3-2-1110
      mysql-bin.000001	137991	Query	2	138077	use `test`; create table t1 (i int)
      connection server_2;
      show binlog events;
      Log_name	Pos	Event_type	Server_id	End_log_pos	Info
      mysql-bin.000001	4	Format_desc	2	249	Server ver: 10.1.8-MariaDB-log, Binlog ver: 4
      mysql-bin.000001	249	Gtid_list	2	274	[]
      mysql-bin.000001	274	Binlog_checkpoint	2	313	mysql-bin.000001
      mysql-bin.000001	313	Gtid	2	351	GTID 2-2-1
      mysql-bin.000001	351	Query	2	437	use `test`; create table t1 (i int)
      connection server_3;
      show binlog events;
      Log_name	Pos	Event_type	Server_id	End_log_pos	Info
      mysql-bin.000001	4	Format_desc	3	249	Server ver: 10.1.8-MariaDB-log, Binlog ver: 4
      mysql-bin.000001	249	Gtid_list	3	274	[]
      mysql-bin.000001	274	Binlog_checkpoint	3	313	mysql-bin.000001
      mysql-bin.000001	313	Gtid	2	351	GTID 3-2-1
      mysql-bin.000001	351	Query	2	437	use `test`; create table t1 (i int)
      mysql-bin.000001	437	Gtid	2	475	GTID 3-2-2
      mysql-bin.000001	475	Query	2	561	use `test`; create table t1 (i int)
      ......
      ......
      mysql-bin.000001	149857	Gtid	2	149895	GTID 3-2-1207
      mysql-bin.000001	149895	Query	2	149981	use `test`; create table t1 (i int)
      mysql-bin.000001	149981	Gtid	2	150019	GTID 3-2-1208
      mysql-bin.000001	150019	Query	2	150105	use `test`; create table t1 (i int)

      I would have thought there is something wrong with the replication topology, but if we remove do_domain_ids clauses, nothing like this happens – the event is written once, with the proper GTID, and that's it.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.