Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-6594

Use separate domain_id for Galera transactions

Details

    Description

      In a Galera cluster, a node can log 2 kinds of transactions.

      1. Galera transactions : Ones which are replicated to other nodes within the cluster post-SST.
      2. Local transactions : Local transactions can be transactions involving non-innodb (except myisam when --wsrep-replicate-myisam is enabled) DMLs which are not relayed to other nodes.

      Now, in a scenario where galera-transactions from one cluster are being
      asynchronously replicated to an another cluster using the traditional
      MariaDB replication, having following 2 additional features would come in
      handy during setup and maintenance of such replication topologies.

      1. Separate domain_id for Galera-transaction :

        An external 'listener' of an n-node galera cluster can essentially
        see n (=local transactions) + 1 (=galera transactions) logical streams
        of transactions. So, having all galera transactions logged using a separate
        domain_id would make it easier to identify, filter (see #2) and replicate
        those transactions to an external replication slave or a different cluster
        altogether.

        This task introduces a new GTID domain ID (wsrep_gtid_domain_id) to be
        used to tag all galera transactions. The local transactions, however,
        will be tagged by gtid_domain_id as usual.

        Now, since the Galera transactions can originate on any of the nodes,
        it is important that all the cluster nodes have same GTID domain ID
        (wsrep_gtid_domain_id). So, in order to enforce this, SST (Snapshot
        State Transfer) scripts have been modified to automatically transfer
        the value of donor's wsrep_gtid_domain_id to the joiner node during
        SST process. This will make sure that all the nodes in the cluster
        have same wsrep_gtid_domain_id and thus all the galera transactions
        will essentially have same domain ID.

        A new server system variable wsrep_gtid_mode has been introduced to
        control this new behavior an this remain backward compatible. When OFF
        (default), wsrep_gtid_domain_id is ignored and all the transactions will
        be tagged based on gtid_domain_id. On the other hand, when wsrep_gtid_mode
        is ON, wsrep_gtid_domain_id takes effect and will be used to mark all
        Galera transactions.

        New Server System Variables:

        1. wsrep_gtid_domain_id:
          • Description: When wsrep_gtid_mode is set, this value is used as gtid_domain_id for galera transactions and also copied to the joiner nodes during state transfer. It is ignored, otherwise.
          • Dynamic: Yes
          • Data Type: Numeric (32-bit unsigned integer)
          • Default : 0
        2. wsrep_gtid_mode:
          • Description: Automatically update the (joiner) node's wsrep_gtid_domain_id value with that of donor's (received during state transfer) and use it in place of gtid_domain_id for all galera transactions. When OFF (default), wsrep_gtid_domain_id is simply ignored (backward compatibility).
          • Scope: Global
          • Dynamic: Yes
          • Data Type: Boolean
          • Default value : False

        Changes in SST scripts:

        • wsrep_sst_rsync :
          The script has been modified to also transfer donor's wsrep_gtid_domain_id alongwith the state ID.
        • wsrep_sst_xtrabackup (-v2) :
          Similar to the rsync SST script, the xtrabackup scripts have been modified to also transfer donor's
          wsrep_gtid_domain_id alongwith the state ID.
        • wsrep_sst_mysqldump :
          The script has been modified to set joiner's wsrep_gtid_domain_id with that of donor's only when Joiner's wsrep_gtid_mode=ON.
      2. Domain_id based replication filters (MDEV-6593) :

        Enable a replication slave to filter and apply transactions based on
        transaction's domain_id. As described above, this would enable the
        asynchronous replication of galera transaction outside the galera
        cluster.

      A use case described here:
      http://linsenraum.de/erkules_int/2014/07/galera-cluster-using-gtid-mysql-vs-mariadb.html

      Attachments

        Issue Links

          Activity

            nirbhay_c Nirbhay Choubey (Inactive) created issue -
            nirbhay_c Nirbhay Choubey (Inactive) made changes -
            Field Original Value New Value
            erkules erkan yanar added a comment -

            Great.
            Is there a way to make sure domain-id 1 is only set for galera.
            Not that it is necessary. Just thinking about having trouble with mixed setups.

            erkules erkan yanar added a comment - Great. Is there a way to make sure domain-id 1 is only set for galera. Not that it is necessary. Just thinking about having trouble with mixed setups.
            serg Sergei Golubchik made changes -
            Fix Version/s 10.1 [ 16100 ]
            nirbhay_c Nirbhay Choubey (Inactive) made changes -
            Assignee Nirbhay Choubey [ nirbhay_c ]
            nirbhay_c Nirbhay Choubey (Inactive) made changes -
            Fix Version/s 10.1.4 [ 18400 ]
            Fix Version/s 10.1 [ 16100 ]
            nirbhay_c Nirbhay Choubey (Inactive) made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            nirbhay_c Nirbhay Choubey (Inactive) made changes -
            Description In a Galera cluster, a node can log 2 kinds of transactions.

            1) Galera transaction :
              Ones which are replicated to other nodes within the cluster post-SST.

            2) Local transactions :
              Local transactions can be transactions involving non-innodb (except
              myisam when --wsrep-replicate-myisam is enabled) DMLs which are not
              relayed to other nodes.

            Now, in a scenario where galera-transactions from one cluster are being
            asynchronously replicated to an another cluster using the traditional
            MariaDB replication, having following 2 additional features would come in
            handy during setup and maintenance of such replication topologies.

            1) Separate domain_id for Galera-transaction :
                An external 'listener' of an n-node galera cluster can essentially see
                n (=local transactions) + 1 (=galera transactions) logical streams of
                transactions. So, having all galera transactions logged using a separate
                domain_id would make it easier to identify, filter (see #2) and replicate
                those transactions to an external replication slave or a cluster
                altogether.

            2) Domain_id based replication filters :
                Enable a replication slave to filter and apply transactions based on
                transaction's domain_id. As described above, this would enable the
                asynchronous replication of galera transaction outside the galera
                cluster.

            Note : #2 would be handled under MDEV-6593

            A use case described here:
            http://linsenraum.de/erkules_int/2014/07/galera-cluster-using-gtid-mysql-vs-mariadb.html
            In a Galera cluster, a node can log 2 kinds of transactions.

            # *Galera transactions* : Ones which are replicated to other nodes within the cluster post-SST.
            # *Local transactions* : Local transactions can be transactions involving non-innodb (except myisam when --wsrep-replicate-myisam is enabled) DMLs which are not relayed to other nodes.

            Now, in a scenario where galera-transactions from one cluster are being
            asynchronously replicated to an another cluster using the traditional
            MariaDB replication, having following 2 additional features would come in
            handy during setup and maintenance of such replication topologies.

            # *Separate domain_id for Galera-transaction* :
            {quote}
            An external 'listener' of an n-node galera cluster can essentially
            see n (=local transactions) + 1 (=galera transactions) logical streams
            of transactions. So, having all galera transactions logged using a separate
            domain_id would make it easier to identify, filter (see #2) and replicate
            those transactions to an external replication slave or a different cluster
            altogether.

            This task introduces a new GTID domain ID (wsrep_gtid_domain_id) to be
            used to tag all galera transactions. The local transactions, however,
            will be tagged by gtid_domain_id as usual.

            Now, since the Galera transactions can originate on any of the nodes,
            it is important that all the cluster nodes have same GTID domain ID
            (wsrep_gtid_domain_id). So, in order to enforce this, SST (Snapshot
            State Transfer) scripts have been modified to automatically transfer
            the value of donor's wsrep_gtid_domain_id to the joiner node during
            SST process. This will make sure that all the nodes in the cluster
            have same wsrep_gtid_domain_id and thus all the galera transactions
            will essentially have same domain ID.

            A new server system variable wsrep_gtid_mode has been introduced to
            control this new behavior an this remain backward compatible. When OFF
            (default), wsrep_gtid_domain_id is ignored and all the transactions will
            be tagged based on gtid_domain_id. On the other hand, when wsrep_gtid_mode
            is ON, wsrep_gtid_domain_id takes effect and will be used to mark all
            Galera transactions.

            *New Server System Variables:*

            # *wsrep_gtid_domain_id:*
            #* Description: When wsrep_gtid_mode is set, this value is used as gtid_domain_id for galera transactions and also copied to the joiner nodes during state transfer. It is ignored, otherwise.
            #* Dynamic: Yes
            #* Data Type: Numeric (32-bit unsigned integer)
            #* Default : 0
            # *wsrep_gtid_mode:*
            #* Description: Automatically update the (joiner) node's wsrep_gtid_domain_id value with that of donor's (received during state transfer) and use it in place of gtid_domain_id for all galera transactions. When OFF (default), wsrep_gtid_domain_id is simply ignored (backward compatibility).
            #* Scope: Global
            #* Dynamic: Yes
            #* Data Type: Boolean
            #* Default value : False

            *Changes in SST scripts:*

            * wsrep_sst_rsync :
                The script has been modified to also transfer donor's wsrep_gtid_domain_id alongwith the state ID.

            * wsrep_sst_xtrabackup (-v2) :
                Similar to the rsync SST script, the xtrabackup scripts have been modified to also transfer donor's
                wsrep_gtid_domain_id alongwith the state ID.

            * wsrep_sst_mysqldump :
                The script has been modified to set joiner's wsrep_gtid_domain_id with that of donor's only when Joiner's wsrep_gtid_mode=ON.
            {quote}
            # *Domain_id based replication filters* (MDEV-6593) :
            {quote}
            Enable a replication slave to filter and apply transactions based on
            transaction's domain_id. As described above, this would enable the
            asynchronous replication of galera transaction outside the galera
            cluster.
            {quote}

            A use case described here:
            http://linsenraum.de/erkules_int/2014/07/galera-cluster-using-gtid-mysql-vs-mariadb.html
            nirbhay_c Nirbhay Choubey (Inactive) added a comment - http://lists.askmonty.org/pipermail/commits/2015-February/007471.html
            nirbhay_c Nirbhay Choubey (Inactive) made changes -
            Assignee Nirbhay Choubey [ nirbhay_c ] Jan Lindström [ jplindst ]
            Status In Progress [ 3 ] In Review [ 10002 ]

            Ok to push with minor comments adressed.

            jplindst Jan Lindström (Inactive) added a comment - Ok to push with minor comments adressed.
            jplindst Jan Lindström (Inactive) made changes -
            Assignee Jan Lindström [ jplindst ] Nirbhay Choubey [ nirbhay_c ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            nirbhay_c Nirbhay Choubey (Inactive) added a comment - https://github.com/MariaDB/server/commit/34d86ac9ff664972d2031e3fabf6c4a407e32c8a
            nirbhay_c Nirbhay Choubey (Inactive) made changes -
            Component/s Galera [ 10124 ]
            Component/s Galera SST [ 10121 ]
            Component/s wsrep [ 11500 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Workflow MariaDB v2 [ 52804 ] MariaDB v3 [ 65091 ]
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 65091 ] MariaDB v4 [ 132394 ]

            People

              nirbhay_c Nirbhay Choubey (Inactive)
              nirbhay_c Nirbhay Choubey (Inactive)
              Votes:
              3 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.