Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-30165

X-lock on supremum for prepared transaction for RR

Details

    Description

      Statement-based replications stops as on slave there is X-lock on supremum in prepared transaction.

      Attachments

        Issue Links

          Activity

            knielsen, thank you, I really appreciate your comment, and that you filed MDEV-32020.

            I agree that this as well as MDEV-28709 or MDEV-26682 are work-arounds for something that would better be fixed at a higher layer, specifically, the replication of distributed transactions.

            marko Marko Mäkelä added a comment - knielsen , thank you, I really appreciate your comment, and that you filed MDEV-32020 . I agree that this as well as MDEV-28709 or MDEV-26682 are work-arounds for something that would better be fixed at a higher layer, specifically, the replication of distributed transactions.
            Elkin Andrei Elkin added a comment -

            knielsen, not less than Marko I appreciate your constructive critique and ideas always full of merit. When it comes to the 'wrong design' I must admit it was not perfect. Few decisions could have been taken earlier. Nevertheless there's nothing fundamentally in the way of two phase XA replication to perform bug-free, including MDEV-32020 - where I mentioned a solution i.e add the forced index(es) into the Rows-log-event context and force using them on slave too, to provide - this must come without saying - fastest (compare with other methods) failover as well as a framework to further online replication.
            That's right it's a field of great technical challenges. Yet so far we've been emerging successfully from any difficulty for which I must be grateful to Innodb colleagues.

            Elkin Andrei Elkin added a comment - knielsen , not less than Marko I appreciate your constructive critique and ideas always full of merit. When it comes to the 'wrong design' I must admit it was not perfect. Few decisions could have been taken earlier. Nevertheless there's nothing fundamentally in the way of two phase XA replication to perform bug-free, including MDEV-32020 - where I mentioned a solution i.e add the forced index(es) into the Rows-log-event context and force using them on slave too , to provide - this must come without saying - fastest (compare with other methods) failover as well as a framework to further online replication. That's right it's a field of great technical challenges. Yet so far we've been emerging successfully from any difficulty for which I must be grateful to Innodb colleagues.

            This change may be useful on its own, independent of replication, to reduce some avoidable locking conflicts. This has been extensively tested, and I do not foresee any correctness problems. But I do tend to agree with knielsen’s views in MDEV-32020.

            marko Marko Mäkelä added a comment - This change may be useful on its own, independent of replication, to reduce some avoidable locking conflicts. This has been extensively tested, and I do not foresee any correctness problems. But I do tend to agree with knielsen ’s views in MDEV-32020 .

            I did a quick test, and I see that the gap locks are released only after the XA PREPARE is binlogged, so the binlog order between conflicting transactions should be ok.

            But now I wonder how it is ensured that the binlog order is consistent with the order of XA PREPARE inside of InnoDB? This is usually handled through innobase_commit_ordered(), but that I think is not involved here as there is no commit during XA PREPARE.

            I'm wondering if this causes a problem for mariabackup --no-lock?
            Can it happen that the binlog has XA PREPARE of transactions T1 and T2 in this order, while InnoDB has in its redo lock the transactions prepared in the opposite order T2, T1?

            And then if mariabackup --no-lock snapshots the database at the point where T2 is XA PREPAREd, but T1 is not, then there will be no valid binlog position corresponding to this state, so a slave cannot be provisioned from the backup?

            knielsen Kristian Nielsen added a comment - I did a quick test, and I see that the gap locks are released only after the XA PREPARE is binlogged, so the binlog order between conflicting transactions should be ok. But now I wonder how it is ensured that the binlog order is consistent with the order of XA PREPARE inside of InnoDB? This is usually handled through innobase_commit_ordered(), but that I think is not involved here as there is no commit during XA PREPARE. I'm wondering if this causes a problem for mariabackup --no-lock? Can it happen that the binlog has XA PREPARE of transactions T1 and T2 in this order, while InnoDB has in its redo lock the transactions prepared in the opposite order T2, T1? And then if mariabackup --no-lock snapshots the database at the point where T2 is XA PREPAREd, but T1 is not, then there will be no valid binlog position corresponding to this state, so a slave cannot be provisioned from the backup?
            Elkin Andrei Elkin added a comment - - edited

            > And then if mariabackup --no-lock snapshots ...

            While Innodb can't (maybe yet) record the last binlog position of XA-PREPARE, those could be recovered on a being provisioned slave from binlog as normal transactions do. Say REDO log contains C1,C2,P3,P4 (C stands for Commit, P for XA-PREPARE), and binlog has only C1,C2,P3. At slave server recovery P4 is to be rolled back, but P3 would advance the replication start position (that initially specifies "next to C2").
            Notice that in place of P3 in binlog can be any P:i. That is binlog does not have to be ordered for P:s in the REDO order. And while it's so I see there's only concern of In doubt P:s identification. For that purpose the last binlog position of a C that Innodb remembers upon crash can safely serve as the origin for scanning of P:s in binlog. E.g C2 is such origin (2 is the binlog index), any P1 or earlier may not be considered as in-doubt. P3 or P4 must be.

            Elkin Andrei Elkin added a comment - - edited > And then if mariabackup --no-lock snapshots ... While Innodb can't (maybe yet) record the last binlog position of XA-PREPARE, those could be recovered on a being provisioned slave from binlog as normal transactions do. Say REDO log contains C1,C2,P3,P4 (C stands for Commit, P for XA-PREPARE), and binlog has only C1,C2,P3. At slave server recovery P4 is to be rolled back, but P3 would advance the replication start position (that initially specifies "next to C2"). Notice that in place of P3 in binlog can be any P:i. That is binlog does not have to be ordered for P:s in the REDO order. And while it's so I see there's only concern of In doubt P:s identification. For that purpose the last binlog position of a C that Innodb remembers upon crash can safely serve as the origin for scanning of P:s in binlog. E.g C2 is such origin (2 is the binlog index), any P1 or earlier may not be considered as in-doubt. P3 or P4 must be.

            People

              vlad.lesin Vladislav Lesin
              vlad.lesin Vladislav Lesin
              Votes:
              1 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.