Uploaded image for project: 'MariaDB Connector/C'
  1. MariaDB Connector/C
  2. CONC-650

Connector/C fails to read MariaDB binlog with XA transactions

Details

    • Bug
    • Status: Closed (View Workflow)
    • Blocker
    • Resolution: Fixed
    • 3.3.5
    • 3.3.6
    • Replication/Binlog API
    • None
    • Client running on Windows
      MariaDB 10.7

    Description

      Hi, I'm using MariaDB Connector/C 3.3.5 to read binary logs.
      It works perfectly for MySQL 8.0 and MariaDB 10.5 databases.
      For MariaDB 10.6 and above, mariadb_rpl_fetch() crashes if the binlog contains XA transactions.
      Specifically, the function fails when it tries to read event type GTID with this contents:
      'XA START X'54584e31',X'',1 GTID 0-1-21'

      If the function doesn't crash, it returns this error:

      Read next binary log event failed; mariadb_rpl_fetch error 1236 (Failed to replace GTID event with backwards-compatible event: corrupt event.;

      Please advise . Thanks.
      Sruli

      Attachments

        1. binlog_reader.c
          2 kB
          Sruli Ganor
        2. image-2023-06-14-17-42-55-285.png
          256 kB
          Sruli Ganor
        3. mysql-bin.000013
          1 kB
          Sruli Ganor

        Activity

          sruli.ganor Sruli Ganor added a comment -

          Hi,
          With the new variables, the program runs smoothly
          Many thanks!

          P.s. Are these variables documented somewhere?

          Regards

          ‫בתאריך יום ה׳, 15 ביוני 2023 ב-15:48 מאת ‪Georg Richter (Jira)‬‏ <‪
          jira@mariadb.org‬‏>:‬

          sruli.ganor Sruli Ganor added a comment - Hi, With the new variables, the program runs smoothly Many thanks! P.s. Are these variables documented somewhere? Regards ‫בתאריך יום ה׳, 15 ביוני 2023 ב-15:48 מאת ‪Georg Richter (Jira)‬‏ <‪ jira@mariadb.org‬‏>:‬
          sruli.ganor Sruli Ganor added a comment -

          Hi, please note that unfortunately we still can't read XA transactions
          properly.
          In MariaDB 10.5 the XA START statement was included in a QUERY event, so we
          could read the statement and mark the beginning of the transaction and
          the transaction ID.
          In MariaDB 10.6 and above, the XA START statement is included in a
          GTID_EVENT (162).
          Function mariadb_rpl_fetch() does NOT return the event body. So we don't
          get the statement and don't know the ID of the new transaction. This
          impacts the logic of our application.

          Please advise how to get the XA START statement, or at least the
          transaction ID.

          Thanks
          Sruli

          ‫בתאריך יום ה׳, 15 ביוני 2023 ב-16:26 מאת ‪Sruli Ganor (Jira)‬‏ <‪
          jira@mariadb.org‬‏>:‬

          sruli.ganor Sruli Ganor added a comment - Hi, please note that unfortunately we still can't read XA transactions properly. In MariaDB 10.5 the XA START statement was included in a QUERY event, so we could read the statement and mark the beginning of the transaction and the transaction ID. In MariaDB 10.6 and above, the XA START statement is included in a GTID_EVENT (162). Function mariadb_rpl_fetch() does NOT return the event body. So we don't get the statement and don't know the ID of the new transaction. This impacts the logic of our application. Please advise how to get the XA START statement, or at least the transaction ID. Thanks Sruli ‫בתאריך יום ה׳, 15 ביוני 2023 ב-16:26 מאת ‪Sruli Ganor (Jira)‬‏ <‪ jira@mariadb.org‬‏>:‬
          georg Georg Richter added a comment -

          The transaction id is included in XID_EVENT, and it will be read in mariadb_rpl_fetch(): https://github.com/mariadb-corporation/mariadb-connector-c/blob/3.3/libmariadb/mariadb_rpl.c#L1501

          georg Georg Richter added a comment - The transaction id is included in XID_EVENT, and it will be read in mariadb_rpl_fetch(): https://github.com/mariadb-corporation/mariadb-connector-c/blob/3.3/libmariadb/mariadb_rpl.c#L1501
          sruli.ganor Sruli Ganor added a comment -

          XID_EVENT appears after COMMIT of a *regular *(not XA) transaction.
          We must know the transaction ID when the XA transaction starts (XA START).
          In MySQL and in MariaDB 10.5, we can read the XA statements and extract the
          transaction id (e.g., TXN1) so we can sort all the transaction's events by
          its ID.

          XA START 'TXN1';
          insert into table1
          insert into table2
          XA END 'TXN1';
          XA PREPARE 'TXN1';
          XA COMMIT 'TXN1';

          In MariaDB 10.6, I cannot read the XA START statement because
          mariadb_rpl_fetch doesn't return it. I can only get the GTID. However, the
          GTID is different for the XA START and XA COMMIT. E.g. in my case, XA START
          has GTID 0-1-36 and the XA COMMIT is 0-1-37. Therefore, there is no common
          ID that applies to all the transaction's events, and our application fails.

          Hope it's clear. If not, we can talk since this is a critical issue for our
          customers.
          Thanks
          Sruli

          ‫בתאריך יום א׳, 18 ביוני 2023 ב-18:12 מאת ‪Georg Richter (Jira)‬‏ <‪
          jira@mariadb.org‬‏>:‬

          sruli.ganor Sruli Ganor added a comment - XID_EVENT appears after COMMIT of a *regular *(not XA) transaction. We must know the transaction ID when the XA transaction starts (XA START). In MySQL and in MariaDB 10.5, we can read the XA statements and extract the transaction id (e.g., TXN1) so we can sort all the transaction's events by its ID. XA START 'TXN1'; insert into table1 insert into table2 XA END 'TXN1'; XA PREPARE 'TXN1'; XA COMMIT 'TXN1'; In MariaDB 10.6, I cannot read the XA START statement because mariadb_rpl_fetch doesn't return it. I can only get the GTID. However, the GTID is different for the XA START and XA COMMIT. E.g. in my case, XA START has GTID 0-1-36 and the XA COMMIT is 0-1-37. Therefore, there is no common ID that applies to all the transaction's events, and our application fails. Hope it's clear. If not, we can talk since this is a critical issue for our customers. Thanks Sruli ‫בתאריך יום א׳, 18 ביוני 2023 ב-18:12 מאת ‪Georg Richter (Jira)‬‏ <‪ jira@mariadb.org‬‏>:‬
          georg Georg Richter added a comment -

          It looks like with implementation of MDEV-742 documentation was forgotten.
          I've updated https://mariadb.com/kb/en/gtid_event/ and will push a fix.

          georg Georg Richter added a comment - It looks like with implementation of MDEV-742 documentation was forgotten. I've updated https://mariadb.com/kb/en/gtid_event/ and will push a fix.

          People

            georg Georg Richter
            sruli.ganor Sruli Ganor
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.