[CONC-650] Connector/C fails to read MariaDB binlog with XA transactions Created: 2023-06-14  Updated: 2023-06-20  Resolved: 2023-06-19

Status: Closed
Project: MariaDB Connector/C
Component/s: Replication/Binlog API
Affects Version/s: 3.3.5
Fix Version/s: 3.3.6

Type: Bug Priority: Blocker
Reporter: Sruli Ganor Assignee: Georg Richter
Resolution: Fixed Votes: 0
Labels: None
Environment:

Client running on Windows
MariaDB 10.7


Attachments: Text File binlog_reader.c     PNG File image-2023-06-14-17-42-55-285.png     File mysql-bin.000013    

 Description   

Hi, I'm using MariaDB Connector/C 3.3.5 to read binary logs.
It works perfectly for MySQL 8.0 and MariaDB 10.5 databases.
For MariaDB 10.6 and above, mariadb_rpl_fetch() crashes if the binlog contains XA transactions.
Specifically, the function fails when it tries to read event type GTID with this contents:
'XA START X'54584e31',X'',1 GTID 0-1-21'

If the function doesn't crash, it returns this error:

Read next binary log event failed; mariadb_rpl_fetch error 1236 (Failed to replace GTID event with backwards-compatible event: corrupt event.;

Please advise . Thanks.
Sruli



 Comments   
Comment by Sruli Ganor [ 2023-06-14 ]

A binary log file is attached.
The failure occurs when reading the Gtid at pos 592.
Thanks!

Comment by Georg Richter [ 2023-06-15 ]

Thanks for your bug report!

How and with which parameters (especially mariadb_slave_capability) does the replica connect to the source?

Comment by Sruli Ganor [ 2023-06-15 ]

Hi, I didn't set any replica parameters and am not familiar with
mariadb_slave_capability.
I ran the attached program on several MySQL-based databases such as MySQL
8, MariaDB 10.5 and RDS for MySQL, and it succeeded for all of them without
setting any special parameters.
It only fails for MariaDB 10.6 or above.

How do I show or set these parameters?

Please note that I cannot reply or attach in the Jira itself, because it's
locked for me.

Regards

‫בתאריך יום ה׳, 15 ביוני 2023 ב-12:10 מאת ‪Georg Richter (Jira)‬‏ <‪
jira@mariadb.org‬‏>:‬

Comment by Georg Richter [ 2023-06-15 ]

Hi,

The problem might be caused by MDEV-742 (fixed in 10.5.2), did you use 10.5.1 before?

Could you please try to set the following variables before calling rpl_open():

mysql_query(hMySql, "SET @mariadb_slave_capability=4");
mysql_query(hMySql, "SET @slave_gtid_strict_mode=1");
mysql_query(hMySql, "SET @slave_gtid_ignore_duplicates=1");

Comment by Sruli Ganor [ 2023-06-15 ]

Hi,
With the new variables, the program runs smoothly
Many thanks!

P.s. Are these variables documented somewhere?

Regards

‫בתאריך יום ה׳, 15 ביוני 2023 ב-15:48 מאת ‪Georg Richter (Jira)‬‏ <‪
jira@mariadb.org‬‏>:‬

Comment by Sruli Ganor [ 2023-06-18 ]

Hi, please note that unfortunately we still can't read XA transactions
properly.
In MariaDB 10.5 the XA START statement was included in a QUERY event, so we
could read the statement and mark the beginning of the transaction and
the transaction ID.
In MariaDB 10.6 and above, the XA START statement is included in a
GTID_EVENT (162).
Function mariadb_rpl_fetch() does NOT return the event body. So we don't
get the statement and don't know the ID of the new transaction. This
impacts the logic of our application.

Please advise how to get the XA START statement, or at least the
transaction ID.

Thanks
Sruli

‫בתאריך יום ה׳, 15 ביוני 2023 ב-16:26 מאת ‪Sruli Ganor (Jira)‬‏ <‪
jira@mariadb.org‬‏>:‬

Comment by Georg Richter [ 2023-06-18 ]

The transaction id is included in XID_EVENT, and it will be read in mariadb_rpl_fetch(): https://github.com/mariadb-corporation/mariadb-connector-c/blob/3.3/libmariadb/mariadb_rpl.c#L1501

Comment by Sruli Ganor [ 2023-06-18 ]

XID_EVENT appears after COMMIT of a *regular *(not XA) transaction.
We must know the transaction ID when the XA transaction starts (XA START).
In MySQL and in MariaDB 10.5, we can read the XA statements and extract the
transaction id (e.g., TXN1) so we can sort all the transaction's events by
its ID.

XA START 'TXN1';
insert into table1
insert into table2
XA END 'TXN1';
XA PREPARE 'TXN1';
XA COMMIT 'TXN1';

In MariaDB 10.6, I cannot read the XA START statement because
mariadb_rpl_fetch doesn't return it. I can only get the GTID. However, the
GTID is different for the XA START and XA COMMIT. E.g. in my case, XA START
has GTID 0-1-36 and the XA COMMIT is 0-1-37. Therefore, there is no common
ID that applies to all the transaction's events, and our application fails.

Hope it's clear. If not, we can talk since this is a critical issue for our
customers.
Thanks
Sruli

‫בתאריך יום א׳, 18 ביוני 2023 ב-18:12 מאת ‪Georg Richter (Jira)‬‏ <‪
jira@mariadb.org‬‏>:‬

Comment by Georg Richter [ 2023-06-19 ]

It looks like with implementation of MDEV-742 documentation was forgotten.
I've updated https://mariadb.com/kb/en/gtid_event/ and will push a fix.

Generated at Thu Feb 08 03:06:50 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.