[MDEV-742] LP:803649 - Xa recovery failed on client disconnection - Jira

Details

Type: Task
Status: Closed (View Workflow)
Priority: Critical
Resolution: Fixed
Fix Version/s: 10.5.2
Component/s: OTHER
Labels:

Description

Dispite prepare phase have been rich you will get nothing when doing a xa recover if the client get disconnected befor commit

In the following php script an error is generated to force the COMMIT to failed cf:XA COMMITEE and close the connection.

<?php

$dt = date_create();

$xid = date_timestamp_get($dt);

echo "xid : ".$xid;

$logger1 = new mysqli('localhost', 'root', 'xxxxx', 'test');

if (mysqli_connect_error()) {

    die('Erreur de connexion (' . mysqli_connect_errno() . ') '. mysqli_connect_error());

$potest = new mysqli('192.168.45.166', 'test', 'xxxxxxx', 'test');

if (mysqli_connect_error()) {

    $logger1->close();

    die('Erreur de connexion (' . mysqli_connect_errno() . ') '. mysqli_connect_error());

echo 'SuccËs logger1 ' . $logger1->host_info . "\n";

echo 'SuccËs potest ' . $potest->host_info . "\n";

##################################

$queryXA = 'XA START "'.$xid.'"';

$queryRes = $logger1->query($queryXA);

if (!$queryRes) {

    $message  = 'RequÍte invalide logger1: ' . $logger1->error . "\n";

    $message .= 'RequÍte complËte : ' . $queryXA;

    $logger1->close();

    $potest->close();

    die($message);

$queryRes = $potest->query($queryXA);

if (!$queryRes) {

    $message  = 'RequÍte invalide potest: ' . $potest->error . "\n";

    $message .= 'RequÍte complËte : ' . $queryXA;

    $logger1->close();

    $potest->close();

    die($message);

##################################

##################################

$queryXA = 'INSERT INTO mthoxa(xamtho_tc) values ("'.$xid.'")';

$queryRes = $logger1->query($queryXA);

if (!$queryRes) {

    $message  = 'RequÍte invalide logger1: ' . $logger1->error . "\n";

    $message .= 'RequÍte complËte : ' . $queryXA;

    $logger1->close();

    $potest->close();

    die($message);

$queryRes = $potest->query($queryXA);

if (!$queryRes) {

    $message  = 'RequÍte invalide logger1: ' . $logger1->error . "\n";

    $message .= 'RequÍte complËte : ' . $queryXA;

    $logger1->close();

    $potest->close();

    die($message);

##################################

##################################

$queryXA = 'XA END "'.$xid.'"';

$queryRes = $logger1->query($queryXA);

if (!$queryRes) {

    $message  = 'RequÍte invalide logger1: ' . $logger1->error . "\n";

    $message .= 'RequÍte complËte : ' . $queryXA;

    $logger1->close();

    $potest->close();

    die($message);

$queryRes = $potest->query($queryXA);

if (!$queryRes) {

    $message  = 'RequÍte invalide potest: ' . $potest->error . "\n";

    $message .= 'RequÍte complËte : ' . $queryXA;

    $logger1->close();

    $potest->close();

    die($message);

##################################

##################################

$queryXA = 'XA PREPARE "'.$xid.'"';

$queryRes = $logger1->query($queryXA);

if (!$queryRes) {

    $message  = 'RequÍte invalide logger1: ' . $logger1->error . "\n";

    $message .= 'RequÍte complËte : ' . $queryXA;

    $logger1->close();

    $potest->close();

    die($message);

$queryRes = $potest->query($queryXA);

if (!$queryRes) {

    $message  = 'RequÍte invalide potest: ' . $potest->error . "\n";

    $message .= 'RequÍte complËte : ' . $queryXA;

    $logger1->close();

    $potest->close();

    die($message);

##################################

##################################

$queryXA = 'XA COMMIT "'.$xid.'"';

$queryRes = $logger1->query($queryXA);

if (!$queryRes) {

    $message  = 'RequÍte invalide logger1: ' . $logger1->error . "\n";

    $message .= 'RequÍte complËte : ' . $queryXA;

    $logger1->close();

    $potest->close();

    die($message);

$queryXA = 'XA COMMITEE "'.$xid.'"';

$queryRes = $potest->query($queryXA);

if (!$queryRes) {

    $message  = 'RequÍte invalide potest: ' . $potest->error . "\n";

    $message .= 'RequÍte complËte : ' . $queryXA;

    $queryXA = 'XA RECOVER';

    $queryRes = $potest->query($queryXA);

    while ($r=$queryRes->fetch_assoc())

    	print_r($r);

    $logger1->close();

    $potest->close();

    die($message);

##################################

$logger1->close();

$potest->close();

echo "Finish !\n"

?>

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

LPexportBug803649.xml
6 kB
2012-10-03 11:32

Issue Links

blocks

MDEV-21168 Active XA transactions stop slave from working after backup was restored.

Closed

MDEV-21854 xa commit 'xid' one phase for already prepared transaction must always error out

Closed

causes

MDEV-22733 XA PREPARE breaks MDL in pseudo_slave_mode=1

In Progress

MDEV-26652 xa transactions binlogged in wrong order

Closed

MDEV-26682 slave lock timeout with xa and gap locks

Closed

MDEV-29410 abort-and-replay prepared XA transactions on the slave

Open

MDEV-31949 slow parallel replication of user xa

Stalled

duplicates

MDEV-21304 kill session deletes XA prepared transaktion

Closed

is blocked by

MDEV-21659 XA rollback 'foreign_xid' is allowed inside active XA

Closed

MDEV-21856 XID_t::formatID has to be contrained to 4 byte size

Closed

is duplicated by

MDEV-7974 backport fix for mysql bug#12161 (XA and binlog)

Closed

relates to

MDEV-15532 XA: Assertion `!log->same_pk' failed in row_log_table_apply_delete

Closed

MDEV-21469 Implement crash-safe logging of the user XA

Stalled

MDEV-21766 Forbid XID with empty 'gtrid'

Closed

MDEV-21777 Implement crash-safe execution the user XA on binlog-less slave

Open

MDEV-25055 XA ROLLBACK reports ER_XAER_NOTA for another connection's XA PREPARE

Open

MDEV-29819 Shutdown unexpectedly executes XA ROLLBACK

Open

MDEV-32813 read-only xa prepare may disappear upon server restart

Open

MDEV-11675 Lag Free Alter On Slave

Closed

MDEV-21168 Active XA transactions stop slave from working after backup was restored.

Closed

MDEV-21602 CREATE TABLE…PRIMARY KEY…SELECT workaround causes DROP TABLE to ignore locks

Closed

MDEV-21644 Assertion `thd->transaction.xid_state.is_explicit_XA()' failed in binlog_commit_flush_trx_cache

Closed

MDEV-22445 Crash on HANDLER READ NEXT after XA PREPARE

Closed

MDEV-22656 Document what rollback-xa mariabackup option does

Closed

MDEV-25117 rpl.rpl_parallel_xa_same_xid failed in bb, sync_with_master failed:

Open

MDEV-25616 Binlog event for XA COMMIT is generated without matching XA START, replication aborts

Closed

MDEV-29642 Server Crash During XA Prepare Can Break Replication

Closed

MDEV-32020 XA transaction replicates incorrectly, must be applied at XA COMMIT, not XA PREPARE

Open

MDEV-33668 Adapt parallel slave's round-robin scheduling to XA events

Closed

MDEV-35019 Provide a way to enable "rollback XA on disconnect" behavior we had before 10.5.2

Stalled

links to

Bug #12161 - Xa recovery and client disconnection

(2 causes, 1 duplicates, 2 is blocked by, 1 is duplicated by, 19 relates to, 1 links to)

Activity

Ascending order - Click to sort in descending order

View 4 older comments

Sergei Petrunia added a comment - 2020-01-13 10:26 - edited

Take-aways from discussions in Frankfurt:

detach_prepared_tx() is ok
he transaction remains prepared inside RocksDB.
MyRocks' Rdb_transaction object is de-associated from RocksDB transaction and
is destroyed. This is fine.

replace_native_transaction_in_thd is redundant

The idea behind this method is that it is needed on the slave. A slave worker
may run the following sequence of operations:

XA BEGIN 'trx1'

... -- actions made by trx1.

XA END 'trx1';

XA PREPARE 'trx1';

-- at this point, all actions by trx1 has been done

-- the only two actions that are possible are commit or rollback.

XA BEGIN 'trx2';

... -- actions made by trx2

XA END 'trx2';

XA PREPARE 'trx2';

XA (COMMIT|ROLLBACK) 'trx1';

-- This is where we would want to "re-attach" to the prepared transaction.

The problem with re-attaching is that Rdb_transaction has a quite a bit of context about the transaction outside the m_rocksdb_tx member.
If one just replaces the m_rocksdb_tx, we end up with a mismatch between Rdb_transaction members.

It is much better if the SQL layer uses the existing API and calls

engine_hton->(commit|rollback)_by_xid(...);

There was another argument: both InnoDB and MyRocks try to re-use their internal transaction object. and this is why replace_trx_in_thd saves it away. And this is how we end up with method signatures like this:

+innodb_replace_trx_in_thd(

+       THD*    thd,

+       void*   new_trx_arg,

+       void**  ptr_trx_arg)

+{

My opinion is that this sort of caching should not be exposed through the storage engine API.

Sergei Petrunia added a comment - 2020-01-13 10:26 - edited Take-aways from discussions in Frankfurt: detach_prepared_tx() is ok he transaction remains prepared inside RocksDB. MyRocks' Rdb_transaction object is de-associated from RocksDB transaction and is destroyed. This is fine. replace_native_transaction_in_thd is redundant The idea behind this method is that it is needed on the slave. A slave worker may run the following sequence of operations: XA BEGIN 'trx1' ... -- actions made by trx1. XA END 'trx1' ; XA PREPARE 'trx1' ; -- at this point, all actions by trx1 has been done -- the only two actions that are possible are commit or rollback. XA BEGIN 'trx2' ; ... -- actions made by trx2 XA END 'trx2' ; XA PREPARE 'trx2' ; XA ( COMMIT | ROLLBACK ) 'trx1' ; -- This is where we would want to "re-attach" to the prepared transaction. The problem with re-attaching is that Rdb_transaction has a quite a bit of context about the transaction outside the m_rocksdb_tx member. If one just replaces the m_rocksdb_tx, we end up with a mismatch between Rdb_transaction members. It is much better if the SQL layer uses the existing API and calls engine_hton->(commit|rollback)_by_xid(...); There was another argument: both InnoDB and MyRocks try to re-use their internal transaction object. and this is why replace_trx_in_thd saves it away. And this is how we end up with method signatures like this: +innodb_replace_trx_in_thd( + THD* thd, + void* new_trx_arg, + void** ptr_trx_arg) +{ My opinion is that this sort of caching should not be exposed through the storage engine API.

Andrei Elkin added a comment - 2020-01-20 14:32 - edited

psergey, thanks for your notes!

I think you have a good point to replace_native_transaction_in_thd which is indeed to arrange a form
of caching which SE actually manages on its own anyway. To use the SE caching instead may not cost much performance wise for the slave. We may measure impact anyway, which I'll try to do.
I am fine to remove this extension meanwhile (and permanently if the measurement proves it's worth of that).

Andrei Elkin added a comment - 2020-01-20 14:32 - edited psergey , thanks for your notes! I think you have a good point to replace_native_transaction_in_thd which is indeed to arrange a form of caching which SE actually manages on its own anyway. To use the SE caching instead may not cost much performance wise for the slave. We may measure impact anyway, which I'll try to do. I am fine to remove this extension meanwhile (and permanently if the measurement proves it's worth of that).

Andrei Elkin added a comment - 2020-02-03 08:31

Howdy, Sergei!

Could you please check the patch. It's latest version is in bb-10.5-mdev_742.
I am also asking Marko, Sergei P., and Sergei Voitovich to take a look. Sergei V could take a close
look on usage of LF_HASH in the parallel slave code.

Thanks and cheers,

Andrei

Andrei Elkin added a comment - 2020-02-03 08:31 Howdy, Sergei! Could you please check the patch. It's latest version is in bb-10.5-mdev_742. I am also asking Marko, Sergei P., and Sergei Voitovich to take a look. Sergei V could take a close look on usage of LF_HASH in the parallel slave code. Thanks and cheers, Andrei

Marko Mäkelä added a comment - 2020-03-13 12:52

I do not think that the XA functionality can meaningfully be tested with DDL operations before ~~MDEV-15532~~ and ~~MDEV-21602~~ have been fixed.

Marko Mäkelä added a comment - 2020-03-13 12:52 I do not think that the XA functionality can meaningfully be tested with DDL operations before MDEV-15532 and MDEV-21602 have been fixed.

Sergey Vojtovich added a comment - 2020-03-13 15:09

marko, definitely. Although the problem is not new: recovery wouldn't recover MDL locks. This patch just extends the problem a bit further.

Sergey Vojtovich added a comment - 2020-03-13 15:09 marko , definitely. Although the problem is not new: recovery wouldn't recover MDL locks. This patch just extends the problem a bit further.

People

Assignee:: Andrei Elkin

Reporter:: Stephane VAROQUI (Inactive)

Votes:: 2 Vote for this issue

Watchers:: 20 Start watching this issue

Dates

Created:: 2011-06-29 21:17

Updated:: 2024-09-26 15:45

Resolved:: 2020-03-16 09:50

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB Server

Details

Description

Attachments

Attachments

Issue Links

Activity

People

Dates

Git Integration