[MDEV-31949] slow parallel replication of user xa - Jira

Details

Type: New Feature
Status: Stalled (View Workflow)
Priority: Critical
Resolution: Unresolved
Fix Version/s: 12.1
Component/s: Replication
Labels:
None

Description

With some type of load, such as one statement "light" transactions, the user xa may perform much poorer on the optimistic parallel slave than an equivalent normal - BEGIN..COMMIT - load.

This ticket aims at fixing the slowness while the XA high-availability is to be preserved.

The whole work is arranged in two parts:
1. refactoring of the user xa binary logging to facilitate slave side parallel execution
MDEV-32830;
2. the slave side change of the parallel scheduler to apply Round-Robin distribution
for the user XA transactions.

Attachments

Issue Links

causes

MDEV-32257 Assertion `thd->is_killed()' failed in XID_cache_element::acquired_to_recovered (opt) or acquire_xid (dbg) on XA ROLLBACK or XA COMMIT

Closed

MDEV-32347 Stack smashing/looping, ASAN use-after-poison in xid_t::eq/event_xid_t::serialize, SIGSEGV in serialize_xid and Assertion `is_async_xac || thd->lex->xid->eq(thd->transaction->xid_state.get_xid())' failed in binlog_rollback_flush_trx_cache upon LOAD INDEX

Closed

MDEV-32463 SIGSEGV in __memmove_avx_unaligned_erms from a memcpy in xid_t::set (sql/handler.h:896) from Gtid_log_event::Gtid_log_event

Closed

MDEV-32469 MDEV-31949: MTR outcome difference: MTR testcase from MDEV-27512 crashes in BASE but not in PATCH

Closed

MDEV-32470 MDEV-31949: use-after-poison in xid_t::key_length()

Closed

MDEV-33673 Parallel replica threads lockup when using many XA threads in bb-10.6-MDEV-31949_ver0

Closed

is blocked by

MDEV-32257 Assertion `thd->is_killed()' failed in XID_cache_element::acquired_to_recovered (opt) or acquire_xid (dbg) on XA ROLLBACK or XA COMMIT

Closed

MDEV-32272 lock_release_on_prepare_try() does not release lock if supremum bit is set along with other bits set in lock's bitmap

Closed

Closed

MDEV-32470 MDEV-31949: use-after-poison in xid_t::key_length()

Closed

MDEV-32830 refactor XA binlogging for better integration with BGC/replication/recovery

In Progress

MDEV-32852 Assertion `is_prepared_xa(thd) || ((thd->lex->sql_command == SQLCOM_XA_ROLLBACK && thd->transaction->xid_state.get_state_code() == XA_IDLE) || thd->lex->xa_opt == XA_ONE_PHASE)' failed in TC_LOG::run_commit_ordered

Closed

MDEV-32857 Assertion `thd->lex->sql_command == SQLCOM_XA_COMMIT || thd->lex->sql_command == SQLCOM_XA_ROLLBACK' failed in TC_LOG::run_commit_ordered

Closed

MDEV-33673 Parallel replica threads lockup when using many XA threads in bb-10.6-MDEV-31949_ver0

Closed

is caused by

MDEV-742 LP:803649 - Xa recovery failed on client disconnection

Closed

relates to

MDEV-33168 XA crash-recovery base on engines prepare first rule

In Progress

MDEV-21469 Implement crash-safe logging of the user XA

Stalled

MDEV-30165 X-lock on supremum for prepared transaction for RR

Closed

MDEV-32455 Implicit rollback by LOAD INDEX or CACHE INDEX should not touch prepared user XA

Confirmed

MDEV-32463 SIGSEGV in __memmove_avx_unaligned_erms from a memcpy in xid_t::set (sql/handler.h:896) from Gtid_log_event::Gtid_log_event

Closed

MDEV-32830 refactor XA binlogging for better integration with BGC/replication/recovery

In Progress

split to

MDEV-33667 strict round-robin scheduling of XA events to improve parallel slave performance

Closed

MDEV-33668 Adapt parallel slave's round-robin scheduling to XA events

Closed

(1 causes, 8 is blocked by, 1 is caused by, 6 relates to, 2 split to)

Activity

Ascending order - Click to sort in descending order

View 25 older comments

Andrei Elkin added a comment - 2024-01-06 15:46

masonmariadb, the whole work includes now the IVth recovery commit. Sergei is reviewing the general binlogging part I. split from this ticket as MDEV-32830, which has been refined to reflect the recovery concern raised by Kristian.
The part II is on Kristian, I'll notify him once the part IV is finally readied for review by Sergei and him (ETA: Mon).
The part III is about few Innodb changes that Marko needs look at, also when the IV is out.

Andrei Elkin added a comment - 2024-01-06 15:46 masonmariadb , the whole work includes now the IVth recovery commit. Sergei is reviewing the general binlogging part I. split from this ticket as MDEV-32830 , which has been refined to reflect the recovery concern raised by Kristian. The part II is on Kristian, I'll notify him once the part IV is finally readied for review by Sergei and him (ETA: Mon). The part III is about few Innodb changes that Marko needs look at, also when the IV is out.

Kristian Nielsen added a comment - 2024-01-07 16:55

I've started implementing MDEV-32020 to fix the XA stuff properly.

Kristian Nielsen added a comment - 2024-01-07 16:55 I've started implementing MDEV-32020 to fix the XA stuff properly.

Andrei Elkin added a comment - 2024-01-08 20:09 - edited

The branch is force-updated

ff8c19e7b64...82aedbe9213 HEAD -> bb-10.6-MDEV-31949 (forced update)

to preserve its structure of four parts.
Part I,II,III remain with In-Review status, the part IV needs incorporation with a Xid_log_list_event piece being worked on by Brandon.

knielsen, 82aedbe9213 combined with the upcoming Xid_log_list_event must be a recovery solution that you prefer.
Feel free to check it soon.
It am sure it should not hold you on anymore for reviewing the part II.

I'd be really glad to discuss your plan of implementation of xa-prepare deferred execution that you must've been about in the MDEV-32020 c omment.
Having your design presented may help me to understand what you mean in 'proper'. I pointed it a number of times already that
that bug case itself relates to the XA replication framework rather remotely. Much more tightly it deals with freaking specifics that unique key absent Rows events have manifested in various ways. It is resolvable within few hours by standard means. While
the deferring XA transaction execution may be a solution too, I do feel uncomfortable about that, especially when it's rather clear that the deferring method must ensue a number of design hurdles esp recovery related.

Andrei Elkin added a comment - 2024-01-08 20:09 - edited The branch is force-updated ff8c19e7b64...82aedbe9213 HEAD -> bb-10.6-MDEV-31949 (forced update) to preserve its structure of four parts. Part I,II,III remain with In-Review status, the part IV needs incorporation with a Xid_log_list_event piece being worked on by Brandon. knielsen , 82aedbe9213 combined with the upcoming Xid_log_list_event must be a recovery solution that you prefer. Feel free to check it soon. It am sure it should not hold you on anymore for reviewing the part II. I'd be really glad to discuss your plan of implementation of xa-prepare deferred execution that you must've been about in the MDEV-32020 c omment. Having your design presented may help me to understand what you mean in 'proper'. I pointed it a number of times already that that bug case itself relates to the XA replication framework rather remotely. Much more tightly it deals with freaking specifics that unique key absent Rows events have manifested in various ways. It is resolvable within few hours by standard means. While the deferring XA transaction execution may be a solution too, I do feel uncomfortable about that, especially when it's rather clear that the deferring method must ensue a number of design hurdles esp recovery related.

Andrei Elkin added a comment - 2024-01-09 20:36 - edited

Updated the branch in the IV recovery part/HEAD of MDEV-33168:

82aedbe9213...a71f13489d0 HEAD -> bb-10.6-MDEV-31949 (forced update)

The current HEAD is enriched with a number of mtr tests and fixes few glitches in implementation of business logics.
Integration with Xid_log_list_event is scheduled for tomorrow.

Andrei Elkin added a comment - 2024-01-09 20:36 - edited Updated the branch in the IV recovery part/HEAD of MDEV-33168 : 82aedbe9213...a71f13489d0 HEAD -> bb-10.6-MDEV-31949 (forced update) The current HEAD is enriched with a number of mtr tests and fixes few glitches in implementation of business logics. Integration with Xid_log_list_event is scheduled for tomorrow.

Andrei Elkin added a comment - 2024-01-22 08:43

Howdy julien.fritsch!

The whole issue is of 3 parts:
1. refactoring of XA binloggin (MDEV-32830)
2. parallel slave XA execution performance
3. XA binlog-recovery (MDEV-33168)

The first two were ready for review for some time. serg ] has provided some notes to the part I. There must be more to come after my reply about a week ago. knielsen did so to the part II of a previous version of the patch and those of his notes have been accounted in the current branch. He also strongly suggested to change the part 3's plan from the ~~MDEV-21649~~ method to the MDEV-33168 one.
The latter p.3 has been under my testing for few last days. I am about to publish it for review and QA preliminary testing: ETA 1-2 days.

Andrei Elkin added a comment - 2024-01-22 08:43 Howdy julien.fritsch ! The whole issue is of 3 parts: 1. refactoring of XA binloggin ( MDEV-32830 ) 2. parallel slave XA execution performance 3. XA binlog-recovery ( MDEV-33168 ) The first two were ready for review for some time. serg ] has provided some notes to the part I. There must be more to come after my reply about a week ago. knielsen did so to the part II of a previous version of the patch and those of his notes have been accounted in the current branch. He also strongly suggested to change the part 3's plan from the MDEV-21649 method to the MDEV-33168 one. The latter p.3 has been under my testing for few last days. I am about to publish it for review and QA preliminary testing: ETA 1-2 days.

People

Assignee:: Andrei Elkin

Reporter:: Andrei Elkin

Votes:: 1 Vote for this issue

Watchers:: 15 Start watching this issue

Dates

Created:: 2023-08-18 06:44

Updated:: Yesterday 13:44

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB Server