[MDEV-12179] Per-engine mysql.gtid_slave_pos tables Created: 2017-03-06  Updated: 2018-02-13  Resolved: 2017-07-03

Status: Closed
Project: MariaDB Server
Component/s: Replication
Fix Version/s: 10.3.1

Type: Task Priority: Critical
Reporter: Kristian Nielsen Assignee: Kristian Nielsen
Resolution: Fixed Votes: 1
Labels: gtid, replication

Issue Links:
Relates
relates to MDEV-11657 Cross-engine transaction metadata Open
relates to MDEV-13236 Document MDEV-12179 (Per-engine mysql... Closed
Sprint: 10.3.1-1

 Description   

Implement that server reads/updates the GTID position using multiple tables
mysql.gtid_slave_pos_XXX, each using a different storage engine. Replicated
transactions update the version of the table in the same engine, if
available. This avoids the overhead of cross-engine transactions on servers
where multiple storage engines are in use at the same time (but not in the
same transactions).

See mailing list thread: https://lists.launchpad.net/maria-developers/msg10453.html



 Comments   
Comment by Marko Mäkelä [ 2017-03-09 ]

MDEV-11657 suggests an alternative approach (with a much wider scope): storing the transaction state in a global table. This would require substantial changes to transactional storage engines, but it would avoid the need of using binlog’s XA 2PC mechanism for multi-engine transactions.
With MDEV-11657, persistent transactions would be registered in only one place, with GTID, XID and everything in a single place.

Comment by Kristian Nielsen [ 2017-03-09 ]

Reading in the associated MDEV-11655:

"There will be a common redo log for and all crash-safe storage engines."

The lack of a common redo log indeed is the source of endless amounts of
complexity and performance penalties, in replication and probably elsewhere
also. Resolving this would be very interesting, (and of much wider scope, as
you said).

The main point of per-engine mysql.gtid_slave_pos table is to allow
replication transactional commits with just one transactional log (if
--log-slave-updates=0). So if my understanding is correct, common redo log
would be a pre-requisite for MDEV-11657 to be a full alternative to this
feature?

It was interesting to see the MDEV-11657 and related designs, I have not
seen any mention of them before.

Comment by Sergei Petrunia [ 2017-06-12 ]

An interesting property of the current patch:

If one does

 alter table mysql.gtid_slave_pos engine=innobase;

(suppose the above changes the storage engine of gtid_slave_pos from MyISAM to InnoDB)
then Transactions_gtid_foreign_engine counter will continue to be incremented as if ALTER TABLE didn't happen. STOP SLAVE ; START SLAVE fixes the issue.

Comment by Sergei Petrunia [ 2017-06-13 ]

Another thing to investigate: I have set

gtid_pos_auto_engines='Aria,HEAP,TokuDB';

and I can observe gtid_slave_pos_TokuDB table to be created when a transaction involving a TokuDB table is replayed on the slave.

However, I dont observe a gtid_slave_pos_Aria table to be created when a transaction with Aria table is replicated. The Aria table does have TRANSACTIONAL=1 flag.

Comment by Kristian Nielsen [ 2017-06-13 ]

Aria (or HEAP, or MyISAM) is not transactional, in the sense that it does
not keep a transaction open.

So after an event has been executed by the slave, and the COMMIT event runs,
there is no open transaction involving the Aria storage engine. Therefore,
record_gtid() cannot choose an Aria table. Nor would there be any benefit,
since whatever is done, record_gtid() needs to commit a new transaction.

One could make the default mysql.gtid_slave_pos table use Aria, if desired.

Comment by Sergei Petrunia [ 2017-06-26 ]

knielsen, thanks for clarification

Comment by Sergei Petrunia [ 2017-06-26 ]

Ok I have no issues with the patch (Review Done), except the question of how to package this, which I have described here: https://lists.launchpad.net/maria-developers/msg10779.html .

Comment by Sergei Petrunia [ 2017-06-26 ]

Merged the code to MariaDB 10.3 (not automatic but nothing complex): https://github.com/MariaDB/server/tree/bb-10.3-mdev12179

Comment by Kristian Nielsen [ 2017-07-03 ]

Pushed to 10.3.1.

For now, the default for --gtid-pos-auto-engines is empty. This can be easily changed if desired, just update the value in sql/mysqld.cc.

Comment by Jean-François Gagné [ 2018-02-13 ]

Leaving some links about this work here:

Generated at Thu Feb 08 07:55:44 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.