[MDEV-7258] Read free replication for TokuDB Created: 2014-12-03  Updated: 2021-07-07  Resolved: 2021-07-07

Status: Closed
Project: MariaDB Server
Component/s: Replication, Storage Engine - TokuDB
Fix Version/s: N/A

Type: Task Priority: Major
Reporter: Rich Prohaska Assignee: Sergei Golubchik
Resolution: Won't Fix Votes: 6
Labels: contribution, foundation

Issue Links:
Relates
relates to MDEV-10976 Port MyRocks' Read Free Replication t... Open

 Description   

Read free replication avoids unnecessary row reads on the replication slave when executing write, update, or delete events. This can significantly increase the throughput of a replication slave that was previously I/O bound.

The feature is described here:
https://github.com/Tokutek/tokudb-engine/wiki/Read-Free-Replication-with-TokuDB

The benefit is described here:
http://www.tokutek.com/2014/09/tokudb-v7-5-read-free-replication-the-benchmark/

A patch for MariaDB 10.0.15:
https://github.com/Tokutek/mariadb-10.0/commit/9992195def7e739b4034902b478ee1b448b0e21a



 Comments   
Comment by Elena Stepanova [ 2014-12-03 ]

I've set the target version to 10.0 because that's what the patch is for; not sure however if it can go to the stable release.

Comment by Yoshinori Matsunobu [ 2015-01-16 ]

I'm interested in this feature but have a question about handler api changes (https://github.com/Tokutek/mariadb-10.0/commit/9992195def7e739b4034902b478ee1b448b0e21a#diff-8c78bfc758be72b7df2306f1d9984c47R12217).

Why are below functions needed?

+ virtual void rpl_before_write_rows() {
+ }
+ virtual void rpl_after_write_rows() {
+ }
+ virtual void rpl_before_delete_rows() {
+ }
+ virtual void rpl_after_delete_rows() {
+ }
+ virtual void rpl_before_update_rows() {
+ }
+ virtual void rpl_after_update_rows() {
+ }

I assume what needs to be done is skipping reads for update/delete row events at Rows_log_event::find_row(), because BI (before image) exists.
Instead of just calling rpl_lookup_rows(), how about checking binlog event types here like this?
current patch:

+ if (!table->file->rpl_lookup_rows())
+   DBUG_RETURN(0);

proposed patch:

+ uint event_type= this->get_type_code();
+  if ((event_type == UPDATE_ROWS_EVENT || event_type == DELETE_ROWS_EVENT) &&
+    !table->file->rpl_lookup_rows())
+ DBUG_RETURN(0);

(and removing all rpl_before/after_write/update/delete* handler apis)

Comment by Rich Prohaska [ 2015-01-18 ]

The replication slave does extra reads in the replication event execution layer and in the storage engine. The reads in the replication layer are there I assume to detect table mismatch between the master and the slave. We think that this check has too much overhead. The reads in the storage engine for write, delete, and update replication events occur to enforce any unique key constraints.

We added the rpl_lookup_rows handler function to avoid the reads in the replication layer.

We added the other handler functions to avoid extra reads in the storage engine. We wanted to know that the replication layer is executing a write rows, delete rows, or update rows replication event.

I suspect that the storage engine handler could maintain a one shot flag which gets set in the rpl_lookup_rows function and used in subsequent calls to the handler write_row, delete_row, and update_row functions. This could eliminate the need for these extra handler functions.

Comment by Sergei Golubchik [ 2015-02-25 ]

What is rpl_lookup_rows() supposed to do? I will need to document all API functions.

A one shot flag is a good idea. But as far as I can see rpl_lookup_rows() isn't called for Write_rows_log_event. Or is it?

Comment by Rich Prohaska [ 2015-03-06 ]

I added comments to the code on my 10.0-rfr branch. See https://github.com/Tokutek/mariadb-server/commit/0e5f4a2bd740502814bdfc652bdd3d4ec08112ef

Comment by jocelyn fournier [ 2016-07-28 ]

Hi Sergei,

Any update on this one ? It's quite disturbing to have the variable available in MariaDB, but no effect in practice!

Thanks,
Jocelyn

Comment by Sergei Golubchik [ 2016-08-02 ]

No, unfortunately, not yet. I'll raise the priority of this task

Comment by Rich Prohaska [ 2016-08-02 ]

Updated the RFR patch for MariaDB 10.1. Running basic tests on this branch: https://github.com/prohaska7/mariadb-server/tree/mdev-7258

Comment by jocelyn fournier [ 2017-03-01 ]

The 2016-10-13 sprint seems to be quite late

Generated at Thu Feb 08 07:18:14 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.