Debarun Banerjee added a comment - 2024-11-21 07:26 In server backup can improve the backup design by allowing easier synchronization with other server activities going on. A BACKUP command to stream data to client has one drawback that it has to use a single socket to stream the data. A powerful alternative is CLONE feature that I had implemented for MySQL8.0. https://dev.mysql.com/doc/refman/8.4/en/clone-plugin.html Clone architecture uses a plugin that orchestrate the operation across multiple SEs and handles the network transfer for remote cloning. Clone uses server to server communication with the "Recipient" server establishing parallel connections to the "Donor" server for the data transfer. The dynamic snapshot creation for a specific engine is abstracted within the engine and currently implemented only for Innodb in MySQL. Clone can be conveniently used for replica provisioning in replication setup. It would be easy to integrate with Galera later as it already uses clone as the SST method for MySQL. https://dev.mysql.com/blog-archive/clone-create-mysql-instance-replica/ https://galeracluster.com/library/documentation/sst-physical.html Clone is also known to provide the best performance from internal and external testing mainly due to the advantage of in server synchronization and 3 phase snapshot strategy with in-memory page tracking. https://www.mydbops.com/blog/mysql-clone-plugin-speed-test Clone page tracking feature can be extended to provide persistent tracking logs and improve incremental backup. https://dev.mysql.com/blog-archive/innodb-clone-and-page-tracking/ A practical approach to implement clone like solution for Mariadb is to first port the clone feature from MySQL. This would require some effort to integrate with Mariadb We can then extend it to provide cloning for all supported storage engines. We can then incrementally add features like incremental clone and partial clone. At this point clone solution in Mariadb would be much more advanced than what is there in MySQL.

Debarun Banerjee added a comment - 2024-12-09 06:55 - edited [Done] 3. Port Server changes: Syntax, Clone Statement, Clone handler to interface with plugin. INSTALL PLUGIN clone SONAME 'mariadb_clone.so'; UNINSTALL PLUGIN clone; Clone syntax works the flow reaches clone plugin and returns from SE interfaces [NO OP] right now. CLONE LOCAL DATA DIRECTORY [=] 'clone_dir'; CLONE INSTANCE FROM 'user'@'host':port IDENTIFIED BY 'password' [DATA DIRECTORY [=] 'clone_dir'] [REQUIRE [NO] SSL]; Started working on Innodb changes for clone. [In Progress] 4. Port Innodb SE changes for snapshot.