paralell-backup.png
|
Relates |
|||
| relates to |
|
mysqldump: use --tab (or another opti... | Closed |
[MDEV-12808] Include mysqlpump parallel backup tool Created: 2017-05-15 Updated: 2023-11-30 |
|
| Status: | Open |
| Project: | MariaDB Server |
| Component/s: | Backup |
| Fix Version/s: | None |
| Type: | New Feature | Priority: | Minor |
| Reporter: | Robert Goley | Assignee: | Unassigned |
| Resolution: | Unresolved | Votes: | 11 |
| Labels: | Client-Tools | ||
| Attachments: |
paralell-backup.png
|
||||||||
| Issue Links: |
|
||||||||
| Description |
|
MySQL now includes a faster backup tool using parallel threads for backup processes in addition to the single threaded mysqldump utility. Now that this tool has transactional consistency, It would be a great benefit for it to be added to MariaDB's default client toolset. Here are links referencing the transactional consistency issue being resolved: http://www.tocker.ca/2015/08/03/mysql-5-7-8-mysqlpump-caveat.html The existing mysqlpump binary in the MySQL or Percona releases fails (trying to connect to MariaDB 10.1.23) with an error message that server version 5.7.8 or later is required. While most of the work is done by the client utility, I am sure this means that there are some server related hooks that must be in place or that the tool wants to ensure are available before executing. Inclusion of this new tool would definitely require reviewing the server hooks vs a simple code recompile before this could be released. It would be a huge improvement on current tools and ensure that MySQL/Percona users would be able to migrate to MariaDB with existing programs/processes largely unmodified though. |
| Comments |
| Comment by akrus [ 2017-11-22 ] |
|
It would be great to have it. mydumper has a bug so we cannot use it and likely no other reliable alternative is available. |
| Comment by Slawomir Pryczek [ 2017-11-22 ] |
|
Hi Guys, im aware of some issues with these paralell backup utilities, so i wrote my own one in golang. It's using mysqldump and optionally pigz to do multithreaded backup and multithreaded compression alongside the backup. For now it has ability to sort tables and do some optimization based on tablesize so backup takes as short as possible, produces one file per table so you can easily do multithreaded import too... Not sure if anyone's interested but i can put it on github as a temporary workaround if you interested. Can shorten backup times from days to hours in linux... |
| Comment by Manjot Singh (Inactive) [ 2018-11-29 ] |
|
Instead of working with mysqlpump, could we possibly just fork mydumper into a mariadb tool? |
| Comment by Kurt Martinsen [ 2018-11-30 ] |
|
Only just found out about mysqlpump and was disappointed to learn that it cannot be used with MariaDB. If indeed there is no interest by the MariaDB team to include the tool, what is the recommended way to do parallel logical backups with MariaDB? |
| Comment by Manjot Singh (Inactive) [ 2019-06-13 ] |
|
Could this be solved in 10.4+ with mysqldump using backup locks? @kurt.martinsen look into the open source tool mydumper. |
| Comment by Daniel Black [ 2021-06-17 ] |
|
note mysqldumper development has resumed https://github.com/maxbube/mydumper |
