A user is facing this scenario where mariabackup --prepare has been running for 12+ hours, and it's reporting page numbers to recover. Then it goes on to multiple iterations of recovery with no indication of how much work is remaining or how much time remains to complete the process.
It will be great if `--prepare` can report the percentage of pages recovery completed per iteration and a global percentage completion consolidating all iteration batches.
There is not enough time to test this thoroughly before the upcoming quarterly releases.
With the current fix that I have for 10.6 (and which might be feasible to port to 10.5 as well), some excessive or unnecessary log scanning will take place. When the buffer pool is small, the log reads will basically be O(n²) instead of O. This will be better in the version that uses the MDEV-14425 log format.
Marko Mäkelä
added a comment - There is not enough time to test this thoroughly before the upcoming quarterly releases.
With the current fix that I have for 10.6 (and which might be feasible to port to 10.5 as well), some excessive or unnecessary log scanning will take place. When the buffer pool is small, the log reads will basically be O(n²) instead of O . This will be better in the version that uses the MDEV-14425 log format.
I am currently working around the slow prepare of our 1.7TB database when doing manual backup and restore by specifying Memory:
sudo mariabackup --prepare --use-memory=464G
But the problem is that when a SST happens as part of normal recovery it uses the default 100MB for use-memory which then results in hours and hours of prep.
Is it not possible to set --use-memory somewhere in Configuration so that SST recovery script will use this?
Stephan Vos
added a comment - - edited I am currently working around the slow prepare of our 1.7TB database when doing manual backup and restore by specifying Memory:
sudo mariabackup --prepare --use-memory=464G
But the problem is that when a SST happens as part of normal recovery it uses the default 100MB for use-memory which then results in hours and hours of prep.
Is it not possible to set --use-memory somewhere in Configuration so that SST recovery script will use this?
stephanvos, can you please file a separate bug against wsrep_sst_mariabackup.sh so that this parameter can be specified? It is also thinkable that mariadb-backup --prepare would read the use_memory parameter from an appropriate section of a configuration file.
Marko Mäkelä
added a comment - stephanvos , can you please file a separate bug against wsrep_sst_mariabackup.sh so that this parameter can be specified? It is also thinkable that mariadb-backup --prepare would read the use_memory parameter from an appropriate section of a configuration file.
From: Marko Mäkelä (Jira) <jira@mariadb.org>
Sent: Friday, 20 December 2024 07:43
To: Stephan Vos <stephan.vos@metacom.net>
Subject: [JIRA] Marko Mäkelä mentioned you on MDEV-29911 InnoDB recovery and mariadb-backup --prepare fail to report detailed progress (Jira)
Marko Mäkelä mentioned you on MDEV-29911
--------------------------------
stephanvos, can you please file a separate bug against wsrep_sst_mariabackup.sh so that this parameter can be specified? It is also thinkable that mariadb-backup --prepare would read the use_memory parameter from an appropriate section of a configuration file.
–
This message was sent by Atlassian Jira
(v8.20.16#820016)
Stephan Vos
added a comment - Thanks I will do so.
From: Marko Mäkelä (Jira) <jira@mariadb.org>
Sent: Friday, 20 December 2024 07:43
To: Stephan Vos <stephan.vos@metacom.net>
Subject: [JIRA] Marko Mäkelä mentioned you on MDEV-29911 InnoDB recovery and mariadb-backup --prepare fail to report detailed progress (Jira)
[ https://jira.mariadb.org/browse/MDEV-29911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel < https://jira.mariadb.org/browse/MDEV-29911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel > ]
Marko Mäkelä mentioned you on MDEV-29911
--------------------------------
stephanvos , can you please file a separate bug against wsrep_sst_mariabackup.sh so that this parameter can be specified? It is also thinkable that mariadb-backup --prepare would read the use_memory parameter from an appropriate section of a configuration file.
–
This message was sent by Atlassian Jira
(v8.20.16#820016)
Bug
Critical
MDEV-14481 Execute InnoDB crash recovery in the background
There is not enough time to test this thoroughly before the upcoming quarterly releases.
With the current fix that I have for 10.6 (and which might be feasible to port to 10.5 as well), some excessive or unnecessary log scanning will take place. When the buffer pool is small, the log reads will basically be O(n²) instead of O
. This will be better in the version that uses the
MDEV-14425log format.