Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Incomplete
-
10.2.16
-
None
-
Ubuntu 18:04, Ubuntu 16:04, Galera cluster
Description
When we run mariabackup on a galera cluster nodes with a lot of data/databases (over 1500 databases with 700GB of data at the moment), if the servers are under moderate use mariabackup tool fails with the following log messages:
Aug 21 17:44:51 maria3 -innobackupex-backup[25752]: 2018-08-21 17:44:51 140002715002816 [Note] InnoDB: Read redo log up to LSN=1691316337664
|
Aug 21 17:44:51 maria3 -innobackupex-backup[25752]: mariabackup: Error: xtrabackup_copy_logfile() failed.
|
Aug 21 17:44:52 maria3 -wsrep-sst-donor[25825]: mariabackup finished with error: 1. Check /data/maria//innobackup.backup.log
|
Aug 21 17:44:52 maria3 -wsrep-sst-donor[25826]: Cleanup after exit with status:22
|
The cluster is under load, but I think these are happening when schema is being modified (databases created, dropped and so on). It could be related to tables flooded with records, because when we create databases we populate them with a lot of records.
This happens both when running mariabackup manually or when it's triggered by SST.
The log file mentioned in the message "innobackup.backup.log" cannot be found anywhere on the system. So we have no more info about the error.
Unfortunately, this is a production cluster and we can't do much in a way of debugging on these machines.
Attachments
Issue Links
- relates to
-
-
- Stalled
-
-
MDEV-16791 mariabackup : allow consistent backup, in presence of concurrent DDL, also without --lock-ddl-per-table
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-