Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-17029

mariabackup fails under load with schema changes



    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Incomplete
    • 10.2.16
    • N/A
    • mariabackup
    • None
    • Ubuntu 18:04, Ubuntu 16:04, Galera cluster


      When we run mariabackup on a galera cluster nodes with a lot of data/databases (over 1500 databases with 700GB of data at the moment), if the servers are under moderate use mariabackup tool fails with the following log messages:

      Aug 21 17:44:51 maria3 -innobackupex-backup[25752]: 2018-08-21 17:44:51 140002715002816 [Note] InnoDB: Read redo log up to LSN=1691316337664
      Aug 21 17:44:51 maria3 -innobackupex-backup[25752]: mariabackup: Error: xtrabackup_copy_logfile() failed.
      Aug 21 17:44:52 maria3 -wsrep-sst-donor[25825]: mariabackup finished with error: 1.  Check /data/maria//innobackup.backup.log
      Aug 21 17:44:52 maria3 -wsrep-sst-donor[25826]: Cleanup after exit with status:22

      The cluster is under load, but I think these are happening when schema is being modified (databases created, dropped and so on). It could be related to tables flooded with records, because when we create databases we populate them with a lot of records.

      This happens both when running mariabackup manually or when it's triggered by SST.

      The log file mentioned in the message "innobackup.backup.log" cannot be found anywhere on the system. So we have no more info about the error.

      Unfortunately, this is a production cluster and we can't do much in a way of debugging on these machines.


        Issue Links



              wlad Vladislav Vaintroub
              kvasserman Konstantin Vasserman
              0 Vote for this issue
              5 Start watching this issue



                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.