Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-31399

mariabackup fails with large innodb redo log - block is overwritten

    XMLWordPrintable

Details

    • Bug
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • 10.5.17
    • None
    • mariabackup

    Description

      We had recent failure where mariabackup failed shortly after starting a full backup of a 1.7TB database.

      [01] 2023-06-04 02:16:20 Copying ./accounting/das_accounting_stats#P#p_202206.ibd to /mnt/local_db_backups/galera_cluster/20230604_full/accounting/das_accounting_stats#P#p_202206.ibd
      FILE_CHECKPOINT(18207486069808) ignored at 18208436535773
      FILE_CHECKPOINT(18207490734604) ignored at 18208443311562
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:27 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] 2023-06-04 02:16:28 Retrying read of log at LSN=18208449177088
      [00] FATAL ERROR: 2023-06-04 02:16:28 xtrabackup_copy_logfile() failed: redo log block is overwritten, please increase redo log size with innodb_log_file_size parameter.
      Jun 04 02:16:28 Backup return status: 1

      I saw some older issues referring to something similar and the suggestion was that redo log was too small. However our redo log is quite large (2 GB) and the backup is taken on a quiet node during off-peak times with little or no traffic expected so cannot imagine that this could be due to influx of transactions and mariabackup not being able to keep up.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              stephanvos Stephan Vos
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.