Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-6771

Incorrect Size for Transfer Reported to pv

    XMLWordPrintable

Details

    Description

      The function that is used by the xtrabackup and xtrabackup-v2 SST scripts to get the size of the backup (get_footprint) can return an incorrect size. From what I've been able to find, it is because xargs batches data above a certain threshold. The issue is in this line:

      payload=$(find . -regex '.*\.ibd$\|.*\.MYI$\|.*\.MYD$\|.*ibdata1$' -type f -print0 | xargs -0 du --block-size=1 -c | awk 'END { print $1 }')

      The batching from xargs can cause du to print multiple total lines. awk then grabs the last one so the reported size is just the size of the files from the last batch from xargs. One option is to not use xargs and instead catch the piped data with --files0-from=-, but get_footprint from du can use file name patterns which is simpler and worked for me. I'm not sure if it will work with other versions of du, though.

      without find:

      payload=$(du --block-size=1 -c  **/*.ibd **/*.MYI **/*.MYD ibdata1  | awk 'END { print $1 }')

      using find:

      payload=$(find . -regex '.*\.ibd$\|.*\.MYI$\|.*\.MYD$\|.*ibdata1$' -type f -print0 | du --files0-from=- --block-size=1 -c | awk 'END { print $1 }')

      Also, since the innodb_data_file_path can be changed, it would be helpful to have the script find the location of the data file and use that path instead of assuming it's in the work directory with everything else.

      Attachments

        Activity

          People

            nirbhay_c Nirbhay Choubey (Inactive)
            bradjorgensen Brad Jorgensen
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.