Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-6771

Incorrect Size for Transfer Reported to pv

    Details

      Description

      The function that is used by the xtrabackup and xtrabackup-v2 SST scripts to get the size of the backup (get_footprint) can return an incorrect size. From what I've been able to find, it is because xargs batches data above a certain threshold. The issue is in this line:

      payload=$(find . -regex '.*\.ibd$\|.*\.MYI$\|.*\.MYD$\|.*ibdata1$' -type f -print0 | xargs -0 du --block-size=1 -c | awk 'END { print $1 }')

      The batching from xargs can cause du to print multiple total lines. awk then grabs the last one so the reported size is just the size of the files from the last batch from xargs. One option is to not use xargs and instead catch the piped data with --files0-from=-, but get_footprint from du can use file name patterns which is simpler and worked for me. I'm not sure if it will work with other versions of du, though.

      without find:

      payload=$(du --block-size=1 -c  **/*.ibd **/*.MYI **/*.MYD ibdata1  | awk 'END { print $1 }')

      using find:

      payload=$(find . -regex '.*\.ibd$\|.*\.MYI$\|.*\.MYD$\|.*ibdata1$' -type f -print0 | du --files0-from=- --block-size=1 -c | awk 'END { print $1 }')

      Also, since the innodb_data_file_path can be changed, it would be helpful to have the script find the location of the data file and use that path instead of assuming it's in the work directory with everything else.

        Attachments

          Activity

            People

            • Assignee:
              nirbhay_c Nirbhay Choubey (Inactive)
              Reporter:
              bradjorgensen Brad Jorgensen
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: