Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-21295

SST should not remove ignore-db-dir directories

Details

    • Bug
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • 10.2.29, 10.3.20, 10.4.10
    • 10.4(EOL)
    • Galera SST
    • None

    Description

      wsrep_sst_mariabackup purges the joiners data directory before copying back backup data.

      A certain set of files is preserved, controlled by a set of regular expression terms in the "[sst] cpat" paramter.

      Datadir subdirectories not under mysqld control (like "lost+found" on mount points), and listed as such with ignore-db-dir=... are still going to be removed though.

      The directory names listed in ignore-db-dir options should be added to the sst:cpat ignore list.

      Attachments

        Issue Links

          Activity

            I had started a patch for this, this does not handle potential problems like e.g. whitespace in file names yet:

            diff --git a/scripts/wsrep_sst_mariabackup.sh b/scripts/wsrep_sst_mariabackup.sh
            index 59d2e12817b..134c250a515 100644
            --- a/scripts/wsrep_sst_mariabackup.sh
            +++ b/scripts/wsrep_sst_mariabackup.sh
            @@ -283,6 +283,21 @@ parse_cnf()
                 echo $reval
             }
             
            +parse_cnf_all()
            +{
            +    local group=$1
            +    local var=$2
            +    # print the default settings for given group using my_print_default.
            +    # normalize the variable names specified in cnf file (user can use _ or - for example log-bin or log_bin)
            +    # then grep for needed variable
            +    # finally get the variable value (if variables has been specified multiple time return all values separated by space)
            +    reval=$($MY_PRINT_DEFAULTS $group | awk -F= '{if ($1 ~ /_/) { gsub(/_/,"-",$1); print $1"="$2 } else { print $0 }}' | grep -- "--$var=" | cut -d= -f2- | sed -e's/"/\\"/g' -e's/^/"/g' -e's/$/"/g')
            +    if [[ -z $reval ]];then
            +        [[ -n $3 ]] && reval=$3
            +    fi
            +    echo $reval
            +}
            +
             get_footprint()
             {
                 pushd $WSREP_SST_OPT_DATA 1>/dev/null
            @@ -340,6 +355,15 @@ read_cnf()
                 ttime=$(parse_cnf sst time 0)
                 cpat=$(parse_cnf sst cpat '.*galera\.cache$\|.*sst_in_progress$\|.*\.sst$\|.*gvwstate\.dat$\|.*grastate\.dat$\|.*\.err$\|.*\.log$\|.*RPM_UPGRADE_MARKER$\|.*RPM_UPGRADE_HISTORY$')
                 [[ $OS == "FreeBSD" ]] && cpat=$(parse_cnf sst cpat '.*galera\.cache$|.*sst_in_progress$|.*\.sst$|.*gvwstate\.dat$|.*grastate\.dat$|.*\.err$|.*\.log$|.*RPM_UPGRADE_MARKER$|.*RPM_UPGRADE_HISTORY$')
            +    for ignore_dir in $(parse_cnf_all mysqld ignore-db-dir '')
            +    do
            +       ignore_dir=$(echo $ignore_dir | sed 's/[.[\*^$()+?{|]/\\&/g')
            +        if [ "$OS" = "FreeBSD" ];then
            +            cpat=$cpat'|.*/'$ignore_dir'$'
            +        else
            +            cpat=$cpat'\|.*/'$ignore_dir'$'
            +        fi
            +    done    
                 ealgo=$(parse_cnf xtrabackup encrypt "")
                 ekey=$(parse_cnf xtrabackup encrypt-key "")
                 ekeyfile=$(parse_cnf xtrabackup encrypt-key-file "")
            

            hholzgra Hartmut Holzgraefe added a comment - I had started a patch for this, this does not handle potential problems like e.g. whitespace in file names yet: diff --git a/scripts/wsrep_sst_mariabackup.sh b/scripts/wsrep_sst_mariabackup.sh index 59d2e12817b..134c250a515 100644 --- a/scripts/wsrep_sst_mariabackup.sh +++ b/scripts/wsrep_sst_mariabackup.sh @@ -283,6 +283,21 @@ parse_cnf() echo $reval } +parse_cnf_all() +{ + local group=$1 + local var=$2 + # print the default settings for given group using my_print_default. + # normalize the variable names specified in cnf file (user can use _ or - for example log-bin or log_bin) + # then grep for needed variable + # finally get the variable value (if variables has been specified multiple time return all values separated by space) + reval=$($MY_PRINT_DEFAULTS $group | awk -F= '{if ($1 ~ /_/) { gsub(/_/,"-",$1); print $1"="$2 } else { print $0 }}' | grep -- "--$var=" | cut -d= -f2- | sed -e's/"/\\"/g' -e's/^/"/g' -e's/$/"/g') + if [[ -z $reval ]];then + [[ -n $3 ]] && reval=$3 + fi + echo $reval +} + get_footprint() { pushd $WSREP_SST_OPT_DATA 1>/dev/null @@ -340,6 +355,15 @@ read_cnf() ttime=$(parse_cnf sst time 0) cpat=$(parse_cnf sst cpat '.*galera\.cache$\|.*sst_in_progress$\|.*\.sst$\|.*gvwstate\.dat$\|.*grastate\.dat$\|.*\.err$\|.*\.log$\|.*RPM_UPGRADE_MARKER$\|.*RPM_UPGRADE_HISTORY$') [[ $OS == "FreeBSD" ]] && cpat=$(parse_cnf sst cpat '.*galera\.cache$|.*sst_in_progress$|.*\.sst$|.*gvwstate\.dat$|.*grastate\.dat$|.*\.err$|.*\.log$|.*RPM_UPGRADE_MARKER$|.*RPM_UPGRADE_HISTORY$') + for ignore_dir in $(parse_cnf_all mysqld ignore-db-dir '') + do + ignore_dir=$(echo $ignore_dir | sed 's/[.[\*^$()+?{|]/\\&/g') + if [ "$OS" = "FreeBSD" ];then + cpat=$cpat'|.*/'$ignore_dir'$' + else + cpat=$cpat'\|.*/'$ignore_dir'$' + fi + done ealgo=$(parse_cnf xtrabackup encrypt "") ekey=$(parse_cnf xtrabackup encrypt-key "") ekeyfile=$(parse_cnf xtrabackup encrypt-key-file "")

            I forgot that for the longest time now database directory names are actually encoded, so special characters are not actually a problem, e.g. with space actually becomes with@020space, so the patch above should be fine as is.

            hholzgra Hartmut Holzgraefe added a comment - I forgot that for the longest time now database directory names are actually encoded, so special characters are not actually a problem, e.g. with space actually becomes with@020space , so the patch above should be fine as is.

            People

              sysprg Julius Goryavsky
              hholzgra Hartmut Holzgraefe
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.