Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-32344

IST "Donor does not know my secret" with ssl-mode=VERIFY_CA

Details

    Description

      Related to MDEV-32342, it seems that the remote SECRET is not set for ISTs when VERIFY_CA mode is on.

      This results in:

      WSREP_SST: [ERROR] Donor does not know my secret! (20231003 15:29:10.448)
      WSREP_SST: [INFO] Donor: '', my: 'd9ca9b998550fafb64c0ccc822dde463' (20231003 15:29:10.449)
      

      After this failed IST, a restart triggers an SST, which then (luckily) succeeds.

      The cause appears to be this changeset:

      commit 1ae7673aae7f82c4e659b1337177f2696c8e45ba (origin/bb-10.2-MDEV-24962-final)
      Author: Julius Goryavsky <julius.goryavsky@mariadb.com>
      Date:   Wed Apr 28 01:39:31 2021 +0200
       
          MDEV-24962: Galera SST innobackupex-move ignores Environment settings
      

      There, a SECRET_TAG is also introduced (not mentioned in the commit message), and it is implemented for rsync and mariabackup, but not for IST:

      (sst)

               # Store donor's wsrep GTID (state ID) and wsrep_gtid_domain_id
               # (separated by a space).
      -        echo "${WSREP_SST_OPT_GTID} ${WSREP_SST_OPT_GTID_DOMAIN_ID}" > "${MAGIC_FILE}"
      +        echo "$WSREP_SST_OPT_GTID $WSREP_SST_OPT_GTID_DOMAIN_ID" > "$MAGIC_FILE"
      +
      +        if [ -n "$WSREP_SST_OPT_REMOTE_PSWD" ]; then
      +            # Let joiner know that we know its secret
      +            echo "$SECRET_TAG $WSREP_SST_OPT_REMOTE_PSWD" >> "$MAGIC_FILE"
      +        fi
      
      

      (rsync)

      +    if [ -n "$WSREP_SST_OPT_REMOTE_PSWD" ]; then
      +        # Let joiner know that we know its secret
      +        echo "$SECRET_TAG $WSREP_SST_OPT_REMOTE_PSWD" >> "$MAGIC_FILE"
      +    fi
      +
           rsync ${STUNNEL:+--rsh="$STUNNEL"} \
                 --archive --quiet --checksum "$MAGIC_FILE" rsync://$WSREP_SST_OPT_ADDR
      
      

      But not for IST:

           else # BYPASS FOR IST
       
               wsrep_log_info "Bypassing the SST for IST"
               echo "continue" # now server can resume updating data
       
               # Store donor's wsrep GTID (state ID) and wsrep_gtid_domain_id
               # (separated by a space).
      -        echo "${WSREP_SST_OPT_GTID} ${WSREP_SST_OPT_GTID_DOMAIN_ID}" > "${MAGIC_FILE}"
      -        echo "1" > "${DATA}/${IST_FILE}"
      +        echo "$WSREP_SST_OPT_GTID $WSREP_SST_OPT_GTID_DOMAIN_ID" > "$MAGIC_FILE"
      +        echo "1" > "$DATA/$IST_FILE"
      

      I added the appropriate changes, and then the IST started to work too:

      --- wsrep_sst_mariabackup 2023-10-03 19:35:52.008645765 +0200
      +++ wsrep_sst_mariabackup 2023-10-03 19:35:43.520780315 +0200
      @@ -1189,6 +1189,12 @@ if [ "$WSREP_SST_OPT_ROLE" = 'donor' ];
               # Store donor's wsrep GTID (state ID) and wsrep_gtid_domain_id
               # (separated by a space).
               echo "$WSREP_SST_OPT_GTID $WSREP_SST_OPT_GTID_DOMAIN_ID" > "$MAGIC_FILE"
      +
      +        if [ -n "$WSREP_SST_OPT_REMOTE_PSWD" ]; then
      +            # Let joiner know that we know its secret
      +            echo "$SECRET_TAG $WSREP_SST_OPT_REMOTE_PSWD" >> "$MAGIC_FILE"
      +        fi
      +
               echo "1" > "$DATA/$IST_FILE"
       
               if [ -n "$scomp" ]; then
      

      See config/version at MDEV-32342.

      Cheers,
      Walter Doekes
      OSSO B.V.

      Attachments

        Issue Links

          Activity

            wdoekes Walter Doekes created issue -
            wdoekes Walter Doekes made changes -
            Field Original Value New Value
            serg Sergei Golubchik made changes -
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.6 [ 24028 ]
            Fix Version/s 10.10 [ 27530 ]
            Fix Version/s 10.11 [ 27614 ]
            Fix Version/s 11.0 [ 28320 ]
            Fix Version/s 11.1 [ 28549 ]
            serg Sergei Golubchik made changes -
            Assignee Julius Goryavsky [ sysprg ]
            julien.fritsch Julien Fritsch made changes -
            Fix Version/s 10.10 [ 27530 ]
            sysprg Julius Goryavsky made changes -
            Fix Version/s 10.4.33 [ 29516 ]
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.6 [ 24028 ]
            Fix Version/s 10.11 [ 27614 ]
            Fix Version/s 11.0 [ 28320 ]
            Fix Version/s 11.1 [ 28549 ]
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Closed [ 6 ]
            JIraAutomate JiraAutomate made changes -
            Fix Version/s 10.5.24 [ 29517 ]
            Fix Version/s 10.6.17 [ 29518 ]
            Fix Version/s 10.11.7 [ 29519 ]
            Fix Version/s 11.0.5 [ 29520 ]
            Fix Version/s 11.1.4 [ 29024 ]
            Fix Version/s 11.2.3 [ 29521 ]
            sysprg Julius Goryavsky made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            martin.reinhardt@mariadb.com Martin Reinhardt made changes -
            ralf.gebhardt Ralf Gebhardt made changes -
            sysprg Julius Goryavsky made changes -
            sysprg Julius Goryavsky made changes -
            mariadb-jira-automation Jira Automation (IT) made changes -
            Zendesk Related Tickets 136650

            People

              sysprg Julius Goryavsky
              wdoekes Walter Doekes
              Votes:
              2 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.