Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-26019

Upgrading MariaDB from 10.5.10 to 10.5.11 breaks TLS mariabackup SST

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 10.6.2, 10.2.39, 10.3.30, 10.4.20, 10.5.11
    • Fix Version/s: 10.2.40, 10.3.31, 10.4.21, 10.5.13, 10.6.3
    • Component/s: Galera SST, mariabackup, wsrep
    • Environment:
      Linux vc-galera01 5.4.114-1-pve #1 SMP PVE 5.4.114-1 (Sun, 09 May 2021 17:13:05 +0200) x86_64 x86_64 x86_64 GNU/Linux

      mysql Ver 15.1 Distrib 10.5.11-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

      Description

      The upgrade from MariaDB from 10.5.10 to 10.5.11 breaks the wsrep_sst_mariabackup script. This is due to adding the commonname option to socat which at least on my end defaults to "localhost". There is also a typo (E and S swapped in ESCAPED) on line 389 that most likely is breaking something as well.

      elif is_local_ip "$WSREP_SST_OPT_HOST_UNESCAPED"; then
          CN_option=',commonname=localhost'
      else
          CN_option=",commonname='$WSREP_SST_OPT_HOST_UNSECAPED'" <- Right here
      fi
      

      To just get the node up I had to make the following change on line 391 of wsrep_sst_mariabackup, which also triggered a very inconvenient SST.

      #tcmd="$tcmd,cert='$tpem',key='$tkey',cafile='$tcert'$CN_option$sockopt"
      tcmd="$tcmd,cert='$tpem',key='$tkey',cafile='$tcert'$sockopt"
      

      Below is my log that led me to looking at the differences between the two versions:

      Jun 24 15:05:03 node1 -wsrep-sst-joiner: Decrypting with cert=/etc/mysql/certs/server-cert.pem, key=/etc/mysql/certs/server-key.pem, cafile=/etc/mysql/certs/ca.pem
      Jun 24 15:05:03 node1 -wsrep-sst-joiner: Evaluating timeout -k 310 300 socat -u openssl-listen:4444,reuseaddr,cert='/etc/mysql/certs/server-cert.pem',key='/etc/mysql/certs/server-key.pem',cafile='/etc/mysql/certs/ca.pem',commonname=localhost stdio | '/usr//bin/mbstream' -x; RC=( ${PIPESTATUS[@]} )
      Jun 24 15:05:03 node1 mariadbd[9378]: 2021-06-24 15:05:03 1 [Note] WSREP: ####### IST uuid:9795eb17-c967-11eb-896e-32dd10aa7427 f: 628742, l: 628743, STRv: 3
      Jun 24 15:05:03 node1 mariadbd[9378]: 2021-06-24 15:05:03 1 [Note] WSREP: IST receiver addr using ssl://node1.mycompany.com:4568
      Jun 24 15:05:03 node1 mariadbd[9378]: 2021-06-24 15:05:03 1 [Note] WSREP: IST receiver using ssl
      Jun 24 15:05:03 node1 mariadbd[9378]: 2021-06-24 15:05:03 1 [Note] WSREP: Prepared IST receiver for 628742-628743, listening at: ssl://node1-ip:4568
      Jun 24 15:05:03 node1 mariadbd[9378]: 2021-06-24 15:05:03 0 [Note] WSREP: Member 2.0 (node1) requested state transfer from '*any*'. Selected 1.0 (node3)(SYNCED) as donor.
      Jun 24 15:05:03 node1 mariadbd[9378]: 2021-06-24 15:05:03 0 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 628743)
      Jun 24 15:05:03 node1 mariadbd[9378]: 2021-06-24 15:05:03 1 [Note] WSREP: Requesting state transfer: success, donor: 1
      Jun 24 15:05:05 node1 mariadbd[9378]: 2021-06-24 15:05:05 0 [Note] WSREP: (75d699bd-98d2, 'ssl://0.0.0.0:4567') turning message relay requesting off
      Jun 24 15:05:06 node1 -wsrep-sst-joiner: 2021/06/24 15:05:06 socat[9577] E certificate is valid but its commonName does not match hostname
      Jun 24 15:05:06 node1 -wsrep-sst-joiner: Error while getting data from donor node:  exit codes: 1 0
      Jun 24 15:05:06 node1 -wsrep-sst-joiner: Cleanup after exit with status:32
      Jun 24 15:05:06 node1 -wsrep-sst-joiner: Removing the sst_in_progress file
      Jun 24 15:05:06 node1 -wsrep-sst-joiner: Cleaning up temporary directories
      Jun 24 15:05:06 node1 mariadbd[9378]: 2021-06-24 15:05:06 0 [ERROR] WSREP: Process completed with error: wsrep_sst_mariabackup --role 'joiner' --address 'node1.mycompany.com' --datadir '/var/lib/mysql/' --parent '9378' --mysqld-args --wsrep_start_position=9795eb17-c967-11eb-896e-32dd10aa7427:628741: 32 (Broken pipe)
      Jun 24 15:05:06 node1 mariadbd[9378]: 2021-06-24 15:05:06 0 [ERROR] WSREP: Failed to read uuid:seqno and wsrep_gtid_domain_id from joiner script.
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              sysprg Julius Goryavsky
              Reporter:
              ib-mlatin Matthew Latin
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Git Integration