[MDEV-26211] Cluster joiner node is failed to start when using TLS mariabackup SST encryption built into socat - Jira

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Fixed
Affects Version/s: 10.2(EOL), 10.3(EOL), 10.4(EOL), 10.5, 10.6, 10.7(EOL)
Fix Version/s: 10.2.41, 10.3.32, 10.4.22, 10.5.13, 10.6.5, 10.7.0
Component/s: Galera SST
Labels:
None

Description

Start 2 node cluster using TLS mariabackup SST encryption built into socat.

config info

wsrep_provider_options='gmcast.listen_addr=tcp://127.0.0.1:4808;'

ssl-ca = /dev/shm/qa/cert/ca.pem

ssl-cert = /dev/shm/qa/cert/server-cert.pem

ssl-key = /dev/shm/qa/cert/server-key.pem

[sst]

encrypt = 2

tca = /dev/shm/qa/cert/sst_encypt2.crt

tcert = /dev/shm/qa/cert/sst_encypt2.pem

Error info from donor node.

2021-07-21 16:49:12 0 [Note] WSREP: Running: 'wsrep_sst_mariabackup --role 'donor' --address 'ax3win:4444/xtrabackup_sst//1' --local-port '4800' --socket '/dev/shm/qa/node1/mysql.sock' --datadir '/dev/shm/qa/node1/' --defaults-file '/dev/shm/qa/conf/node1.cnf' --gtid '92245b05-ea43-11eb-89cc-ba6f0392455f:0' --gtid-domain-id '0' --mysqld-args --defaults-file=/dev/shm/qa/conf/node1.cnf --wsrep-new-cluster'

2021-07-21 16:49:12 2 [Note] WSREP: sst_donor_thread signaled with 0

WSREP_SST: [INFO] SSL configuration: CA='/dev/shm/qa/cert/sst_encypt2.crt', CERT='/dev/shm/qa/cert/sst_encypt2.pem', KEY='', MODE='DISABLED', encrypt='2' (20210721 16:49:12.800)

WSREP_SST: [INFO] Streaming with mbstream (20210721 16:49:12.950)

WSREP_SST: [INFO] Using socat as streamer (20210721 16:49:12.952)

WSREP_SST: [INFO] Using openssl based encryption with socat: with crt and pem (20210721 16:49:12.959)

WSREP_SST: [INFO] Encrypting with cert=/dev/shm/qa/cert/sst_encypt2.pem, cafile=/dev/shm/qa/cert/sst_encypt2.crt (20210721 16:49:12.968)

WSREP_SST: [INFO] Using '/tmp/tmp.PqLfYMKEGR' as mariabackup temporary directory (20210721 16:49:12.990)

WSREP_SST: [INFO] Using '/tmp/tmp.wt3mCpRqiI' as mariabackup working directory (20210721 16:49:12.995)

WSREP_SST: [INFO] Streaming GTID file before SST (20210721 16:49:12.999)

WSREP_SST: [INFO] Evaluating '/home/ramesh/framework/GAL_MD200721-mariadb-10.3.30-linux-x86_64-opt//bin/mbstream' -c 'xtrabackup_galera_info' | socat -u stdio openssl-connect:ax3win:4444,cert='/dev/shm/qa/cert/sst_encypt2.pem',cafile='/dev/shm/qa/cert/sst_encypt2.crt'; RC=( ${PIPESTATUS[@]} ) (20210721 16:49:13.002)

2021/07/21 16:49:13 socat[2709097] E certificate is valid but its commonName does not match hostname

WSREP_SST: [ERROR] Error while sending data to joiner node:  exit codes: 0 1 (20210721 16:49:13.020)

WSREP_SST: [ERROR] Cleanup after exit with status:32 (20210721 16:49:13.022)

WSREP_SST: [INFO] Cleaning up temporary directories (20210721 16:49:13.025)

Attachments

Activity

Ascending order - Click to sort in descending order

Ramesh Sivaraman added a comment - 2021-08-10 11:49

The issue is present even if we use galera-compatible certificates and keys

WSREP_SST: [INFO] SSL configuration: CA='/home/vagrant/ca.pem', CERT='/home/vagrant/server-cert.pem', KEY='/home/vagrant/server-key.pem', MODE='DISABLED', encrypt='3' (20210809 10:42:32.645)

WSREP_SST: [INFO] Moving '/home/vagrant/data/mariabackup.prepare.log' to '/tmp/sst_log_archive/mariabackup.prepare.log.2021.08.09-10.42.32.714837449' (20210809 10:42:32.717)

WSREP_SST: [INFO] Moving '/home/vagrant/data/mariabackup.move.log' to '/tmp/sst_log_archive/mariabackup.move.log.2021.08.09-10.42.32.714837449' (20210809 10:42:32.721)

WSREP_SST: [INFO] Streaming with mbstream (20210809 10:42:32.726)

WSREP_SST: [INFO] Using socat as streamer (20210809 10:42:32.727)

WSREP_SST: [INFO] Using openssl based encryption with socat: with key and crt (20210809 10:42:32.731)

WSREP_SST: [INFO] Decrypting with cert=/home/vagrant/server-cert.pem, key=/home/vagrant/server-key.pem, cafile=/home/vagrant/ca.pem (20210809 10:42:32.746)

WSREP_SST: [INFO] Evaluating timeout -k 310 300 socat -u openssl-listen:4444,reuseaddr,cert='/home/vagrant/server-cert.pem',key='/home/vagrant/server-key.pem',cafile='/home/vagrant/ca.pem',commonname=localhost stdio | '/home/vagrant/mariadb-10.6.4-1-linux-x86_64//bin/mbstream' -x; RC=( ${PIPESTATUS[@]} ) (20210809 10:42:32.762)

2021-08-09 10:42:32 1 [Note] WSREP: ####### IST uuid:3cb6d88c-f8fd-11eb-9496-7a09349f8515 f: 9, l: 11, STRv: 3

2021-08-09 10:42:32 1 [Note] WSREP: IST receiver addr using ssl://192.168.100.20:4568

2021-08-09 10:42:32 1 [Note] WSREP: IST receiver using ssl

2021-08-09 10:42:32 1 [Note] WSREP: Prepared IST receiver for 9-11, listening at: ssl://192.168.100.20:4568

2021-08-09 10:42:32 0 [Note] WSREP: Member 1.0 (galera-node2) requested state transfer from '*any*'. Selected 0.0 (galera-node1)(SYNCED) as donor.

2021-08-09 10:42:32 0 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 11)

2021-08-09 10:42:32 1 [Note] WSREP: Requesting state transfer: success, donor: 0

2021/08/09 10:42:32 socat[4181] E certificate is valid but its commonName does not match hostname

WSREP_SST: [ERROR] Error while getting data from donor node:  exit codes: 1 0 (20210809 10:42:32.974)

WSREP_SST: [ERROR] Cleanup after exit with status:32 (20210809 10:42:32.976)

Config info

wsrep_provider_options="socket.ssl_cert=/home/vagrant/server-cert.pem;socket.ssl_key=/home/vagrant/server-key.pem;socket.ssl_ca=/home/vagrant/ca.pem"

ssl-ca=/home/vagrant/ca.pem

ssl-key=/home/vagrant/server-key.pem

ssl-cert=/home/vagrant/server-cert.pem

[sst]

encrypt=3

tca=/home/vagrant/ca.pem

tkey=/home/vagrant/server-key.pem

tcert=/home/vagrant/server-cert.pem

Ramesh Sivaraman added a comment - 2021-08-10 11:49 The issue is present even if we use galera-compatible certificates and keys WSREP_SST: [INFO] SSL configuration: CA='/home/vagrant/ca.pem', CERT='/home/vagrant/server-cert.pem', KEY='/home/vagrant/server-key.pem', MODE='DISABLED', encrypt='3' (20210809 10:42:32.645) WSREP_SST: [INFO] Moving '/home/vagrant/data/mariabackup.prepare.log' to '/tmp/sst_log_archive/mariabackup.prepare.log.2021.08.09-10.42.32.714837449' (20210809 10:42:32.717) WSREP_SST: [INFO] Moving '/home/vagrant/data/mariabackup.move.log' to '/tmp/sst_log_archive/mariabackup.move.log.2021.08.09-10.42.32.714837449' (20210809 10:42:32.721) WSREP_SST: [INFO] Streaming with mbstream (20210809 10:42:32.726) WSREP_SST: [INFO] Using socat as streamer (20210809 10:42:32.727) WSREP_SST: [INFO] Using openssl based encryption with socat: with key and crt (20210809 10:42:32.731) WSREP_SST: [INFO] Decrypting with cert=/home/vagrant/server-cert.pem, key=/home/vagrant/server-key.pem, cafile=/home/vagrant/ca.pem (20210809 10:42:32.746) WSREP_SST: [INFO] Evaluating timeout -k 310 300 socat -u openssl-listen:4444,reuseaddr,cert='/home/vagrant/server-cert.pem',key='/home/vagrant/server-key.pem',cafile='/home/vagrant/ca.pem',commonname=localhost stdio | '/home/vagrant/mariadb-10.6.4-1-linux-x86_64//bin/mbstream' -x; RC=( ${PIPESTATUS[@]} ) (20210809 10:42:32.762) 2021-08-09 10:42:32 1 [Note] WSREP: ####### IST uuid:3cb6d88c-f8fd-11eb-9496-7a09349f8515 f: 9, l: 11, STRv: 3 2021-08-09 10:42:32 1 [Note] WSREP: IST receiver addr using ssl://192.168.100.20:4568 2021-08-09 10:42:32 1 [Note] WSREP: IST receiver using ssl 2021-08-09 10:42:32 1 [Note] WSREP: Prepared IST receiver for 9-11, listening at: ssl://192.168.100.20:4568 2021-08-09 10:42:32 0 [Note] WSREP: Member 1.0 (galera-node2) requested state transfer from '*any*'. Selected 0.0 (galera-node1)(SYNCED) as donor. 2021-08-09 10:42:32 0 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 11) 2021-08-09 10:42:32 1 [Note] WSREP: Requesting state transfer: success, donor: 0 2021/08/09 10:42:32 socat[4181] E certificate is valid but its commonName does not match hostname WSREP_SST: [ERROR] Error while getting data from donor node: exit codes: 1 0 (20210809 10:42:32.974) WSREP_SST: [ERROR] Cleanup after exit with status:32 (20210809 10:42:32.976) Config info wsrep_provider_options="socket.ssl_cert=/home/vagrant/server-cert.pem;socket.ssl_key=/home/vagrant/server-key.pem;socket.ssl_ca=/home/vagrant/ca.pem" ssl-ca=/home/vagrant/ca.pem ssl-key=/home/vagrant/server-key.pem ssl-cert=/home/vagrant/server-cert.pem [sst] encrypt=3 tca=/home/vagrant/ca.pem tkey=/home/vagrant/server-key.pem tcert=/home/vagrant/server-cert.pem

Jan Lindström (Inactive) added a comment - 2021-08-16 05:23

ok to push

Jan Lindström (Inactive) added a comment - 2021-08-16 05:23 ok to push

Julius Goryavsky added a comment - 2021-08-16 08:58

Fixed, https://github.com/MariaDB/server/commit/d1a948cfaaab67e699674af4c11efad3868a629d

Julius Goryavsky added a comment - 2021-08-16 08:58 Fixed, https://github.com/MariaDB/server/commit/d1a948cfaaab67e699674af4c11efad3868a629d

MariaDB Server

Cluster joiner node is failed to start when using TLS mariabackup SST encryption built into socat

Details

Description

Attachments

Activity

People

Dates

Git Integration