[MDEV-16147] Galera Arbitrator fails to join the cluster with SSL Created: 2018-05-11  Updated: 2022-12-08  Resolved: 2022-12-08

Status: Closed
Project: MariaDB Server
Component/s: Galera Arbitrator garbd
Affects Version/s: 10.3.6
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Zdravelina Sokolovska (Inactive) Assignee: Ramesh Sivaraman
Resolution: Not a Bug Votes: 0
Labels: None
Environment:

CentOS 7.4


 Description   

Galera Arbitrator fails to join the cluster with SSL

Enable SSL on Galera Node .
Provide the SSL credentials to other Node and run garbd with the related wsrep_provider_options
socket.ssl_key ,socket.ssl_cert and socket.ssl_ca .

garbd failed due to missing SSL parameter socket.ssl_cipher which is set in galera to AES128-SHA by default .

#  garbd --address gcomm://192.168.104.191:4567,192.168.104.195:4567,192.168.104.196:4567?gmcast.listen_addr=tcp://0.0.0.0:4444  -o "socket.ssl_key=/etc/mysql/cc/server.key;socket.ssl_cert=/etc/mysql/cc/server.pem;socket.ssl_ca=/etc/mysql/cc/server.crt" --group cluster1
 
2018-05-11 18:23:16.756  INFO: CRC-32C: using "slicing-by-8" algorithm.
 
2018-05-11 18:23:16.756  INFO: Read config:
 
        daemon:  0
 
        name:    garb
 
        address: gcomm://192.168.104.191:4567,192.168.104.195:4567,192.168.104.196:4567?gmcast.listen_addr=tcp://0.0.0.0:4444
 
        group:   cluster1
 
        sst:     trivial
 
        donor:
 
        options: socket.ssl_key=/etc/mysql/cc/server.key;socket.ssl_cert=/etc/mysql/cc/server.pem;socket.ssl_ca=/etc/mysql/cc/server.crt; gcs.fc_limit=9999999; gcs.fc_factor=1.0; gcs.fc_master_slave=yes
 
        cfg:
 
        log:
 
 
 
2018-05-11 18:23:16.758  INFO: protonet asio version 0
 
2018-05-11 18:23:16.758  INFO: Using CRC-32C for message checksums.
 
2018-05-11 18:23:16.758  INFO: initializing ssl context
 
2018-05-11 18:23:16.759 ERROR: failed to create gcomm backend connection: 22: Missing required value for SSL parameter 'socket.ssl_cipher': 22 (Invalid argument)
 
         at galerautils/src/gu_asio.cpp:ssl_prepare_context():158
 
2018-05-11 18:23:16.759 ERROR: gcs/src/gcs_core.cpp:gcs_core_open():215: Failed to initialize backend using 'gcomm://192.168.104.191:4567,192.168.104.195:4567,192.168.104.196:4567?gmcast.listen_addr=tcp://0.0.0.0:4444': -22 (Invalid argument)
 
2018-05-11 18:23:16.759 ERROR: gcs/src/gcs.cpp:gcs_open():1458: Failed to open channel 'cluster1' at 'gcomm://192.168.104.191:4567,192.168.104.195:4567,192.168.104.196:4567?gmcast.listen_addr=tcp://0.0.0.0:4444': -22 (Invalid argument)
 
2018-05-11 18:23:16.759 FATAL: Exception in creating receive loop: Failed to open connection to group: 22 (Invalid argument)
 
         at garb/garb_gcs.cpp:Gcs():35
 
[root@t4w3 ~]#

add socket.ssl_cipher=AES128-SHA to wsrep_provider_options and rerun garbd will
join Arbitrator successfully 

 garbd --address gcomm://192.168.104.191:4567,192.168.104.195:4567,192.168.104.196:4567?gmcast.listen_addr=tcp://0.0.0.0:4444  \
 
-o "socket.ssl_key=/etc/mysql/cc/server.key;socket.ssl_cert=/etc/mysql/cc/server.pem;socket.ssl_ca=/etc/mysql/cc/server.crt;socket.ssl_cipher=AES128-SHA"  \
 
--group cluster1


#  garbd --address gcomm://192.168.104.191:4567,192.168.104.195:4567,192.168.104.196:4567?gmcast.listen_addr=tcp://0.0.0.0:4444  -o "socket.ssl_key=/etc/mysql/cc/server.key;socket.ssl_cert=/etc/mysql/cc/server.pem;socket.ssl_ca=/etc/mysql/cc/server.crt;socket.ssl_cipher=AES128-SHA" --group cluster1
 
2018-05-11 18:23:45.740  INFO: CRC-32C: using "slicing-by-8" algorithm.
 
2018-05-11 18:23:45.740  INFO: Read config:
 
        daemon:  0
 
        name:    garb
 
        address: gcomm://192.168.104.191:4567,192.168.104.195:4567,192.168.104.196:4567?gmcast.listen_addr=tcp://0.0.0.0:4444
 
        group:   cluster1
 
        sst:     trivial
 
        donor:
 
        options: socket.ssl_key=/etc/mysql/cc/server.key;socket.ssl_cert=/etc/mysql/cc/server.pem;socket.ssl_ca=/etc/mysql/cc/server.crt;socket.ssl_cipher=AES128-SHA; gcs.fc_limit=9999999; gcs.fc_factor=1.0; gcs.fc_master_slave=yes
 
        cfg:
 
        log:
 
 
 
2018-05-11 18:23:45.742  INFO: protonet asio version 0
 
2018-05-11 18:23:45.743  INFO: Using CRC-32C for message checksums.
 
2018-05-11 18:23:45.743  INFO: initializing ssl context
 
2018-05-11 18:23:45.743  INFO: backend: asio
 
2018-05-11 18:23:45.744  INFO: gcomm thread scheduling priority set to other:0
 
2018-05-11 18:23:45.744  INFO: restore pc from disk successfully
 
2018-05-11 18:23:45.744  INFO: GMCast version 0
 
2018-05-11 18:23:45.745  INFO: (356bbd01, 'ssl://0.0.0.0:4444') listening at ssl://0.0.0.0:4444
 
2018-05-11 18:23:45.745  INFO: (356bbd01, 'ssl://0.0.0.0:4444') multicast: , ttl: 1
 
2018-05-11 18:23:45.746  INFO: EVS version 0
 
2018-05-11 18:23:45.746  INFO: gcomm: connecting to group 'cluster1', peer '192.168.104.191:4567,192.168.104.195:4567,192.168.104.196:4567'
 
2018-05-11 18:23:45.751  INFO: SSL handshake successful, remote endpoint ssl://192.168.104.195:4567 local endpoint ssl://192.168.104.193:41710 cipher: AES128-SHA compression: none
 
2018-05-11 18:23:45.752  INFO: (356bbd01, 'ssl://0.0.0.0:4444') connection established to 3086e40d ssl://192.168.104.195:4567
 
2018-05-11 18:23:45.752  INFO: (356bbd01, 'ssl://0.0.0.0:4444') turning message relay requesting on, nonlive peers:
 
2018-05-11 18:23:45.753  INFO: SSL handshake successful, remote endpoint ssl://192.168.104.196:4567 local endpoint ssl://192.168.104.193:34040 cipher: AES128-SHA compression: none
 
2018-05-11 18:23:45.754  INFO: (356bbd01, 'ssl://0.0.0.0:4444') connection established to 4efcf962 ssl://192.168.104.196:4567
 
2018-05-11 18:23:46.250  INFO: declaring 3086e40d at ssl://192.168.104.195:4567 stable
 
2018-05-11 18:23:46.250  INFO: declaring 4efcf962 at ssl://192.168.104.196:4567 stable
 
2018-05-11 18:23:46.252  INFO: Node 3086e40d state prim
 
2018-05-11 18:23:46.253  INFO: view(view_id(PRIM,3086e40d,39) memb {
 
        3086e40d,0
 
        356bbd01,0
 
        4efcf962,0
 
} joined {
 
} left {
 
} partitioned {
 
})
 
2018-05-11 18:23:46.253  INFO: save pc into disk
 
2018-05-11 18:23:46.253  INFO: discarding pending addr without UUID: ssl://192.168.104.191:4567
 
2018-05-11 18:23:46.253  INFO: clear restored view
 
2018-05-11 18:23:46.747  INFO: gcomm: connected
 
2018-05-11 18:23:46.747  INFO: Changing maximum packet size to 64500, resulting msg size: 32636
 
2018-05-11 18:23:46.747  INFO: Shifting CLOSED -> OPEN (TO: 0)
 
2018-05-11 18:23:46.747  INFO: Opened channel 'cluster1'
 
2018-05-11 18:23:46.748  INFO: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 3
 
2018-05-11 18:23:46.748  INFO: STATE EXCHANGE: Waiting for state UUID.
 
2018-05-11 18:23:46.748  INFO: STATE EXCHANGE: sent state msg: 4c0ceffa-552f-11e8-a16c-938cc350f7f9
 
2018-05-11 18:23:46.748  INFO: STATE EXCHANGE: got state msg: 4c0ceffa-552f-11e8-a16c-938cc350f7f9 from 0 (t4w5)
 
2018-05-11 18:23:46.748  INFO: STATE EXCHANGE: got state msg: 4c0ceffa-552f-11e8-a16c-938cc350f7f9 from 2 (t4w6)
 
2018-05-11 18:23:46.749  INFO: STATE EXCHANGE: got state msg: 4c0ceffa-552f-11e8-a16c-938cc350f7f9 from 1 (garb)
 
2018-05-11 18:23:46.749  INFO: Quorum results:
 
        version    = 4,
 
        component  = PRIMARY,
 
        conf_id    = 37,
 
        members    = 2/3 (joined/total),
 
        act_id     = 147684,
 
        last_appl. = -1,
 
        protocols  = 0/7/3 (gcs/repl/appl),
 
        group UUID = b4c974d2-49fe-11e8-b950-9b4c947b49f6
 
2018-05-11 18:23:46.749  INFO: Flow-control interval: [9999999, 9999999]
 
2018-05-11 18:23:46.749  INFO: Trying to continue unpaused monitor
 
2018-05-11 18:23:46.749  INFO: Shifting OPEN -> PRIMARY (TO: 147684)
 
2018-05-11 18:23:46.749  INFO: Sending state transfer request: 'trivial', size: 7
 
2018-05-11 18:23:46.750  INFO: Member 1.0 (garb) requested state transfer from '*any*'. Selected 0.0 (t4w5)(SYNCED) as donor.
 
2018-05-11 18:23:46.750  INFO: Shifting PRIMARY -> JOINER (TO: 147684)
 
2018-05-11 18:23:46.751  INFO: 0.0 (t4w5): State transfer to 1.0 (garb) complete.
 
2018-05-11 18:23:46.751  INFO: 1.0 (garb): State transfer from 0.0 (t4w5) complete.
 
2018-05-11 18:23:46.751  INFO: Shifting JOINER -> JOINED (TO: 147684)
 
2018-05-11 18:23:46.753  INFO: Member 0.0 (t4w5) synced with group.
 
2018-05-11 18:23:46.753  INFO: Member 1.0 (garb) synced with group.
 
2018-05-11 18:23:46.753  INFO: Shifting JOINED -> SYNCED (TO: 147684)
 
2018-05-11 18:23:49.247  INFO: (356bbd01, 'ssl://0.0.0.0:4444') turning message relay requesting off
 
 



 Comments   
Comment by Ramesh Sivaraman [ 2022-12-08 ]

As per Galera doc we need to specify ssl_cipher in garbd options if we use SSL.
https://galeracluster.com/library/documentation/arbitrator.html

If you use SSL, it’s necessary to specify the cipher. Otherwise, after initializing the ssl context an error will occur with a message saying, “Terminate called after throwing an instance of ‘gu::NotSet’”.

Generated at Thu Feb 08 08:26:43 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.