|
Hello,
We're using MariaDB 10.1 configured with SSL on Debian 9 and have been using it for about a year now without issues. As far as I am aware, SST has worked previously but since we don't need it very often (normally just IST) I am not sure when the issue was introduced.
We recently needed a full SST for one of our nodes within the 3-node Galera cluster. I will include the config options I think are relevant:
/etc/mysql/my.cnf
[client]
|
<snipped>
|
ssl=1
|
ssl-ca=/etc/ssl/certs/icingasql01-mariadb-ca.pem
|
ssl-cert=/etc/ssl/certs/icingasql01-mariadb-cert.pem
|
ssl-key=/etc/ssl/private/icingasql01-mariadb-key.pem
|
|
[mysqld_safe]
|
<snipped>
|
|
[mysqld]
|
<snipped>
|
ssl-ca=/etc/ssl/certs/icingasql01-mariadb-ca.pem
|
ssl-cert=/etc/ssl/certs/icingasql01-mariadb-cert.pem
|
ssl-key=/etc/ssl/private/icingasql01-mariadb-key.pem
|
|
[mysqldump]
|
<snipped>
|
|
[isamchk]
|
<snipped>
|
/etc/mysql/conf.d/00-galera.cnf:
[mysqld]
|
wsrep_on=ON
|
wsrep_provider=/usr/lib/galera/libgalera_smm.so
|
wsrep_cluster_name='galera01_icinga'
|
wsrep_node_address=<redacted>
|
wsrep_cluster_address='gcomm://<redacted>'
|
wsrep_sst_method=xtrabackup-v2
|
wsrep_sst_auth=sst_xtrabackup:<redacted>
|
wsrep_retry_autocommit=4
|
wsrep_slave_threads=4
|
wsrep_provider_options="gcache.size=256M;socket.ssl=yes;socket.ssl_key=/etc/ssl/private/icingasql01-mariadb-key.pem;socket.ssl_cert=/etc/ssl/certs/icingasql01-mariadb-cert.pem;socket.ssl_ca=/etc/ssl/certs/icingasql01-mariadb-ca.pem"
|
binlog_format=ROW
|
innodb_autoinc_lock_mode=2
|
innodb_doublewrite=1
|
innodb_log_file_size=512M
|
innodb_flush_log_at_trx_commit=0
|
enforce_storage_engine=innodb
|
sql_mode="NO_AUTO_CREATE_USER"
|
[sst]
|
encrypt=4
|
ssl-ca=/etc/ssl/certs/icingasql01-mariadb-ca.pem
|
ssl-cert=/etc/ssl/certs/icingasql01-mariadb-cert.pem
|
ssl-key=/etc/ssl/private/icingasql01-mariadb-key.pem
|
The certificates have chmod 0440 and ownership root:ssl-cert. The mysql user is part of the ssl-cert group and is able to access the cert/key from as far as I can see.
During the SST, the donor shows the following error:
WSREP_SST: [ERROR] innobackupex finished with error: 1. Check /var/lib/mysql//innobackup.backup.log (20181114 12:00:13.940)
|
I have added a set -x in the xtrabackup script and ran the exact command being executed, this shows the following (identical to what is in the above logfile)
innobackupex --no-version-check --user=sst_xtrabackup --password=<redacted> --socket=/var/run/mysqld/mysqld.sock --galera-info --stream=xbstream /tmp/tmp.oFMtzcQalr
|
xtrabackup: recognized server arguments: --datadir=/var/lib/mysql --tmpdir=/tmp --innodb_buffer_pool_size=997M --innodb_log_buffer_size=16M --innodb_file_per_table=1 --innodb_open_files=2000 --innodb_io_capacity=4000 --innodb_flush_method=O_DIRECT --open_files_limit=65536 --innodb_doublewrite=1 --innodb_log_file_size=512M --innodb_flush_log_at_trx_commit=0
|
WARNING: --ssl is deprecated and will be removed in a future version. Use --ssl-mode instead.
|
xtrabackup: recognized client arguments: --datadir=/var/lib/mysql --tmpdir=/tmp --innodb_buffer_pool_size=997M --innodb_log_buffer_size=16M --innodb_file_per_table=1 --innodb_open_files=2000 --innodb_io_capacity=4000 --innodb_flush_method=O_DIRECT --open_files_limit=65536 --innodb_doublewrite=1 --innodb_log_file_size=512M --innodb_flush_log_at_trx_commit=0 --ssl=1 --ssl-ca=/etc/ssl/certs/icingasql02-mariadb-ca.pem --ssl-cert=/etc/ssl/certs/icingasql02-mariadb-cert.pem --ssl-key=/etc/ssl/private/icingasql02-mariadb-key.pem
|
encryption: using gcrypt 1.7.6-beta
|
181114 12:02:40 innobackupex: Starting the backup operation
|
|
IMPORTANT: Please check that the backup run completes successfully.
|
At the end of a successful backup run innobackupex
|
prints "completed OK!".
|
|
181114 12:02:40 Connecting to MySQL server host: localhost, user: sst_xtrabackup, password: set, port: 3306, socket: /var/run/mysqld/mysqld.sock
|
Failed to connect to MySQL server: Access denied for user 'sst_xtrabackup'@'localhost' (using password: YES).
|
I have the following xtrabackup scripts:
apt-cache policy percona-xtrabackup-24
|
percona-xtrabackup-24:
|
Installed: 2.4.12-1.stretch
|
Candidate: 2.4.12-1.stretch
|
Version table:
|
*** 2.4.12-1.stretch 500
|
500 http://repo.percona.com/apt stretch/main amd64 Packages
|
100 /var/lib/dpkg/status
|
Manually connecting with this user does work. When I try to connect with --no-defaults, it does not work. This is because SSL is required. We configured the SST user like this:
GRANT ALL PRIVILEGES ON *.* TO 'sst_xtrabackup'@'127.0.0.1' IDENTIFIED BY PASSWORD '<redacted>' REQUIRE SSL WITH GRANT OPTION
|
After I removed the REQUIRE SSL part, SST did work correctly. However since we have galera running across DCs we want to use SSL instead. Is this misconfiguration or a bug?
|