A Galera cluster uses encryption for SST with the following configuration:
[sst]
|
encrypt=3
|
tkey=/path/to/key.pem
|
tcert=/path/to/cert.pem
|
tca=/path/to/ca.pem
|
Upgrading a node to 10.1.28 (in a Galera cluster of 10.1.23 nodes) fails with:
2017-11-04 18:23:17 140462627223296 [Note] WSREP: (3804918b, 'tcp://0.0.0.0:4567') turning message relay requesting off
|
xb_stream_read_chunk(): wrong chunk magic at offset 0x0.
|
WSREP_SST: [ERROR] Error while getting data from donor node: exit codes: 137 1 (20171104 18:24:55.419)
|
WSREP_SST: [ERROR] Cleanup after exit with status:32 (20171104 18:24:55.421)
|
One explication is that the joiner doesn't expect an encrypted stream, poining at a problem with the configuration under [sst] section.
After some investigation I restricted the problem to the wsrep_sst_xtrabackup-v2's function parse_cnf() which was moved to wsrep_sst_common.
In 10.1.23 parse_cnf() is in wsrep_sst_xtrabackup-v2 and it's:
parse_cnf()
|
{
|
local group=$1
|
local var=$2
|
# print the default settings for given group using my_print_default.
|
# normalize the variable names specified in cnf file (user can use _ or - for example log-bin or log_bin)
|
# then grep for needed variable
|
# finally get the variable value (if variables has been specified multiple time use the last value only)
|
reval=$($MY_PRINT_DEFAULTS $group | awk -F= '{if ($1 ~ /_/) { gsub(/_/,"-",$1); print $1"="$2 } else { print $0 }}' | grep -- "--$var=" | cut -d= -f2- | tail -1)
|
if [[ -z $reval ]];then
|
[[ -n $3 ]] && reval=$3
|
fi
|
echo $reval
|
}
|
In 10.1.28 it's moved to wsrep_sst_common and it's:
parse_cnf()
|
{
|
local group=$1
|
local var=$2
|
local reval=""
|
|
# print the default settings for given group using my_print_default.
|
# normalize the variable names specified in cnf file (user can use _ or - for example log-bin or log_bin)
|
# then grep for needed variable
|
# finally get the variable value (if variables has been specified multiple time use the last value only)
|
|
# look in group+suffix
|
if [[ -n $WSREP_SST_OPT_CONF_SUFFIX ]]; then
|
reval=$($MY_PRINT_DEFAULTS -c $WSREP_SST_OPT_CONF "${group}${WSREP_SST_OPT_CONF_SUFFIX}" | awk -F= '{if ($1 ~ /_/) { gsub(/_/,"-",$1); print $1"="$2 } else { print $0 }}' | grep -- "--$var=" | cut -d= -f2- | tail -1)
|
fi
|
|
# look in group
|
if [[ -z $reval ]]; then
|
reval=$($MY_PRINT_DEFAULTS -c $WSREP_SST_OPT_CONF $group | awk -F= '{if ($1 ~ /_/) { gsub(/_/,"-",$1); print $1"="$2 } else { print $0 }}' | grep -- "--$var=" | cut -d= -f2- | tail -1)
|
fi
|
|
# use default if we haven't found a value
|
if [[ -z $reval ]]; then
|
[[ -n $3 ]] && reval=$3
|
fi
|
echo $reval
|
}
|
|
Using wsrep_sst_common and wsrep_sst_xtrabackup-v2 from version 10.1.23 on 10.1.28 fixes the problem and the node succesfully joins.
- relates to
-
MDEV-15254
10.1.31 does not join an existing cluster with SST xtrabackup-v2
-
-
Closed
Transition |
Time In Source Status |
Execution Times |
Open |
|
Closed |
|
80d 22h 26m
|
1
|
{"report":{"fcp":1620.3999999761581,"ttfb":370.5999999642372,"pageVisibility":"visible","entityId":64167,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"84ba3023-6c72-4775-8876-0c04df6eed24","navigationType":0,"readyForUser":1710.0999999642372,"redirectCount":0,"resourceLoadedEnd":2107.5,"resourceLoadedStart":376.0999999642372,"resourceTiming":[{"duration":522.1000000238419,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":376.0999999642372,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":376.0999999642372,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":898.1999999880791,"responseStart":0,"secureConnectionStart":0},{"duration":522.1000000238419,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":376.39999997615814,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":376.39999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":898.5,"responseStart":0,"secureConnectionStart":0},{"duration":708.5999999642372,"initiatorType":"script","name":"https://jira.mariadb.org/s/fbf975c0cce4b1abf04784eeae9ba1f4-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":376.5,"connectEnd":376.5,"connectStart":376.5,"domainLookupEnd":376.5,"domainLookupStart":376.5,"fetchStart":376.5,"redirectEnd":0,"redirectStart":0,"requestStart":376.5,"responseEnd":1085.0999999642372,"responseStart":1085.0999999642372,"secureConnectionStart":376.5},{"duration":840,"initiatorType":"script","name":"https://jira.mariadb.org/s/099b33461394b8015fc36c0a4b96e19f-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":376.69999998807907,"connectEnd":376.69999998807907,"connectStart":376.69999998807907,"domainLookupEnd":376.69999998807907,"domainLookupStart":376.69999998807907,"fetchStart":376.69999998807907,"redirectEnd":0,"redirectStart":0,"requestStart":376.69999998807907,"responseEnd":1216.699999988079,"responseStart":1216.699999988079,"secureConnectionStart":376.69999998807907},{"duration":843.6999999880791,"initiatorType":"script","name":"https://jira.mariadb.org/s/94c15bff32baef80f4096a08aceae8bc-CDN/lu2bu7/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":376.80000001192093,"connectEnd":376.80000001192093,"connectStart":376.80000001192093,"domainLookupEnd":376.80000001192093,"domainLookupStart":376.80000001192093,"fetchStart":376.80000001192093,"redirectEnd":0,"redirectStart":0,"requestStart":376.80000001192093,"responseEnd":1220.5,"responseStart":1220.5,"secureConnectionStart":376.80000001192093},{"duration":843.8000000119209,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":377.19999998807907,"connectEnd":377.19999998807907,"connectStart":377.19999998807907,"domainLookupEnd":377.19999998807907,"domainLookupStart":377.19999998807907,"fetchStart":377.19999998807907,"redirectEnd":0,"redirectStart":0,"requestStart":377.19999998807907,"responseEnd":1221,"responseStart":1221,"secureConnectionStart":377.19999998807907},{"duration":844.2999999523163,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":377.30000001192093,"connectEnd":377.30000001192093,"connectStart":377.30000001192093,"domainLookupEnd":377.30000001192093,"domainLookupStart":377.30000001192093,"fetchStart":377.30000001192093,"redirectEnd":0,"redirectStart":0,"requestStart":377.30000001192093,"responseEnd":1221.5999999642372,"responseStart":1221.5999999642372,"secureConnectionStart":377.30000001192093},{"duration":882.0999999642372,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2bu7/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":377.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":377.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1259.5999999642372,"responseStart":0,"secureConnectionStart":0},{"duration":844.3999999761581,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":377.69999998807907,"connectEnd":377.69999998807907,"connectStart":377.69999998807907,"domainLookupEnd":377.69999998807907,"domainLookupStart":377.69999998807907,"fetchStart":377.69999998807907,"redirectEnd":0,"redirectStart":0,"requestStart":377.69999998807907,"responseEnd":1222.0999999642372,"responseStart":1222.0999999642372,"secureConnectionStart":377.69999998807907},{"duration":882,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":377.89999997615814,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":377.89999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1259.8999999761581,"responseStart":0,"secureConnectionStart":0},{"duration":844.6999999880791,"initiatorType":"script","name":"https://jira.mariadb.org/s/3339d87fa2538a859872f2df449bf8d0-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":378,"connectEnd":378,"connectStart":378,"domainLookupEnd":378,"domainLookupStart":378,"fetchStart":378,"redirectEnd":0,"redirectStart":0,"requestStart":378,"responseEnd":1222.699999988079,"responseStart":1222.5999999642372,"secureConnectionStart":378},{"duration":1698.7999999523163,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":383.80000001192093,"connectEnd":383.80000001192093,"connectStart":383.80000001192093,"domainLookupEnd":383.80000001192093,"domainLookupStart":383.80000001192093,"fetchStart":383.80000001192093,"redirectEnd":0,"redirectStart":0,"requestStart":383.80000001192093,"responseEnd":2082.599999964237,"responseStart":2082.599999964237,"secureConnectionStart":383.80000001192093},{"duration":1699.199999988079,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":384.30000001192093,"connectEnd":384.30000001192093,"connectStart":384.30000001192093,"domainLookupEnd":384.30000001192093,"domainLookupStart":384.30000001192093,"fetchStart":384.30000001192093,"redirectEnd":0,"redirectStart":0,"requestStart":384.30000001192093,"responseEnd":2083.5,"responseStart":2083.5,"secureConnectionStart":384.30000001192093},{"duration":154.5,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1301.199999988079,"connectEnd":1301.199999988079,"connectStart":1301.199999988079,"domainLookupEnd":1301.199999988079,"domainLookupStart":1301.199999988079,"fetchStart":1301.199999988079,"redirectEnd":0,"redirectStart":0,"requestStart":1301.199999988079,"responseEnd":1455.699999988079,"responseStart":1455.699999988079,"secureConnectionStart":1301.199999988079},{"duration":515.6000000238419,"initiatorType":"link","name":"https://jira.mariadb.org/s/d5715adaadd168a9002b108b2b039b50-CDN/lu2bu7/820016/12ta74/be4b45e9cec53099498fa61c8b7acba4/_/download/contextbatch/css/jira.project.sidebar,-_super,-project.issue.navigator,-jira.general,-jira.browse.project,-jira.view.issue,-jira.global,-atl.general,-com.atlassian.jira.projects.sidebar.init/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":1569.5999999642372,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1569.5999999642372,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":2085.199999988079,"responseStart":0,"secureConnectionStart":0},{"duration":532.8000000119209,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/e65b778d185daf5aee24936755b43da6/_/download/contextbatch/js/browser-metrics-plugin.contrib,-_super,-project.issue.navigator,-jira.view.issue,-atl.general/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":1570.3999999761581,"connectEnd":1570.3999999761581,"connectStart":1570.3999999761581,"domainLookupEnd":1570.3999999761581,"domainLookupStart":1570.3999999761581,"fetchStart":1570.3999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":1570.3999999761581,"responseEnd":2103.199999988079,"responseStart":2103.199999988079,"secureConnectionStart":1570.3999999761581},{"duration":536.6000000238419,"initiatorType":"script","name":"https://jira.mariadb.org/s/f51ef5507eea4c158f257c66c93b2a3f-CDN/lu2bu7/820016/12ta74/be4b45e9cec53099498fa61c8b7acba4/_/download/contextbatch/js/jira.project.sidebar,-_super,-project.issue.navigator,-jira.general,-jira.browse.project,-jira.view.issue,-jira.global,-atl.general,-com.atlassian.jira.projects.sidebar.init/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":1570.8999999761581,"connectEnd":1570.8999999761581,"connectStart":1570.8999999761581,"domainLookupEnd":1570.8999999761581,"domainLookupStart":1570.8999999761581,"fetchStart":1570.8999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":1570.8999999761581,"responseEnd":2107.5,"responseStart":2107.5,"secureConnectionStart":1570.8999999761581}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":167,"responseStart":370,"responseEnd":381,"domLoading":374,"domInteractive":2124,"domContentLoadedEventStart":2124,"domContentLoadedEventEnd":2180,"domComplete":2551,"loadEventStart":2551,"loadEventEnd":2551,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":2090.699999988079},{"name":"bigPipe.sidebar-id.end","time":2091.800000011921},{"name":"bigPipe.activity-panel-pipe-id.start","time":2092},{"name":"bigPipe.activity-panel-pipe-id.end","time":2092.800000011921},{"name":"activityTabFullyLoaded","time":2193.300000011921}],"measures":[],"correlationId":"1258b1de883525","effectiveType":"4g","downlink":10,"rtt":0,"serverDuration":131,"dbReadsTimeInMs":22,"dbConnsTimeInMs":38,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}