The second node can't join the first node because SST will get killed by systemd after the default timeout hits.
systemctl show mariadb.service | grep Timeout will show timeout set to 1m 30s for startup, but an SST can last hours with large dataset and/or slow disks and/or slow networks.
In fact, it is common for an SST to take several hours in production.
Setting TimeoutSec=0 under Services in the mariadb.service config file under systemd fixes the problem.
Right now, it is impossible to deploy Galera Cluster under 10.2.13 and CentOS 7 unless the above workaround is in place.
{"report":{"fcp":1029,"ttfb":401.89999997615814,"pageVisibility":"visible","entityId":66403,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"91dd6c03-7a12-4b0c-93ec-1caca8ca52a8","navigationType":0,"readyForUser":1134.1000000238419,"redirectCount":0,"resourceLoadedEnd":752.7999999523163,"resourceLoadedStart":410.60000002384186,"resourceTiming":[{"duration":57.299999952316284,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":410.60000002384186,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":410.60000002384186,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":467.89999997615814,"responseStart":0,"secureConnectionStart":0},{"duration":57.60000002384186,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":411,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":411,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":468.60000002384186,"responseStart":0,"secureConnectionStart":0},{"duration":233.39999997615814,"initiatorType":"script","name":"https://jira.mariadb.org/s/fbf975c0cce4b1abf04784eeae9ba1f4-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":411.2000000476837,"connectEnd":411.2000000476837,"connectStart":411.2000000476837,"domainLookupEnd":411.2000000476837,"domainLookupStart":411.2000000476837,"fetchStart":411.2000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":471.2999999523163,"responseEnd":644.6000000238419,"responseStart":485,"secureConnectionStart":411.2000000476837},{"duration":341,"initiatorType":"script","name":"https://jira.mariadb.org/s/099b33461394b8015fc36c0a4b96e19f-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":411.7999999523163,"connectEnd":411.7999999523163,"connectStart":411.7999999523163,"domainLookupEnd":411.7999999523163,"domainLookupStart":411.7999999523163,"fetchStart":411.7999999523163,"redirectEnd":0,"redirectStart":0,"requestStart":471.5,"responseEnd":752.7999999523163,"responseStart":490.7000000476837,"secureConnectionStart":411.7999999523163},{"duration":71.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/94c15bff32baef80f4096a08aceae8bc-CDN/lu2bu7/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":412,"connectEnd":412,"connectStart":412,"domainLookupEnd":412,"domainLookupStart":412,"fetchStart":412,"redirectEnd":0,"redirectStart":0,"requestStart":472,"responseEnd":483.5,"responseStart":481.7000000476837,"secureConnectionStart":412},{"duration":79.59999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":412.2000000476837,"connectEnd":412.2000000476837,"connectStart":412.2000000476837,"domainLookupEnd":412.2000000476837,"domainLookupStart":412.2000000476837,"fetchStart":412.2000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":473,"responseEnd":491.7999999523163,"responseStart":489.60000002384186,"secureConnectionStart":412.2000000476837},{"duration":81.39999997615814,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":412.39999997615814,"connectEnd":412.39999997615814,"connectStart":412.39999997615814,"domainLookupEnd":412.39999997615814,"domainLookupStart":412.39999997615814,"fetchStart":412.39999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":474.60000002384186,"responseEnd":493.7999999523163,"responseStart":492,"secureConnectionStart":412.39999997615814},{"duration":61,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2bu7/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":412.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":412.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":473.5,"responseStart":0,"secureConnectionStart":0},{"duration":84.5,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":412.7000000476837,"connectEnd":412.7000000476837,"connectStart":412.7000000476837,"domainLookupEnd":412.7000000476837,"domainLookupStart":412.7000000476837,"fetchStart":412.7000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":476.2999999523163,"responseEnd":497.2000000476837,"responseStart":494.10000002384186,"secureConnectionStart":412.7000000476837},{"duration":62.300000071525574,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":412.89999997615814,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":412.89999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":475.2000000476837,"responseStart":0,"secureConnectionStart":0},{"duration":84.80000007152557,"initiatorType":"script","name":"https://jira.mariadb.org/s/3339d87fa2538a859872f2df449bf8d0-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":412.89999997615814,"connectEnd":412.89999997615814,"connectStart":412.89999997615814,"domainLookupEnd":412.89999997615814,"domainLookupStart":412.89999997615814,"fetchStart":412.89999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":478,"responseEnd":497.7000000476837,"responseStart":495.2000000476837,"secureConnectionStart":412.89999997615814},{"duration":328.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":414.60000002384186,"connectEnd":414.60000002384186,"connectStart":414.60000002384186,"domainLookupEnd":414.60000002384186,"domainLookupStart":414.60000002384186,"fetchStart":414.60000002384186,"redirectEnd":0,"redirectStart":0,"requestStart":544.2000000476837,"responseEnd":743.1000000238419,"responseStart":736.8999999761581,"secureConnectionStart":414.60000002384186},{"duration":325.6999999284744,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":422.7000000476837,"connectEnd":422.7000000476837,"connectStart":422.7000000476837,"domainLookupEnd":422.7000000476837,"domainLookupStart":422.7000000476837,"fetchStart":422.7000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":603.2999999523163,"responseEnd":748.3999999761581,"responseStart":741.3999999761581,"secureConnectionStart":422.7000000476837},{"duration":103.60000002384186,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":788.6000000238419,"connectEnd":788.6000000238419,"connectStart":788.6000000238419,"domainLookupEnd":788.6000000238419,"domainLookupStart":788.6000000238419,"fetchStart":788.6000000238419,"redirectEnd":0,"redirectStart":0,"requestStart":849.7000000476837,"responseEnd":892.2000000476837,"responseStart":890.8999999761581,"secureConnectionStart":788.6000000238419},{"duration":237.79999995231628,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1028,"connectEnd":1028,"connectStart":1028,"domainLookupEnd":1028,"domainLookupStart":1028,"fetchStart":1028,"redirectEnd":0,"redirectStart":0,"requestStart":1230.7000000476837,"responseEnd":1265.7999999523163,"responseStart":1264.2000000476837,"secureConnectionStart":1028}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":203,"responseStart":401,"responseEnd":422,"domLoading":405,"domInteractive":1195,"domContentLoadedEventStart":1195,"domContentLoadedEventEnd":1243,"domComplete":1687,"loadEventStart":1687,"loadEventEnd":1687,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1167.1000000238419},{"name":"bigPipe.sidebar-id.end","time":1168},{"name":"bigPipe.activity-panel-pipe-id.start","time":1168.1000000238419},{"name":"bigPipe.activity-panel-pipe-id.end","time":1170.6000000238419},{"name":"activityTabFullyLoaded","time":1267.1000000238419}],"measures":[],"correlationId":"e55274aa0a026c","effectiveType":"4g","downlink":9.1,"rtt":0,"serverDuration":135,"dbReadsTimeInMs":16,"dbConnsTimeInMs":38,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
Bug
Critical
MDEV-14705 systemd: EXTEND_TIMEOUT_USEC= to avoid startup and shutdown timeouts
the same issue was observed with data set of ~12G when 3rd Node was joining
sst failed with wsrep_sst_method=mariabackup but also with set rsync
joiner: => Rate:[ 39MiB/s] Avg:[32.9MiB/s] Elapsed:0:01:20
WSREP_SST: [ERROR] Removing /var/lib/mysql//.sst/xtrabackup_galera_info file due to signal (20180320 16:13:50.761)
WSREP_SST: [ERROR] Cleanup after exit with status:143 (20180320 16:13:50.765)
2018-03-20 16:13:50 140406339643136 [ERROR] WSREP: Process completed with error: wsrep_sst_mariabackup --role 'joiner' --address '192.168.104.193' --datadir '/var/lib/mysql/' --parent '13420' '' : 4 (Interrupted system call)