Type:
Bug
Priority:
Major
Resolution:
Fixed
Affects Version/s:
2.3.7
I'm using MaxScale 2.3 with 3 MariaDB servers, 1 of them is master and the others are slaves and I set max_slave_replication_lag to 5 secs
[Read-Write-Service]
type=service
router=readwritesplit
servers=server1,server3,server2
max_slave_replication_lag=5
master_failure_mode=fail_on_write
After that I stop slaves for a period of time, and kept master running, all queries redirected to master, then slaves come back online, but they were hours behind master, when Monitor check slave status, sometime SHOW SLAVE STATUS return:
Slave_IO_Running Preparing
Seconds_Behind_Master NULL
then the Monitor decides that slave is up to date and redirect some queries to that slave, even slave is hours behind master!
I took a look over source code and IMHO I think this block of code is the reason
static inline bool rpl_lag_is_ok(SRWBackend& backend, int max_rlag)
{
return max_rlag == MXS_RLAG_UNDEFINED || backend->server()->rlag <= max_rlag;
}
So maybe removing ( max_rlag == MXS_RLAG_UNDEFINED) from condition might help in this case and damage in case slave was really up to date, maybe can check GTID for master and slave to determine that master and slave are really at same point of transaction
Master and slaves are running MariaDB 10.1.40
relates to
MXS-1720
Priori causal read
Closed
{"report":{"fcp":876.5,"ttfb":266.90000009536743,"pageVisibility":"visible","entityId":76031,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":1,"journeyId":"b8a5e499-53db-44de-bfb8-00c35d59ce1a","navigationType":0,"readyForUser":983.5999999046326,"redirectCount":0,"resourceLoadedEnd":1015.2999997138977,"resourceLoadedStart":274.90000009536743,"resourceTiming":[{"duration":145.7999997138977,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":274.90000009536743,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":274.90000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":420.69999980926514,"responseStart":0,"secureConnectionStart":0},{"duration":145.80000019073486,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":275.19999980926514,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":275.19999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":421,"responseStart":0,"secureConnectionStart":0},{"duration":154.09999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":275.5,"connectEnd":275.5,"connectStart":275.5,"domainLookupEnd":275.5,"domainLookupStart":275.5,"fetchStart":275.5,"redirectEnd":0,"redirectStart":0,"requestStart":275.5,"responseEnd":429.59999990463257,"responseStart":429.59999990463257,"secureConnectionStart":275.5},{"duration":239.59999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":276.09999990463257,"connectEnd":276.09999990463257,"connectStart":276.09999990463257,"domainLookupEnd":276.09999990463257,"domainLookupStart":276.09999990463257,"fetchStart":276.09999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":276.09999990463257,"responseEnd":515.6999998092651,"responseStart":515.6999998092651,"secureConnectionStart":276.09999990463257},{"duration":242.90000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":276.2999997138977,"connectEnd":276.2999997138977,"connectStart":276.2999997138977,"domainLookupEnd":276.2999997138977,"domainLookupStart":276.2999997138977,"fetchStart":276.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":276.2999997138977,"responseEnd":519.1999998092651,"responseStart":519.1999998092651,"secureConnectionStart":276.2999997138977},{"duration":243.2999997138977,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":276.40000009536743,"connectEnd":276.40000009536743,"connectStart":276.40000009536743,"domainLookupEnd":276.40000009536743,"domainLookupStart":276.40000009536743,"fetchStart":276.40000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":276.40000009536743,"responseEnd":519.6999998092651,"responseStart":519.6999998092651,"secureConnectionStart":276.40000009536743},{"duration":243.40000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":276.69999980926514,"connectEnd":276.69999980926514,"connectStart":276.69999980926514,"domainLookupEnd":276.69999980926514,"domainLookupStart":276.69999980926514,"fetchStart":276.69999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":276.69999980926514,"responseEnd":520.0999999046326,"responseStart":520.0999999046326,"secureConnectionStart":276.69999980926514},{"duration":293.8999996185303,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":276.90000009536743,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":276.90000009536743,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":570.7999997138977,"responseStart":0,"secureConnectionStart":0},{"duration":243.59999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":277,"connectEnd":277,"connectStart":277,"domainLookupEnd":277,"domainLookupStart":277,"fetchStart":277,"redirectEnd":0,"redirectStart":0,"requestStart":277,"responseEnd":520.5999999046326,"responseStart":520.5999999046326,"secureConnectionStart":277},{"duration":293.80000019073486,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":277.19999980926514,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":277.19999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":571,"responseStart":0,"secureConnectionStart":0},{"duration":243.7000002861023,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":277.2999997138977,"connectEnd":277.2999997138977,"connectStart":277.2999997138977,"domainLookupEnd":277.2999997138977,"domainLookupStart":277.2999997138977,"fetchStart":277.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":277.2999997138977,"responseEnd":521,"responseStart":521,"secureConnectionStart":277.2999997138977},{"duration":338.80000019073486,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":280.2999997138977,"connectEnd":280.2999997138977,"connectStart":280.2999997138977,"domainLookupEnd":280.2999997138977,"domainLookupStart":280.2999997138977,"fetchStart":280.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":280.2999997138977,"responseEnd":619.0999999046326,"responseStart":619.0999999046326,"secureConnectionStart":280.2999997138977},{"duration":339.90000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":280.69999980926514,"connectEnd":280.69999980926514,"connectStart":280.69999980926514,"domainLookupEnd":280.69999980926514,"domainLookupStart":280.69999980926514,"fetchStart":280.69999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":280.69999980926514,"responseEnd":620.5999999046326,"responseStart":620.5999999046326,"secureConnectionStart":280.69999980926514},{"duration":20.800000190734863,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":598.7999997138977,"connectEnd":598.7999997138977,"connectStart":598.7999997138977,"domainLookupEnd":598.7999997138977,"domainLookupStart":598.7999997138977,"fetchStart":598.7999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":598.7999997138977,"responseEnd":619.5999999046326,"responseStart":619.5999999046326,"secureConnectionStart":598.7999997138977},{"duration":392.59999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/e65b778d185daf5aee24936755b43da6/_/download/contextbatch/js/browser-metrics-plugin.contrib,-_super,-project.issue.navigator,-jira.view.issue,-atl.general/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":622.6999998092651,"connectEnd":622.6999998092651,"connectStart":622.6999998092651,"domainLookupEnd":622.6999998092651,"domainLookupStart":622.6999998092651,"fetchStart":622.6999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":622.6999998092651,"responseEnd":1015.2999997138977,"responseStart":1015.2999997138977,"secureConnectionStart":622.6999998092651}],"fetchStart":1,"domainLookupStart":18,"domainLookupEnd":82,"connectStart":83,"connectEnd":102,"secureConnectionStart":91,"requestStart":103,"responseStart":267,"responseEnd":281,"domLoading":270,"domInteractive":1036,"domContentLoadedEventStart":1036,"domContentLoadedEventEnd":1076,"domComplete":1275,"loadEventStart":1275,"loadEventEnd":1275,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1016.4000000953674},{"name":"bigPipe.sidebar-id.end","time":1017.2999997138977},{"name":"bigPipe.activity-panel-pipe-id.start","time":1017.5},{"name":"bigPipe.activity-panel-pipe-id.end","time":1018.6999998092651},{"name":"activityTabFullyLoaded","time":1096.0999999046326}],"measures":[],"correlationId":"bb41b06d017ce9","effectiveType":"4g","downlink":9.2,"rtt":0,"serverDuration":98,"dbReadsTimeInMs":14,"dbConnsTimeInMs":23,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
Seems that the monitor shouldn't update the replication lag if the replication hasn't started.