I have configured a MariaDB 10.2.4 GTID Slave with master_delay=3600, and it seems to cause replication to lag behind more and more. Waits trigger even though Seconds_Behind_Master is > 3600.
Hi tanj
I have created a patch for this issue which solves this wrong timestamp issue, This is because when binlog_background thread logs BInlog_checkpoint_log_event time is not set in the thread so , Binlog_checkpoint_log_event->when is zero.
But I am not able to simulate seconds behind master case. Can you please provide some reproducible test case ?
I tried this
master - slave configuration
both use binlog_format = row
I have used gtid
sql_delay=60
And run lacs of (insert )query on server 1 , with after every 10 insert there was flush logs
Sachin Setiya (Inactive)
added a comment - Hi tanj
I have created a patch for this issue which solves this wrong timestamp issue, This is because when binlog_background thread logs BInlog_checkpoint_log_event time is not set in the thread so , Binlog_checkpoint_log_event->when is zero.
But I am not able to simulate seconds behind master case. Can you please provide some reproducible test case ?
I tried this
master - slave configuration
both use binlog_format = row
I have used gtid
sql_delay=60
And run lacs of (insert )query on server 1 , with after every 10 insert there was flush logs
patch
Doesn't the set_time() need to go inside the loop, before the call to
mysql_bin_log.mark_xid_done()? Otherwise the time will always be the time
the binlog thread started, which is not correct - it needs to be the current
time.
But nice catch, good to get this fixed.
Kristian Nielsen
added a comment - Doesn't the set_time() need to go inside the loop, before the call to
mysql_bin_log.mark_xid_done()? Otherwise the time will always be the time
the binlog thread started, which is not correct - it needs to be the current
time.
But nice catch, good to get this fixed.
I am experiencing this issue when attaching a 10.2.8 Slave to a 10.0.24 Master. At every #700101 Binlog checkpoint event, the SQL Delay counter is reapplied and the Slave is continually delayed.
master# mysql -e 'select @@version;'
-----------------
@@version
-----------------
10.0.24-MariaDB
-----------------
master# mysqlbinlog bin-log.023901 | grep ^#700
#700101 2:00:00 server id 123 end_log_pos 423 Binlog checkpoint bin-log.023901
slave# mysql -e 'select @@version;'
--------------------
@@version
--------------------
10.2.8-MariaDB-log
--------------------
Seconds_Behind_Master: 2170
SQL_Delay: 900
SQL_Remaining_Delay: 825
Slave_SQL_Running_State: Waiting until MASTER_DELAY seconds after master executed event
Even if the Master is upgraded to 10.0.32 (latest), the bogus timestamps will still be present - as confirmed in a sandbox:
../sandbox.sh mysql -e 'select @@version;'
-----------------
@@version
-----------------
10.0.32-MariaDB
-----------------
mysqlbinlog bin-log.00000* | grep ^#700
#700101 2:00:00 server id 10032 end_log_pos 347 Binlog checkpoint bin-log.000002
#700101 2:00:00 server id 10032 end_log_pos 347 Binlog checkpoint bin-log.000003
Marcus Zambrano
added a comment - I am experiencing this issue when attaching a 10.2.8 Slave to a 10.0.24 Master. At every #700101 Binlog checkpoint event, the SQL Delay counter is reapplied and the Slave is continually delayed.
master# mysql -e 'select @@version;'
-----------------
@@version
-----------------
10.0.24-MariaDB
-----------------
master# mysqlbinlog bin-log.023901 | grep ^#700
#700101 2:00:00 server id 123 end_log_pos 423 Binlog checkpoint bin-log.023901
slave# mysql -e 'select @@version;'
--------------------
@@version
--------------------
10.2.8-MariaDB-log
--------------------
Seconds_Behind_Master: 2170
SQL_Delay: 900
SQL_Remaining_Delay: 825
Slave_SQL_Running_State: Waiting until MASTER_DELAY seconds after master executed event
Even if the Master is upgraded to 10.0.32 (latest), the bogus timestamps will still be present - as confirmed in a sandbox:
../sandbox.sh mysql -e 'select @@version;'
-----------------
@@version
-----------------
10.0.32-MariaDB
-----------------
mysqlbinlog bin-log.00000* | grep ^#700
#700101 2:00:00 server id 10032 end_log_pos 347 Binlog checkpoint bin-log.000002
#700101 2:00:00 server id 10032 end_log_pos 347 Binlog checkpoint bin-log.000003
This patched was pushed only to 10.2, I have back-ported this patch to 10.0 , by this merge this will also be available to 10.1.
Sachin
Sachin Setiya (Inactive)
added a comment - Hi zam1081 !
This patched was pushed only to 10.2, I have back-ported this patch to 10.0 , by this merge this will also be available to 10.1.
Sachin
People
Sachin Setiya (Inactive)
Guillaume Lefranc
Votes:
1Vote for this issue
Watchers:
8Start watching this issue
Dates
Created:
Updated:
Resolved:
Git Integration
Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.
{"report":{"fcp":1980,"ttfb":660.5,"pageVisibility":"visible","entityId":60611,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"5e19b979-de73-4cdc-9282-3c92d33b6c13","navigationType":0,"readyForUser":2055.600000023842,"redirectCount":0,"resourceLoadedEnd":2365.8000000715256,"resourceLoadedStart":667.1000000238419,"resourceTiming":[{"duration":817.5,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":667.1000000238419,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":667.1000000238419,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1484.6000000238419,"responseStart":0,"secureConnectionStart":0},{"duration":817.3999999761581,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":667.3000000715256,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":667.3000000715256,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1484.7000000476837,"responseStart":0,"secureConnectionStart":0},{"duration":826,"initiatorType":"script","name":"https://jira.mariadb.org/s/fbf975c0cce4b1abf04784eeae9ba1f4-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":667.5,"connectEnd":667.5,"connectStart":667.5,"domainLookupEnd":667.5,"domainLookupStart":667.5,"fetchStart":667.5,"redirectEnd":0,"redirectStart":0,"requestStart":667.5,"responseEnd":1493.5,"responseStart":1493.5,"secureConnectionStart":667.5},{"duration":1005.8999999761581,"initiatorType":"script","name":"https://jira.mariadb.org/s/099b33461394b8015fc36c0a4b96e19f-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":667.6000000238419,"connectEnd":667.6000000238419,"connectStart":667.6000000238419,"domainLookupEnd":667.6000000238419,"domainLookupStart":667.6000000238419,"fetchStart":667.6000000238419,"redirectEnd":0,"redirectStart":0,"requestStart":667.6000000238419,"responseEnd":1673.5,"responseStart":1673.5,"secureConnectionStart":667.6000000238419},{"duration":1009.2999999523163,"initiatorType":"script","name":"https://jira.mariadb.org/s/94c15bff32baef80f4096a08aceae8bc-CDN/lu2bu7/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":667.7000000476837,"connectEnd":667.7000000476837,"connectStart":667.7000000476837,"domainLookupEnd":667.7000000476837,"domainLookupStart":667.7000000476837,"fetchStart":667.7000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":667.7000000476837,"responseEnd":1677,"responseStart":1677,"secureConnectionStart":667.7000000476837},{"duration":1009.7999999523163,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":667.7000000476837,"connectEnd":667.7000000476837,"connectStart":667.7000000476837,"domainLookupEnd":667.7000000476837,"domainLookupStart":667.7000000476837,"fetchStart":667.7000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":667.7000000476837,"responseEnd":1677.5,"responseStart":1677.5,"secureConnectionStart":667.7000000476837},{"duration":1010.3000000715256,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2bu7/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":667.8999999761581,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":667.8999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1678.2000000476837,"responseStart":0,"secureConnectionStart":0},{"duration":1009.9000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":667.8999999761581,"connectEnd":667.8999999761581,"connectStart":667.8999999761581,"domainLookupEnd":667.8999999761581,"domainLookupStart":667.8999999761581,"fetchStart":667.8999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":667.8999999761581,"responseEnd":1677.8000000715256,"responseStart":1677.8000000715256,"secureConnectionStart":667.8999999761581},{"duration":1010.1000000238419,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":668.1000000238419,"connectEnd":668.1000000238419,"connectStart":668.1000000238419,"domainLookupEnd":668.1000000238419,"domainLookupStart":668.1000000238419,"fetchStart":668.1000000238419,"redirectEnd":0,"redirectStart":0,"requestStart":668.1000000238419,"responseEnd":1678.2000000476837,"responseStart":1678.2000000476837,"secureConnectionStart":668.1000000238419},{"duration":1010.3999999761581,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":668.2000000476837,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":668.2000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1678.6000000238419,"responseStart":0,"secureConnectionStart":0},{"duration":1010.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/3339d87fa2538a859872f2df449bf8d0-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":668.3000000715256,"connectEnd":668.3000000715256,"connectStart":668.3000000715256,"domainLookupEnd":668.3000000715256,"domainLookupStart":668.3000000715256,"fetchStart":668.3000000715256,"redirectEnd":0,"redirectStart":0,"requestStart":668.3000000715256,"responseEnd":1678.8000000715256,"responseStart":1678.7000000476837,"secureConnectionStart":668.3000000715256},{"duration":1696.4000000953674,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":668.8999999761581,"connectEnd":668.8999999761581,"connectStart":668.8999999761581,"domainLookupEnd":668.8999999761581,"domainLookupStart":668.8999999761581,"fetchStart":668.8999999761581,"redirectEnd":0,"redirectStart":0,"requestStart":668.8999999761581,"responseEnd":2365.3000000715256,"responseStart":2365.3000000715256,"secureConnectionStart":668.8999999761581},{"duration":1696.8000000715256,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":669,"connectEnd":669,"connectStart":669,"domainLookupEnd":669,"domainLookupStart":669,"fetchStart":669,"redirectEnd":0,"redirectStart":0,"requestStart":669,"responseEnd":2365.8000000715256,"responseStart":2365.8000000715256,"secureConnectionStart":669},{"duration":461.60000002384186,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1642.6000000238419,"connectEnd":1642.6000000238419,"connectStart":1642.6000000238419,"domainLookupEnd":1642.6000000238419,"domainLookupStart":1642.6000000238419,"fetchStart":1642.6000000238419,"redirectEnd":0,"redirectStart":0,"requestStart":1642.6000000238419,"responseEnd":2104.2000000476837,"responseStart":2104.2000000476837,"secureConnectionStart":1642.6000000238419},{"duration":393,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":1973.5,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1973.5,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":2366.5,"responseStart":0,"secureConnectionStart":0}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":480,"responseStart":660,"responseEnd":664,"domLoading":664,"domInteractive":2408,"domContentLoadedEventStart":2408,"domContentLoadedEventEnd":2451,"domComplete":3287,"loadEventStart":3287,"loadEventEnd":3288,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":2369.100000023842},{"name":"bigPipe.sidebar-id.end","time":2369.8000000715256},{"name":"bigPipe.activity-panel-pipe-id.start","time":2370},{"name":"bigPipe.activity-panel-pipe-id.end","time":2371.8000000715256},{"name":"activityTabFullyLoaded","time":2466.8000000715256}],"measures":[],"correlationId":"c10cde4b8a4911","effectiveType":"4g","downlink":10,"rtt":0,"serverDuration":114,"dbReadsTimeInMs":14,"dbConnsTimeInMs":23,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
Bug
Critical
Hi tanj
I have created a patch for this issue which solves this wrong timestamp issue, This is because when binlog_background thread logs BInlog_checkpoint_log_event time is not set in the thread so , Binlog_checkpoint_log_event->when is zero.
But I am not able to simulate seconds behind master case. Can you please provide some reproducible test case ?
I tried this
master - slave configuration
both use binlog_format = row
I have used gtid
sql_delay=60
And run lacs of (insert )query on server 1 , with after every 10 insert there was flush logs
patch