https://buildbot.mariadb.org/#/builders/98/builds/9752
10.11 16c971875
|
rpl.rpl_change_master_demote 'mix' w2 [ fail ]
|
Test ended at 2022-09-12 06:32:13
|
|
CURRENT_TEST: rpl.rpl_change_master_demote
|
mysqltest: In included file "./include/wait_for_slave_param.inc":
|
included from ./include/wait_for_slave_io_to_start.inc at line 40:
|
included from ./include/wait_for_slave_to_start.inc at line 26:
|
included from ./include/start_slave.inc at line 35:
|
included from /home/buildbot/aarch64-ubuntu-2004/build/mysql-test/suite/rpl/include/rpl_change_master_demote.inc at line 44:
|
included from /home/buildbot/aarch64-ubuntu-2004/build/mysql-test/suite/rpl/t/rpl_change_master_demote.test at line 147:
|
At line 119: Timeout in include/wait_for_slave_param.inc
|
|
The result from queries just before the failure was:
|
< snip >
|
master-bin.000001 256 Gtid_list 1 285 []
|
master-bin.000001 285 Binlog_checkpoint 1 329 master-bin.000001
|
master-bin.000001 329 Gtid 1 371 GTID 0-1-1
|
master-bin.000001 371 Query 1 470 use `test`; CREATE TABLE t1 (a int)
|
master-bin.000001 470 Gtid 1 512 BEGIN GTID 0-1-2
|
master-bin.000001 512 Query 1 604 use `test`; INSERT INTO t1 VALUES (1)
|
master-bin.000001 604 Query 1 677 COMMIT
|
master-bin.000001 677 Gtid 1 719 BEGIN GTID 0-1-6
|
master-bin.000001 719 Query 1 811 use `test`; INSERT INTO t1 VALUES (2)
|
master-bin.000001 811 Query 1 884 COMMIT
|
master-bin.000001 884 Gtid 1 926 BEGIN GTID 0-1-10
|
master-bin.000001 926 Query 1 1018 use `test`; INSERT INTO t1 VALUES (3)
|
master-bin.000001 1018 Query 1 1091 COMMIT
|
|
**** SHOW RELAYLOG EVENTS on server_1 ****
|
relaylog_name = 'master-relay-bin.000001'
|
SHOW RELAYLOG EVENTS IN 'master-relay-bin.000001';
|
Log_name Pos Event_type Server_id End_log_pos Info
|
master-relay-bin.000001 4 Format_desc 1 256 Server ver: 10.11.0-MariaDB-log, Binlog ver: 4
|
connection master;
|
This happens during Test Case 4, when the primary has changed to become the new replica and it is attempted to start it.
The primary/server 1 error log shows:
2022-09-12 6:26:49 9 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='', master_port='3306', master_log_file='', master_log_pos='4'. New state master_host='127.0.0.1', master_port='16081', master_log_file='', master_log_pos='4'.
|
2022-09-12 6:26:49 9 [Note] Previous Using_Gtid=Slave_Pos. New Using_Gtid=Slave_Pos
|
2022-09-12 6:26:49 23 [Note] Slave I/O thread: Start asynchronous replication to master 'root@127.0.0.1:16081' in log '' at position 4
|
2022-09-12 6:26:49 23 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16081',replication starts at GTID position '0-1-10'
|
2022-09-12 6:26:49 23 [ERROR] Slave I/O: Got fatal error 1236 from master when reading data from binary log: 'Error: connecting slave requested to start from GTID 0-1-10, which is not in the master's binlog', Internal MariaDB error code: 1236
|
2022-09-12 6:26:49 23 [Note] Slave I/O thread exiting, read up to log 'FIRST', position 4; GTID position 0-1-10, master 127.0.0.1:16081
|
2022-09-12 6:26:49 24 [Note] Slave SQL thread initialized, starting replication in log 'FIRST' at position 4, relay log './master-relay-bin.000001' position: 4; GTID position '0-1-10'
|
2022-09-12 6:32:12 0 [Note] /home/buildbot/aarch64-ubuntu-2004/build/sql/mariadbd (initiated by: unknown): Normal shutdown
|
2022-09-12 6:32:12 24 [Note] Error reading relay log event: slave SQL thread was killed
|
2022-09-12 6:32:12 24 [Note] Slave SQL thread exiting, replication stopped in log 'FIRST' at position 4; GTID position '0-1-10', master: 127.0.0.1:16081
|
2022-09-12 6:32:12 0 [Note] /home/buildbot/aarch64-ubuntu-2004/build/sql/mariadbd: Shutdown complete
|
The replica/server 2 error log shows:
2022-09-12 6:26:49 24 [Note] Start binlog_dump to slave_server(1), pos(, 4), using_gtid(1), gtid('0-1-10')
|
2022-09-12 6:32:12 22 [Note] Slave: received end packet from server, apparent master shutdown:
|
2022-09-12 6:32:12 22 [Note] Slave I/O thread: Failed reading log event, reconnecting to retry, log 'master-bin.000001' at position 1091; GTID position '0-1-10'
|
2022-09-12 6:32:12 22 [ERROR] Slave I/O: error reconnecting to master 'root@127.0.0.1:16080' - retry-time: 1 maximum-retries: 10 message: Can't connect to server on '127.0.0.1' (111 "Connection refused"), Internal MariaDB error code: 2003
|
2022-09-12 6:32:12 0 [Note] /home/buildbot/aarch64-ubuntu-2004/build/sql/mariadbd (initiated by: unknown): Normal shutdown
|
2022-09-12 6:32:12 23 [Note] Error reading relay log event: slave SQL thread was killed
|
2022-09-12 6:32:12 23 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1091; GTID position '0-1-10', master: 127.0.0.1:16080
|
2022-09-12 6:32:12 22 [Note] Slave I/O thread killed during or after a reconnect done to recover from failed read
|
2022-09-12 6:32:12 22 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 1091; GTID position 0-1-10, master 127.0.0.1:16080
|
2022-09-12 6:32:12 0 [Note] InnoDB: FTS optimize thread exiting.
|
2022-09-12 6:32:12 0 [Note] InnoDB: Starting shutdown...
|
2022-09-12 6:32:12 0 [Note] InnoDB: Dumping buffer pool(s) to /dev/shm/var_auto_caLU/2/mysqld.2/data/ib_buffer_pool
|
2022-09-12 6:32:12 0 [Note] InnoDB: Restricted to 126 pages due to innodb_buf_pool_dump_pct=25
|
2022-09-12 6:32:12 0 [Note] InnoDB: Buffer pool(s) dump completed at 220912 6:32:12
|
2022-09-12 6:32:13 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
|
2022-09-12 6:32:13 0 [Note] InnoDB: Shutdown completed; log sequence number 48342; transaction id 17
|
2022-09-12 6:32:13 0 [Note] /home/buildbot/aarch64-ubuntu-2004/build/sql/mariadbd: Shutdown complete
|
Error logs attached.
- relates to
-
MDEV-33316
rpl_change_master_demote binlog race condition
-
-
Stalled
There are no comments yet on this issue.
{"report":{"fcp":749.2999999523163,"ttfb":216,"pageVisibility":"visible","entityId":114611,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":1,"journeyId":"b40961ae-9573-4e0b-a4a6-ebda04340c4d","navigationType":0,"readyForUser":826.7000000476837,"redirectCount":0,"resourceLoadedEnd":872.7000000476837,"resourceLoadedStart":221.39999997615814,"resourceTiming":[{"duration":11.200000047683716,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":221.39999997615814,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":221.39999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":232.60000002384186,"responseStart":0,"secureConnectionStart":0},{"duration":13.799999952316284,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":221.70000004768372,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":221.70000004768372,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":235.5,"responseStart":0,"secureConnectionStart":0},{"duration":69.90000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/s/fbf975c0cce4b1abf04784eeae9ba1f4-CDN/lu2bu7/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":221.79999995231628,"connectEnd":221.79999995231628,"connectStart":221.79999995231628,"domainLookupEnd":221.79999995231628,"domainLookupStart":221.79999995231628,"fetchStart":221.79999995231628,"redirectEnd":0,"redirectStart":0,"requestStart":221.79999995231628,"responseEnd":291.7000000476837,"responseStart":291.7000000476837,"secureConnectionStart":221.79999995231628},{"duration":144.60000002384186,"initiatorType":"script","name":"https://jira.mariadb.org/s/099b33461394b8015fc36c0a4b96e19f-CDN/lu2bu7/820016/12ta74/8679b4946efa1a0bb029a3a22206fb5d/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":221.79999995231628,"connectEnd":221.79999995231628,"connectStart":221.79999995231628,"domainLookupEnd":221.79999995231628,"domainLookupStart":221.79999995231628,"fetchStart":221.79999995231628,"redirectEnd":0,"redirectStart":0,"requestStart":221.79999995231628,"responseEnd":366.39999997615814,"responseStart":366.39999997615814,"secureConnectionStart":221.79999995231628},{"duration":147.79999995231628,"initiatorType":"script","name":"https://jira.mariadb.org/s/94c15bff32baef80f4096a08aceae8bc-CDN/lu2bu7/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":222,"connectEnd":222,"connectStart":222,"domainLookupEnd":222,"domainLookupStart":222,"fetchStart":222,"redirectEnd":0,"redirectStart":0,"requestStart":222,"responseEnd":369.7999999523163,"responseStart":369.7999999523163,"secureConnectionStart":222},{"duration":148.29999995231628,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":222,"connectEnd":222,"connectStart":222,"domainLookupEnd":222,"domainLookupStart":222,"fetchStart":222,"redirectEnd":0,"redirectStart":0,"requestStart":222,"responseEnd":370.2999999523163,"responseStart":370.2999999523163,"secureConnectionStart":222},{"duration":148.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":222.10000002384186,"connectEnd":222.10000002384186,"connectStart":222.10000002384186,"domainLookupEnd":222.10000002384186,"domainLookupStart":222.10000002384186,"fetchStart":222.10000002384186,"redirectEnd":0,"redirectStart":0,"requestStart":222.10000002384186,"responseEnd":370.60000002384186,"responseStart":370.60000002384186,"secureConnectionStart":222.10000002384186},{"duration":148.70000004768372,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2bu7/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":222.29999995231628,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":222.29999995231628,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":371,"responseStart":0,"secureConnectionStart":0},{"duration":149.10000002384186,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":222.39999997615814,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":222.39999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":371.5,"responseStart":0,"secureConnectionStart":0},{"duration":148.60000002384186,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":222.39999997615814,"connectEnd":222.39999997615814,"connectStart":222.39999997615814,"domainLookupEnd":222.39999997615814,"domainLookupStart":222.39999997615814,"fetchStart":222.39999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":222.39999997615814,"responseEnd":371,"responseStart":371,"secureConnectionStart":222.39999997615814},{"duration":149,"initiatorType":"script","name":"https://jira.mariadb.org/s/3339d87fa2538a859872f2df449bf8d0-CDN/lu2bu7/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":222.60000002384186,"connectEnd":222.60000002384186,"connectStart":222.60000002384186,"domainLookupEnd":222.60000002384186,"domainLookupStart":222.60000002384186,"fetchStart":222.60000002384186,"redirectEnd":0,"redirectStart":0,"requestStart":222.60000002384186,"responseEnd":371.60000002384186,"responseStart":371.60000002384186,"secureConnectionStart":222.60000002384186},{"duration":348.39999997615814,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":223.20000004768372,"connectEnd":223.20000004768372,"connectStart":223.20000004768372,"domainLookupEnd":223.20000004768372,"domainLookupStart":223.20000004768372,"fetchStart":223.20000004768372,"redirectEnd":0,"redirectStart":0,"requestStart":223.20000004768372,"responseEnd":571.6000000238419,"responseStart":571.6000000238419,"secureConnectionStart":223.20000004768372},{"duration":416.2999999523163,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":227.5,"connectEnd":227.5,"connectStart":227.5,"domainLookupEnd":227.5,"domainLookupStart":227.5,"fetchStart":227.5,"redirectEnd":0,"redirectStart":0,"requestStart":227.5,"responseEnd":643.7999999523163,"responseStart":643.7999999523163,"secureConnectionStart":227.5},{"duration":131.20000004768372,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":503.39999997615814,"connectEnd":503.39999997615814,"connectStart":503.39999997615814,"domainLookupEnd":503.39999997615814,"domainLookupStart":503.39999997615814,"fetchStart":503.39999997615814,"redirectEnd":0,"redirectStart":0,"requestStart":503.39999997615814,"responseEnd":634.6000000238419,"responseStart":634.6000000238419,"secureConnectionStart":503.39999997615814},{"duration":167,"initiatorType":"link","name":"https://jira.mariadb.org/s/d5715adaadd168a9002b108b2b039b50-CDN/lu2bu7/820016/12ta74/be4b45e9cec53099498fa61c8b7acba4/_/download/contextbatch/css/jira.project.sidebar,-_super,-project.issue.navigator,-jira.general,-jira.browse.project,-jira.view.issue,-jira.global,-atl.general,-com.atlassian.jira.projects.sidebar.init/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":698.2999999523163,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":698.2999999523163,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":865.2999999523163,"responseStart":0,"secureConnectionStart":0},{"duration":166.69999992847443,"initiatorType":"link","name":"https://jira.mariadb.org/s/50bc9be5bfead1a25e72c1a9338c94f6-CDN/lu2bu7/820016/12ta74/03ec8e95fb0700cf309083e2358eb54b/_/download/contextbatch/css/com.atlassian.jira.plugins.jira-development-integration-plugin:0,com.atlassian.jira.plugins.jira-quicksearch-plugin:5,-_super,-jira.view.issue,-jira.global,-jira.general,-jira.browse.project,-project.issue.navigator,-atl.general/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":698.7000000476837,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":698.7000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":865.3999999761581,"responseStart":0,"secureConnectionStart":0},{"duration":166.29999995231628,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2bu7/820016/12ta74/e65b778d185daf5aee24936755b43da6/_/download/contextbatch/js/browser-metrics-plugin.contrib,-_super,-project.issue.navigator,-jira.view.issue,-atl.general/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":699.5,"connectEnd":699.5,"connectStart":699.5,"domainLookupEnd":699.5,"domainLookupStart":699.5,"fetchStart":699.5,"redirectEnd":0,"redirectStart":0,"requestStart":699.5,"responseEnd":865.7999999523163,"responseStart":865.7999999523163,"secureConnectionStart":699.5},{"duration":171.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/f51ef5507eea4c158f257c66c93b2a3f-CDN/lu2bu7/820016/12ta74/be4b45e9cec53099498fa61c8b7acba4/_/download/contextbatch/js/jira.project.sidebar,-_super,-project.issue.navigator,-jira.general,-jira.browse.project,-jira.view.issue,-jira.global,-atl.general,-com.atlassian.jira.projects.sidebar.init/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":699.7999999523163,"connectEnd":699.7999999523163,"connectStart":699.7999999523163,"domainLookupEnd":699.7999999523163,"domainLookupStart":699.7999999523163,"fetchStart":699.7999999523163,"redirectEnd":0,"redirectStart":0,"requestStart":699.7999999523163,"responseEnd":871.2999999523163,"responseStart":871.2999999523163,"secureConnectionStart":699.7999999523163},{"duration":172.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/0b63859f61e7b6c2acdd2eef749beab8-CDN/lu2bu7/820016/12ta74/03ec8e95fb0700cf309083e2358eb54b/_/download/contextbatch/js/com.atlassian.jira.plugins.jira-development-integration-plugin:0,com.atlassian.jira.plugins.jira-quicksearch-plugin:5,-_super,-jira.view.issue,-jira.global,-jira.general,-jira.browse.project,-project.issue.navigator,-atl.general/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":700.2000000476837,"connectEnd":700.2000000476837,"connectStart":700.2000000476837,"domainLookupEnd":700.2000000476837,"domainLookupStart":700.2000000476837,"fetchStart":700.2000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":700.2000000476837,"responseEnd":872.7000000476837,"responseStart":872.7000000476837,"secureConnectionStart":700.2000000476837},{"duration":152.19999992847443,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":729.7000000476837,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":729.7000000476837,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":881.8999999761581,"responseStart":0,"secureConnectionStart":0}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":51,"responseStart":216,"responseEnd":221,"domLoading":219,"domInteractive":907,"domContentLoadedEventStart":907,"domContentLoadedEventEnd":945,"domComplete":1124,"loadEventStart":1124,"loadEventEnd":1127,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":869.2000000476837},{"name":"bigPipe.sidebar-id.end","time":869.7999999523163},{"name":"bigPipe.activity-panel-pipe-id.start","time":870},{"name":"bigPipe.activity-panel-pipe-id.end","time":870.5},{"name":"activityTabFullyLoaded","time":955.5}],"measures":[],"correlationId":"c1b68cbc6ef48f","effectiveType":"4g","downlink":9.6,"rtt":0,"serverDuration":91,"dbReadsTimeInMs":11,"dbConnsTimeInMs":18,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}