https://buildbot.mariadb.org/#/builders/98/builds/9752
10.11 16c971875
|
rpl.rpl_change_master_demote 'mix' w2 [ fail ]
|
Test ended at 2022-09-12 06:32:13
|
|
CURRENT_TEST: rpl.rpl_change_master_demote
|
mysqltest: In included file "./include/wait_for_slave_param.inc":
|
included from ./include/wait_for_slave_io_to_start.inc at line 40:
|
included from ./include/wait_for_slave_to_start.inc at line 26:
|
included from ./include/start_slave.inc at line 35:
|
included from /home/buildbot/aarch64-ubuntu-2004/build/mysql-test/suite/rpl/include/rpl_change_master_demote.inc at line 44:
|
included from /home/buildbot/aarch64-ubuntu-2004/build/mysql-test/suite/rpl/t/rpl_change_master_demote.test at line 147:
|
At line 119: Timeout in include/wait_for_slave_param.inc
|
|
The result from queries just before the failure was:
|
< snip >
|
master-bin.000001 256 Gtid_list 1 285 []
|
master-bin.000001 285 Binlog_checkpoint 1 329 master-bin.000001
|
master-bin.000001 329 Gtid 1 371 GTID 0-1-1
|
master-bin.000001 371 Query 1 470 use `test`; CREATE TABLE t1 (a int)
|
master-bin.000001 470 Gtid 1 512 BEGIN GTID 0-1-2
|
master-bin.000001 512 Query 1 604 use `test`; INSERT INTO t1 VALUES (1)
|
master-bin.000001 604 Query 1 677 COMMIT
|
master-bin.000001 677 Gtid 1 719 BEGIN GTID 0-1-6
|
master-bin.000001 719 Query 1 811 use `test`; INSERT INTO t1 VALUES (2)
|
master-bin.000001 811 Query 1 884 COMMIT
|
master-bin.000001 884 Gtid 1 926 BEGIN GTID 0-1-10
|
master-bin.000001 926 Query 1 1018 use `test`; INSERT INTO t1 VALUES (3)
|
master-bin.000001 1018 Query 1 1091 COMMIT
|
|
**** SHOW RELAYLOG EVENTS on server_1 ****
|
relaylog_name = 'master-relay-bin.000001'
|
SHOW RELAYLOG EVENTS IN 'master-relay-bin.000001';
|
Log_name Pos Event_type Server_id End_log_pos Info
|
master-relay-bin.000001 4 Format_desc 1 256 Server ver: 10.11.0-MariaDB-log, Binlog ver: 4
|
connection master;
|
This happens during Test Case 4, when the primary has changed to become the new replica and it is attempted to start it.
The primary/server 1 error log shows:
2022-09-12 6:26:49 9 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='', master_port='3306', master_log_file='', master_log_pos='4'. New state master_host='127.0.0.1', master_port='16081', master_log_file='', master_log_pos='4'.
|
2022-09-12 6:26:49 9 [Note] Previous Using_Gtid=Slave_Pos. New Using_Gtid=Slave_Pos
|
2022-09-12 6:26:49 23 [Note] Slave I/O thread: Start asynchronous replication to master 'root@127.0.0.1:16081' in log '' at position 4
|
2022-09-12 6:26:49 23 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16081',replication starts at GTID position '0-1-10'
|
2022-09-12 6:26:49 23 [ERROR] Slave I/O: Got fatal error 1236 from master when reading data from binary log: 'Error: connecting slave requested to start from GTID 0-1-10, which is not in the master's binlog', Internal MariaDB error code: 1236
|
2022-09-12 6:26:49 23 [Note] Slave I/O thread exiting, read up to log 'FIRST', position 4; GTID position 0-1-10, master 127.0.0.1:16081
|
2022-09-12 6:26:49 24 [Note] Slave SQL thread initialized, starting replication in log 'FIRST' at position 4, relay log './master-relay-bin.000001' position: 4; GTID position '0-1-10'
|
2022-09-12 6:32:12 0 [Note] /home/buildbot/aarch64-ubuntu-2004/build/sql/mariadbd (initiated by: unknown): Normal shutdown
|
2022-09-12 6:32:12 24 [Note] Error reading relay log event: slave SQL thread was killed
|
2022-09-12 6:32:12 24 [Note] Slave SQL thread exiting, replication stopped in log 'FIRST' at position 4; GTID position '0-1-10', master: 127.0.0.1:16081
|
2022-09-12 6:32:12 0 [Note] /home/buildbot/aarch64-ubuntu-2004/build/sql/mariadbd: Shutdown complete
|
The replica/server 2 error log shows:
2022-09-12 6:26:49 24 [Note] Start binlog_dump to slave_server(1), pos(, 4), using_gtid(1), gtid('0-1-10')
|
2022-09-12 6:32:12 22 [Note] Slave: received end packet from server, apparent master shutdown:
|
2022-09-12 6:32:12 22 [Note] Slave I/O thread: Failed reading log event, reconnecting to retry, log 'master-bin.000001' at position 1091; GTID position '0-1-10'
|
2022-09-12 6:32:12 22 [ERROR] Slave I/O: error reconnecting to master 'root@127.0.0.1:16080' - retry-time: 1 maximum-retries: 10 message: Can't connect to server on '127.0.0.1' (111 "Connection refused"), Internal MariaDB error code: 2003
|
2022-09-12 6:32:12 0 [Note] /home/buildbot/aarch64-ubuntu-2004/build/sql/mariadbd (initiated by: unknown): Normal shutdown
|
2022-09-12 6:32:12 23 [Note] Error reading relay log event: slave SQL thread was killed
|
2022-09-12 6:32:12 23 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1091; GTID position '0-1-10', master: 127.0.0.1:16080
|
2022-09-12 6:32:12 22 [Note] Slave I/O thread killed during or after a reconnect done to recover from failed read
|
2022-09-12 6:32:12 22 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 1091; GTID position 0-1-10, master 127.0.0.1:16080
|
2022-09-12 6:32:12 0 [Note] InnoDB: FTS optimize thread exiting.
|
2022-09-12 6:32:12 0 [Note] InnoDB: Starting shutdown...
|
2022-09-12 6:32:12 0 [Note] InnoDB: Dumping buffer pool(s) to /dev/shm/var_auto_caLU/2/mysqld.2/data/ib_buffer_pool
|
2022-09-12 6:32:12 0 [Note] InnoDB: Restricted to 126 pages due to innodb_buf_pool_dump_pct=25
|
2022-09-12 6:32:12 0 [Note] InnoDB: Buffer pool(s) dump completed at 220912 6:32:12
|
2022-09-12 6:32:13 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
|
2022-09-12 6:32:13 0 [Note] InnoDB: Shutdown completed; log sequence number 48342; transaction id 17
|
2022-09-12 6:32:13 0 [Note] /home/buildbot/aarch64-ubuntu-2004/build/sql/mariadbd: Shutdown complete
|
Error logs attached.
- relates to
-
MDEV-33316
rpl_change_master_demote binlog race condition
-
-
Stalled
There are no comments yet on this issue.
{"report":{"fcp":2641.800000190735,"ttfb":1096.1999998092651,"pageVisibility":"visible","entityId":114611,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"cf2573c7-8459-4386-b507-015894bc55f5","navigationType":0,"readyForUser":2979.199999809265,"redirectCount":0,"resourceLoadedEnd":2905.199999809265,"resourceLoadedStart":1226.3000001907349,"resourceTiming":[{"duration":881.8000001907349,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":1226.3000001907349,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1226.3000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":2108.1000003814697,"responseStart":0,"secureConnectionStart":0},{"duration":881.8999996185303,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/2bf333562ca6724060a9d5f1535471f6/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true","startTime":1226.6000003814697,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1226.6000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":2108.5,"responseStart":0,"secureConnectionStart":0},{"duration":891.3000001907349,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":1226.6999998092651,"connectEnd":1226.6999998092651,"connectStart":1226.6999998092651,"domainLookupEnd":1226.6999998092651,"domainLookupStart":1226.6999998092651,"fetchStart":1226.6999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":1226.6999998092651,"responseEnd":2118,"responseStart":2118,"secureConnectionStart":1226.6999998092651},{"duration":955,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/2bf333562ca6724060a9d5f1535471f6/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true","startTime":1227,"connectEnd":1227,"connectStart":1227,"domainLookupEnd":1227,"domainLookupStart":1227,"fetchStart":1227,"redirectEnd":0,"redirectStart":0,"requestStart":1227,"responseEnd":2182,"responseStart":2182,"secureConnectionStart":1227},{"duration":958.1999998092651,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":1227.3000001907349,"connectEnd":1227.3000001907349,"connectStart":1227.3000001907349,"domainLookupEnd":1227.3000001907349,"domainLookupStart":1227.3000001907349,"fetchStart":1227.3000001907349,"redirectEnd":0,"redirectStart":0,"requestStart":1227.3000001907349,"responseEnd":2185.5,"responseStart":2185.5,"secureConnectionStart":1227.3000001907349},{"duration":958.3999996185303,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":1227.5,"connectEnd":1227.5,"connectStart":1227.5,"domainLookupEnd":1227.5,"domainLookupStart":1227.5,"fetchStart":1227.5,"redirectEnd":0,"redirectStart":0,"requestStart":1227.5,"responseEnd":2185.8999996185303,"responseStart":2185.8999996185303,"secureConnectionStart":1227.5},{"duration":957.9000005722046,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":1228.3999996185303,"connectEnd":1228.3999996185303,"connectStart":1228.3999996185303,"domainLookupEnd":1228.3999996185303,"domainLookupStart":1228.3999996185303,"fetchStart":1228.3999996185303,"redirectEnd":0,"redirectStart":0,"requestStart":1228.3999996185303,"responseEnd":2186.300000190735,"responseStart":2186.300000190735,"secureConnectionStart":1228.3999996185303},{"duration":1053.7000007629395,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":1229.3999996185303,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1229.3999996185303,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":2283.1000003814697,"responseStart":0,"secureConnectionStart":0},{"duration":957.1999998092651,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":1229.6000003814697,"connectEnd":1229.6000003814697,"connectStart":1229.6000003814697,"domainLookupEnd":1229.6000003814697,"domainLookupStart":1229.6000003814697,"fetchStart":1229.6000003814697,"redirectEnd":0,"redirectStart":0,"requestStart":1229.6000003814697,"responseEnd":2186.800000190735,"responseStart":2186.800000190735,"secureConnectionStart":1229.6000003814697},{"duration":1053.3000001907349,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":1229.8999996185303,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1229.8999996185303,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":2283.199999809265,"responseStart":0,"secureConnectionStart":0},{"duration":957.5,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":1230,"connectEnd":1230,"connectStart":1230,"domainLookupEnd":1230,"domainLookupStart":1230,"fetchStart":1230,"redirectEnd":0,"redirectStart":0,"requestStart":1230,"responseEnd":2187.5,"responseStart":2187.5,"secureConnectionStart":1230},{"duration":1506.4000005722046,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":1230.8999996185303,"connectEnd":1230.8999996185303,"connectStart":1230.8999996185303,"domainLookupEnd":1230.8999996185303,"domainLookupStart":1230.8999996185303,"fetchStart":1230.8999996185303,"redirectEnd":0,"redirectStart":0,"requestStart":1230.8999996185303,"responseEnd":2737.300000190735,"responseStart":2737.300000190735,"secureConnectionStart":1230.8999996185303},{"duration":1674.1999998092651,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":1231,"connectEnd":1231,"connectStart":1231,"domainLookupEnd":1231,"domainLookupStart":1231,"fetchStart":1231,"redirectEnd":0,"redirectStart":0,"requestStart":1231,"responseEnd":2905.199999809265,"responseStart":2905.199999809265,"secureConnectionStart":1231},{"duration":445.9000005722046,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":2295.3999996185303,"connectEnd":2295.3999996185303,"connectStart":2295.3999996185303,"domainLookupEnd":2295.3999996185303,"domainLookupStart":2295.3999996185303,"fetchStart":2295.3999996185303,"redirectEnd":0,"redirectStart":0,"requestStart":2295.3999996185303,"responseEnd":2741.300000190735,"responseStart":2741.300000190735,"secureConnectionStart":2295.3999996185303},{"duration":595.6999998092651,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":2631,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":2631,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":3226.699999809265,"responseStart":0,"secureConnectionStart":0}],"fetchStart":0,"domainLookupStart":0,"domainLookupEnd":0,"connectStart":0,"connectEnd":0,"requestStart":901,"responseStart":1096,"responseEnd":1145,"domLoading":1180,"domInteractive":3314,"domContentLoadedEventStart":3314,"domContentLoadedEventEnd":3358,"domComplete":4296,"loadEventStart":4296,"loadEventEnd":4297,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":3291.6000003814697},{"name":"bigPipe.sidebar-id.end","time":3292.3999996185303},{"name":"bigPipe.activity-panel-pipe-id.start","time":3292.699999809265},{"name":"bigPipe.activity-panel-pipe-id.end","time":3293.1000003814697},{"name":"activityTabFullyLoaded","time":3366.699999809265}],"measures":[],"correlationId":"7dc10831551d5c","effectiveType":"4g","downlink":9.1,"rtt":0,"serverDuration":113,"dbReadsTimeInMs":15,"dbConnsTimeInMs":25,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}