UpdateCTestConfiguration from :/home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/DartConfiguration.tcl Parse Config file:/home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/DartConfiguration.tcl UpdateCTestConfiguration from :/home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/DartConfiguration.tcl Parse Config file:/home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/DartConfiguration.tcl Test project /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test Constructing a list of tests Done constructing a list of tests Checking test dependency graph... Checking test dependency graph end test 235 Start 235: binlog_change_master 235: Test command: /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/binlog_change_master "binlog_change_master" 235: Test timeout computed to be: 900 235: Redirecting to /bin/systemctl stop maxscale.service 235: Redirecting to /bin/systemctl stop maxscale.service 235: Number of slave hosts is 1 when it should be 3 235: Checking nodes... 235: Number of slave hosts is 1 when it should be 3 235: 02:33:13.218 30.038803: Template file is /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/cnf/maxscale.cnf.template.setup_binlog_tx_safe 235: maxscale: no process found 235: Redirecting to /bin/systemctl restart maxscale.service 235: 02:33:16.603 33.424430: Starting test 235: 02:33:23.529 6.925665: Starting binlog configuration 235: Stopping node 0 235: Stopping node 1 235: Stopping node 2 235: Stopping node 3 235: 02:33:35.031 18.427285: Stopping maxscale 235: Redirecting to /bin/systemctl stop maxscale.service 235: 02:33:45.872 29.268613: Connecting to all backend nodes 235: 02:33:45.888 29.285092: Stopping everything 235: 02:33:46.642 30.038549: Master server version '10.2.17-MariaDB-log' 235: 02:33:46.642 30.038601: Removing all binlog data from Maxscale node 235: 02:33:47.162 30.558911: Creating binlog dir 235: 02:33:47.681 31.078121: Set 'maxscale' as a owner of binlog dir 235: 02:33:48.180 31.576207: Testing binlog when MariaDB is started with ' ' option 235: 02:33:48.180 31.576233: ls binlog data dir on Maxscale node 235: 02:33:48.729 32.126100: show master status 235: 02:33:48.730 32.126498: Real master file: mar-bin.000001 235: 02:33:48.730 32.126509: Real master pos : 326 235: 02:33:48.730 32.126512: Stopping first slave (node 1) 235: 02:33:48.730 32.126771: Configure first backend slave node to be slave of real master 235: 02:33:49.161 32.557593: Starting back Maxscale 235: Redirecting to /bin/systemctl start maxscale.service 235: 02:33:49.964 33.360830: Connecting to MaxScale binlog router (with any DB) 235: 02:33:49.970 33.366913: configuring Maxscale binlog router 235: 02:33:50.008 33.404336: show master status 235: 02:33:50.008 33.404959: Maxscale binlog master file: mar-bin.000001 235: 02:33:50.008 33.404970: Maxscale binlog master pos : 4 235: 02:33:50.008 33.404974: Setup all backend nodes except first one to be slaves of binlog Maxscale node 235: 02:33:50.644 34.040338: Starting query thread 235: 02:33:50.644 34.040429: Sleeping 235: Creating test table 235: 02:33:50.651 34.048110: Flush logs on master 235: 02:34:06.287 49.683624: Flush logs on master 235: 02:35:57.949 161.346140: Blocking master 235: 02:36:01.080 164.477110: Done! Waiting for thread 235: 02:36:01.080 164.477148: Done! 235: 02:36:01.080 164.477153: Checking data on the node3 (slave) 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: WARNING: Slave has not caught up in 240 seconds. Test will most likely fail. Test #235: binlog_change_master ............. Passed 501.15 sec Start 235: binlog_change_master 235: Test command: /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/binlog_change_master "binlog_change_master" 235: Test timeout computed to be: 900 235: Redirecting to /bin/systemctl stop maxscale.service 235: Redirecting to /bin/systemctl stop maxscale.service 235: Number of slave hosts is 2 when it should be 3 235: Checking nodes... 235: Number of slave hosts is 2 when it should be 3 235: 02:41:34.318 29.493723: Template file is /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/cnf/maxscale.cnf.template.setup_binlog_tx_safe 235: maxscale: no process found 235: Redirecting to /bin/systemctl restart maxscale.service 235: 02:41:37.591 32.766443: Starting test 235: 02:41:44.260 6.668927: Starting binlog configuration 235: Stopping node 0 235: Stopping node 1 235: Stopping node 2 235: Stopping node 3 235: 02:41:55.575 17.984539: Stopping maxscale 235: Redirecting to /bin/systemctl stop maxscale.service 235: 02:42:06.599 29.008461: Connecting to all backend nodes 235: 02:42:06.610 29.019672: Stopping everything 235: 02:42:07.261 29.670758: Master server version '10.2.17-MariaDB-log' 235: 02:42:07.262 29.670793: Removing all binlog data from Maxscale node 235: 02:42:07.816 30.225338: Creating binlog dir 235: 02:42:08.347 30.756061: Set 'maxscale' as a owner of binlog dir 235: 02:42:08.859 31.267924: Testing binlog when MariaDB is started with ' ' option 235: 02:42:08.859 31.267949: ls binlog data dir on Maxscale node 235: 02:42:09.396 31.805412: show master status 235: 02:42:09.397 31.805896: Real master file: mar-bin.000001 235: 02:42:09.397 31.805907: Real master pos : 326 235: 02:42:09.397 31.805910: Stopping first slave (node 1) 235: 02:42:09.397 31.806153: Configure first backend slave node to be slave of real master 235: 02:42:09.552 31.961157: Starting back Maxscale 235: Redirecting to /bin/systemctl start maxscale.service 235: 02:42:10.330 32.739027: Connecting to MaxScale binlog router (with any DB) 235: 02:42:10.335 32.744291: configuring Maxscale binlog router 235: 02:42:10.360 32.768968: show master status 235: 02:42:10.361 32.770516: Maxscale binlog master file: mar-bin.000001 235: 02:42:10.361 32.770531: Maxscale binlog master pos : 4 235: 02:42:10.361 32.770536: Setup all backend nodes except first one to be slaves of binlog Maxscale node 235: 02:42:10.987 33.396313: Starting query thread 235: 02:42:10.987 33.396400: Sleeping 235: Creating test table 235: 02:42:10.991 33.400359: Flush logs on master 235: 02:42:26.626 49.034973: Flush logs on master 235: 02:44:05.676 148.085705: Blocking master 235: 02:44:08.827 151.235854: Done! Waiting for thread 235: 02:44:08.827 151.235892: Done! 235: 02:44:08.827 151.235897: Checking data on the node3 (slave) 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: WARNING: Slave has not caught up in 240 seconds. Test will most likely fail. Test #235: binlog_change_master ............. Passed 487.20 sec Start 235: binlog_change_master 235: Test command: /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/binlog_change_master "binlog_change_master" 235: Test timeout computed to be: 900 235: Redirecting to /bin/systemctl stop maxscale.service 235: Redirecting to /bin/systemctl stop maxscale.service 235: Number of slave hosts is 2 when it should be 3 235: Checking nodes... 235: Number of slave hosts is 2 when it should be 3 235: 02:49:40.934 28.908835: Template file is /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/cnf/maxscale.cnf.template.setup_binlog_tx_safe 235: maxscale: no process found 235: Redirecting to /bin/systemctl restart maxscale.service 235: 02:49:44.791 32.765693: Starting test 235: 02:49:51.280 6.489501: Starting binlog configuration 235: Stopping node 0 235: Stopping node 1 235: Stopping node 2 235: Stopping node 3 235: 02:50:06.985 22.194759: Stopping maxscale 235: Redirecting to /bin/systemctl stop maxscale.service 235: 02:50:17.823 33.031942: Connecting to all backend nodes 235: 02:50:17.838 33.047768: Stopping everything 235: 02:50:18.393 33.602258: Master server version '10.2.17-MariaDB-log' 235: 02:50:18.393 33.602279: Removing all binlog data from Maxscale node 235: 02:50:18.932 34.141120: Creating binlog dir 235: 02:50:19.462 34.671533: Set 'maxscale' as a owner of binlog dir 235: 02:50:19.975 35.183991: Testing binlog when MariaDB is started with ' ' option 235: 02:50:19.975 35.184015: ls binlog data dir on Maxscale node 235: 02:50:20.497 35.705985: show master status 235: 02:50:20.497 35.706475: Real master file: mar-bin.000001 235: 02:50:20.497 35.706488: Real master pos : 326 235: 02:50:20.497 35.706491: Stopping first slave (node 1) 235: 02:50:20.497 35.706754: Configure first backend slave node to be slave of real master 235: 02:50:20.639 35.848680: Starting back Maxscale 235: Redirecting to /bin/systemctl start maxscale.service 235: 02:50:21.390 36.599893: Connecting to MaxScale binlog router (with any DB) 235: 02:50:21.396 36.605872: configuring Maxscale binlog router 235: 02:50:21.438 36.647433: show master status 235: 02:50:21.439 36.648025: Maxscale binlog master file: mar-bin.000001 235: 02:50:21.439 36.648036: Maxscale binlog master pos : 4 235: 02:50:21.439 36.648039: Setup all backend nodes except first one to be slaves of binlog Maxscale node 235: 02:50:22.141 37.350042: Starting query thread 235: 02:50:22.141 37.350130: Sleeping 235: Creating test table 235: 02:50:22.145 37.354526: Flush logs on master 235: 02:50:37.677 52.886703: Flush logs on master 235: 02:52:15.527 150.735950: Blocking master 235: 02:52:18.682 153.891477: Done! Waiting for thread 235: 02:52:18.682 153.891512: Done! 235: 02:52:18.682 153.891517: Checking data on the node3 (slave) 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: WARNING: Slave has not caught up in 240 seconds. Test will most likely fail. Test #235: binlog_change_master ............. Passed 489.76 sec Start 235: binlog_change_master 235: Test command: /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/binlog_change_master "binlog_change_master" 235: Test timeout computed to be: 900 235: Redirecting to /bin/systemctl stop maxscale.service 235: Redirecting to /bin/systemctl stop maxscale.service 235: Number of slave hosts is 2 when it should be 3 235: Checking nodes... 235: Number of slave hosts is 2 when it should be 3 235: 02:57:50.806 29.020677: Template file is /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/cnf/maxscale.cnf.template.setup_binlog_tx_safe 235: maxscale: no process found 235: Redirecting to /bin/systemctl restart maxscale.service 235: 02:57:54.298 32.513124: Starting test 235: 02:58:01.238 6.939547: Starting binlog configuration 235: Stopping node 0 235: Stopping node 1 235: Stopping node 2 235: Stopping node 3 235: 02:58:12.484 18.186039: Stopping maxscale 235: Redirecting to /bin/systemctl stop maxscale.service 235: 02:58:23.309 29.011396: Connecting to all backend nodes 235: 02:58:23.319 29.021493: Stopping everything 235: 02:58:23.807 29.508730: Master server version '10.2.17-MariaDB-log' 235: 02:58:23.807 29.508761: Removing all binlog data from Maxscale node 235: 02:58:24.335 30.036899: Creating binlog dir 235: 02:58:24.868 30.570491: Set 'maxscale' as a owner of binlog dir 235: 02:58:25.392 31.094093: Testing binlog when MariaDB is started with ' ' option 235: 02:58:25.392 31.094120: ls binlog data dir on Maxscale node 235: 02:58:25.924 31.626053: show master status 235: 02:58:25.925 31.626600: Real master file: mar-bin.000001 235: 02:58:25.925 31.626613: Real master pos : 326 235: 02:58:25.925 31.626618: Stopping first slave (node 1) 235: 02:58:25.925 31.626915: Configure first backend slave node to be slave of real master 235: 02:58:26.084 31.786116: Starting back Maxscale 235: Redirecting to /bin/systemctl start maxscale.service 235: 02:58:26.797 32.498773: Connecting to MaxScale binlog router (with any DB) 235: 02:58:26.800 32.502332: configuring Maxscale binlog router 235: 02:58:26.822 32.523985: show master status 235: 02:58:26.823 32.524734: Maxscale binlog master file: mar-bin.000001 235: 02:58:26.823 32.524745: Maxscale binlog master pos : 4 235: 02:58:26.823 32.524749: Setup all backend nodes except first one to be slaves of binlog Maxscale node 235: 02:58:27.487 33.188809: Starting query thread 235: 02:58:27.487 33.188893: Sleeping 235: Creating test table 235: 02:58:27.490 33.191813: Flush logs on master 235: 02:58:43.054 48.756462: Flush logs on master 235: 03:00:09.406 135.107856: Blocking master 235: 03:00:12.485 138.187441: Done! Waiting for thread 235: 03:00:12.485 138.187478: Done! 235: 03:00:12.485 138.187483: Checking data on the node3 (slave) 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: WARNING: Slave has not caught up in 240 seconds. Test will most likely fail. Test #235: binlog_change_master ............. Passed 474.29 sec Start 235: binlog_change_master 235: Test command: /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/binlog_change_master "binlog_change_master" 235: Test timeout computed to be: 900 235: Redirecting to /bin/systemctl stop maxscale.service 235: Redirecting to /bin/systemctl stop maxscale.service 235: Number of slave hosts is 2 when it should be 3 235: Checking nodes... 235: Number of slave hosts is 2 when it should be 3 235: 03:05:43.941 27.867306: Template file is /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/cnf/maxscale.cnf.template.setup_binlog_tx_safe 235: maxscale: no process found 235: Redirecting to /bin/systemctl restart maxscale.service 235: 03:05:47.601 31.527607: Starting test 235: 03:05:53.657 6.055378: Starting binlog configuration 235: Stopping node 0 235: Stopping node 1 235: Stopping node 2 235: Stopping node 3 235: 03:06:04.898 17.297184: Stopping maxscale 235: Redirecting to /bin/systemctl stop maxscale.service 235: 03:06:15.344 27.742443: Connecting to all backend nodes 235: 03:06:15.360 27.758497: Stopping everything 235: 03:06:15.812 28.210468: Master server version '10.2.17-MariaDB-log' 235: 03:06:15.812 28.210498: Removing all binlog data from Maxscale node 235: 03:06:16.322 28.720819: Creating binlog dir 235: 03:06:16.811 29.210185: Set 'maxscale' as a owner of binlog dir 235: 03:06:17.313 29.711633: Testing binlog when MariaDB is started with ' ' option 235: 03:06:17.313 29.711658: ls binlog data dir on Maxscale node 235: 03:06:17.850 30.248820: show master status 235: 03:06:17.850 30.249340: Real master file: mar-bin.000001 235: 03:06:17.851 30.249353: Real master pos : 326 235: 03:06:17.851 30.249357: Stopping first slave (node 1) 235: 03:06:17.851 30.249655: Configure first backend slave node to be slave of real master 235: 03:06:18.120 30.519281: Starting back Maxscale 235: Redirecting to /bin/systemctl start maxscale.service 235: 03:06:18.869 31.267965: Connecting to MaxScale binlog router (with any DB) 235: 03:06:18.878 31.276507: configuring Maxscale binlog router 235: 03:06:18.917 31.315670: show master status 235: 03:06:18.918 31.316857: Maxscale binlog master file: mar-bin.000001 235: 03:06:18.918 31.316871: Maxscale binlog master pos : 4 235: 03:06:18.918 31.316875: Setup all backend nodes except first one to be slaves of binlog Maxscale node 235: 03:06:19.336 31.735340: Starting query thread 235: 03:06:19.337 31.735420: Sleeping 235: Creating test table 235: 03:06:19.341 31.739374: Flush logs on master 235: 03:06:34.777 47.176181: Flush logs on master 235: 03:08:02.300 134.698348: Blocking master 235: 03:08:05.403 137.802206: Done! Waiting for thread 235: 03:08:05.403 137.802248: Done! 235: 03:08:05.403 137.802251: Checking data on the node3 (slave) 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: WARNING: Slave has not caught up in 240 seconds. Test will most likely fail. Test #235: binlog_change_master ............. Passed 472.87 sec Start 235: binlog_change_master 235: Test command: /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/binlog_change_master "binlog_change_master" 235: Test timeout computed to be: 900 235: Redirecting to /bin/systemctl stop maxscale.service 235: Redirecting to /bin/systemctl stop maxscale.service 235: Number of slave hosts is 2 when it should be 3 235: Checking nodes... 235: Number of slave hosts is 2 when it should be 3 235: 03:13:38.035 29.094495: Template file is /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/cnf/maxscale.cnf.template.setup_binlog_tx_safe 235: maxscale: no process found 235: Redirecting to /bin/systemctl restart maxscale.service 235: 03:13:41.309 32.368508: Starting test 235: 03:13:47.859 6.550471: Starting binlog configuration 235: Stopping node 0 235: Stopping node 1 235: Stopping node 2 235: Stopping node 3 235: 03:13:59.778 18.468862: Stopping maxscale 235: Redirecting to /bin/systemctl stop maxscale.service 235: 03:14:10.257 28.948464: Connecting to all backend nodes 235: 03:14:10.271 28.962685: Stopping everything 235: 03:14:10.852 29.543230: Master server version '10.2.17-MariaDB-log' 235: 03:14:10.852 29.543257: Removing all binlog data from Maxscale node 235: 03:14:11.389 30.080231: Creating binlog dir 235: 03:14:11.968 30.659804: Set 'maxscale' as a owner of binlog dir 235: 03:14:12.480 31.171653: Testing binlog when MariaDB is started with ' ' option 235: 03:14:12.480 31.171679: ls binlog data dir on Maxscale node 235: 03:14:13.043 31.734635: show master status 235: 03:14:13.044 31.735291: Real master file: mar-bin.000001 235: 03:14:13.044 31.735304: Real master pos : 326 235: 03:14:13.044 31.735308: Stopping first slave (node 1) 235: 03:14:13.044 31.735583: Configure first backend slave node to be slave of real master 235: 03:14:13.282 31.972935: Starting back Maxscale 235: Redirecting to /bin/systemctl start maxscale.service 235: 03:14:14.038 32.729460: Connecting to MaxScale binlog router (with any DB) 235: 03:14:14.044 32.734874: configuring Maxscale binlog router 235: 03:14:14.069 32.760837: show master status 235: 03:14:14.071 32.762036: Maxscale binlog master file: mar-bin.000001 235: 03:14:14.071 32.762051: Maxscale binlog master pos : 4 235: 03:14:14.071 32.762056: Setup all backend nodes except first one to be slaves of binlog Maxscale node 235: 03:14:14.865 33.556786: Starting query thread 235: 03:14:14.866 33.556871: Sleeping 235: Creating test table 235: 03:14:14.869 33.560082: Flush logs on master 235: 03:14:30.132 48.823220: Flush logs on master 235: 03:16:10.959 149.650820: Blocking master 235: Error: can't execute SQL-query: START TRANSACTION 235: Lost connection to MySQL server during query 235: 235: 03:16:34.059 172.750003: START TRANSACTION Failed 235: 03:16:34.059 172.750028: Transaction 298 failed, doing master failover 235: 03:16:34.059 172.750033: Closing connection 235: 03:16:34.059 172.750047: Waiting for replication 235: 03:16:49.059 187.750139: Calling select_new_master() 235: 03:16:49.059 187.750170: Connection to backend 235: 03:17:39.109 237.799928: 'stop slave' to node2 235: 03:17:39.189 237.880537: 'reset slave all' to node2 235: 03:17:39.191 237.882356: show master status 235: 03:17:39.193 237.884035: Real master file: mar-bin.000001 235: 03:17:39.193 237.884056: Real master pos : 613937 235: 03:17:39.193 237.884063: Connecting to MaxScale binlog router (with any DB) 235: 03:17:39.194 237.885107: show master status on maxscale 235: 03:17:39.195 237.886071: Real master file: mar-bin.000003 235: 03:17:39.195 237.886089: Real master pos : 396336 235: 03:17:39.195 237.886099: log file num 000003 235: 03:17:39.195 237.886111: log file num (d) 3 235: 03:17:39.195 237.886116: log file name mar-bin. 235: 03:17:39.195 237.886124: Flush logs 4 times 235: 03:17:40.036 238.727592: configuring Maxscale binlog router 235: 03:17:40.036 238.727622: reconnect to binlog 235: 03:17:40.038 238.729154: Configuring new master with FILE and POS 235: 03:17:40.038 238.729177: change master query: change master to MASTER_HOST='192.168.121.247', MASTER_USER='repl', MASTER_PASSWORD='repl', MASTER_LOG_FILE='mar-bin.000004', MASTER_LOG_POS=4, MASTER_PORT=3306 235: 03:17:40.147 238.838203: start slave 235: 03:17:40.219 238.910061: start slave one more 235: 03:17:40.219 238.910447: slave started! 235: 03:17:40.220 238.911556: Retrying transaction 298 235: 03:17:45.041 243.732599: Done! Waiting for thread 235: 03:17:45.041 243.732646: Done! 235: 03:17:45.041 243.732654: Checking data on the node3 (slave) 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory Test #235: binlog_change_master ............. Passed 288.65 sec Start 235: binlog_change_master 235: Test command: /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/binlog_change_master "binlog_change_master" 235: Test timeout computed to be: 900 235: Redirecting to /bin/systemctl stop maxscale.service 235: Redirecting to /bin/systemctl stop maxscale.service 235: Number of slave hosts is 2 when it should be 3 235: Checking nodes... 235: Number of slave hosts is 1 when it should be 3 235: 03:18:25.801 28.211437: Template file is /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/cnf/maxscale.cnf.template.setup_binlog_tx_safe 235: maxscale: no process found 235: Redirecting to /bin/systemctl restart maxscale.service 235: 03:18:29.172 31.581750: Starting test 235: 03:18:35.656 6.484055: Starting binlog configuration 235: Stopping node 0 235: Stopping node 1 235: Stopping node 2 235: Stopping node 3 235: 03:18:51.599 22.427154: Stopping maxscale 235: Redirecting to /bin/systemctl stop maxscale.service 235: 03:19:02.190 33.018308: Connecting to all backend nodes 235: 03:19:02.205 33.032992: Stopping everything 235: 03:19:02.749 33.577299: Master server version '10.2.17-MariaDB-log' 235: 03:19:02.749 33.577326: Removing all binlog data from Maxscale node 235: 03:19:03.280 34.108888: Creating binlog dir 235: 03:19:03.803 34.631895: Set 'maxscale' as a owner of binlog dir 235: 03:19:04.340 35.168468: Testing binlog when MariaDB is started with ' ' option 235: 03:19:04.340 35.168494: ls binlog data dir on Maxscale node 235: 03:19:04.887 35.715881: show master status 235: 03:19:04.888 35.716556: Real master file: mar-bin.000001 235: 03:19:04.888 35.716572: Real master pos : 326 235: 03:19:04.888 35.716577: Stopping first slave (node 1) 235: 03:19:04.888 35.716931: Configure first backend slave node to be slave of real master 235: 03:19:05.133 35.961287: Starting back Maxscale 235: Redirecting to /bin/systemctl start maxscale.service 235: 03:19:05.920 36.748195: Connecting to MaxScale binlog router (with any DB) 235: 03:19:05.926 36.753998: configuring Maxscale binlog router 235: 03:19:05.963 36.791907: show master status 235: 03:19:05.964 36.792334: Maxscale binlog master file: mar-bin.000001 235: 03:19:05.964 36.792347: Maxscale binlog master pos : 4 235: 03:19:05.964 36.792351: Setup all backend nodes except first one to be slaves of binlog Maxscale node 235: 03:19:06.683 37.511886: Starting query thread 235: 03:19:06.684 37.511975: Sleeping 235: Creating test table 235: 03:19:06.688 37.516147: Flush logs on master 235: 03:19:22.239 53.067784: Flush logs on master 235: 03:20:56.454 147.282476: Blocking master 235: Error: can't execute SQL-query: START TRANSACTION 235: Lost connection to MySQL server during query 235: 235: 03:21:19.562 170.390050: START TRANSACTION Failed 235: 03:21:19.562 170.390076: Transaction 273 failed, doing master failover 235: 03:21:19.562 170.390092: Closing connection 235: 03:21:19.562 170.390105: Waiting for replication 235: 03:21:34.562 185.390204: Calling select_new_master() 235: 03:21:34.562 185.390235: Connection to backend 235: 03:22:24.613 235.441459: 'stop slave' to node2 235: 03:22:24.659 235.487502: 'reset slave all' to node2 235: 03:22:24.661 235.489210: show master status 235: 03:22:24.662 235.490928: Real master file: mar-bin.000001 235: 03:22:24.662 235.490948: Real master pos : 561562 235: 03:22:24.663 235.490955: Connecting to MaxScale binlog router (with any DB) 235: 03:22:24.664 235.492012: show master status on maxscale 235: 03:22:24.664 235.492668: Real master file: mar-bin.000003 235: 03:22:24.664 235.492680: Real master pos : 362816 235: 03:22:24.664 235.492689: log file num 000003 235: 03:22:24.664 235.492700: log file num (d) 3 235: 03:22:24.664 235.492705: log file name mar-bin. 235: 03:22:24.664 235.492711: Flush logs 4 times 235: 03:22:25.061 235.889299: configuring Maxscale binlog router 235: 03:22:25.061 235.889340: reconnect to binlog 235: 03:22:25.062 235.890750: Configuring new master with FILE and POS 235: 03:22:25.062 235.890773: change master query: change master to MASTER_HOST='192.168.121.247', MASTER_USER='repl', MASTER_PASSWORD='repl', MASTER_LOG_FILE='mar-bin.000004', MASTER_LOG_POS=4, MASTER_PORT=3306 235: 03:22:25.212 236.040009: start slave 235: 03:22:25.264 236.092770: start slave one more 235: 03:22:25.265 236.093200: slave started! 235: 03:22:25.266 236.094103: Retrying transaction 273 235: 03:22:30.543 241.371565: Done! Waiting for thread 235: 03:22:30.543 241.371602: Done! 235: 03:22:30.543 241.371606: Checking data on the node3 (slave) 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory Test #235: binlog_change_master ............. Passed 290.09 sec Start 235: binlog_change_master 235: Test command: /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/binlog_change_master "binlog_change_master" 235: Test timeout computed to be: 900 235: Redirecting to /bin/systemctl stop maxscale.service 235: Redirecting to /bin/systemctl stop maxscale.service 235: Number of slave hosts is 1 when it should be 3 235: Checking nodes... 235: Number of slave hosts is 1 when it should be 3 235: 03:23:16.473 28.793200: Template file is /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/cnf/maxscale.cnf.template.setup_binlog_tx_safe 235: maxscale: no process found 235: Redirecting to /bin/systemctl restart maxscale.service 235: 03:23:19.976 32.296658: Starting test 235: 03:23:26.447 6.470637: Starting binlog configuration 235: Stopping node 0 235: Stopping node 1 235: Stopping node 2 235: Stopping node 3 235: 03:23:37.692 17.716413: Stopping maxscale 235: Redirecting to /bin/systemctl stop maxscale.service 235: 03:23:48.096 28.119532: Connecting to all backend nodes 235: 03:23:48.111 28.135290: Stopping everything 235: 03:23:48.596 28.619625: Master server version '10.2.17-MariaDB-log' 235: 03:23:48.596 28.619647: Removing all binlog data from Maxscale node 235: 03:23:49.118 29.141807: Creating binlog dir 235: 03:23:49.641 29.665426: Set 'maxscale' as a owner of binlog dir 235: 03:23:50.154 30.178362: Testing binlog when MariaDB is started with ' ' option 235: 03:23:50.154 30.178387: ls binlog data dir on Maxscale node 235: 03:23:50.716 30.739824: show master status 235: 03:23:50.716 30.740483: Real master file: mar-bin.000001 235: 03:23:50.716 30.740495: Real master pos : 326 235: 03:23:50.716 30.740498: Stopping first slave (node 1) 235: 03:23:50.717 30.740848: Configure first backend slave node to be slave of real master 235: 03:23:50.950 30.973523: Starting back Maxscale 235: Redirecting to /bin/systemctl start maxscale.service 235: 03:23:51.704 31.728077: Connecting to MaxScale binlog router (with any DB) 235: 03:23:51.708 31.732485: configuring Maxscale binlog router 235: 03:23:51.735 31.758677: show master status 235: 03:23:51.735 31.759301: Maxscale binlog master file: mar-bin.000001 235: 03:23:51.735 31.759315: Maxscale binlog master pos : 4 235: 03:23:51.735 31.759320: Setup all backend nodes except first one to be slaves of binlog Maxscale node 235: 03:23:52.302 32.325723: Starting query thread 235: 03:23:52.302 32.325811: Sleeping 235: Creating test table 235: 03:23:52.306 32.330414: Flush logs on master 235: 03:24:07.845 47.868687: Flush logs on master 235: 03:25:49.257 149.281171: Blocking master 235: 03:25:52.495 152.518785: Done! Waiting for thread 235: 03:25:52.495 152.518817: Done! 235: 03:25:52.495 152.518820: Checking data on the node3 (slave) 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: WARNING: Slave has not caught up in 240 seconds. Test will most likely fail. Test #235: binlog_change_master ............. Passed 487.95 sec Start 235: binlog_change_master 235: Test command: /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/binlog_change_master "binlog_change_master" 235: Test timeout computed to be: 900 235: Redirecting to /bin/systemctl stop maxscale.service 235: Redirecting to /bin/systemctl stop maxscale.service 235: Number of slave hosts is 2 when it should be 3 235: Checking nodes... 235: Number of slave hosts is 2 when it should be 3 235: 03:31:24.817 29.185758: Template file is /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/cnf/maxscale.cnf.template.setup_binlog_tx_safe 235: maxscale: no process found 235: Redirecting to /bin/systemctl restart maxscale.service 235: 03:31:28.423 32.791590: Starting test 235: 03:31:34.969 6.545630: Starting binlog configuration 235: Stopping node 0 235: Stopping node 1 235: Stopping node 2 235: Stopping node 3 235: 03:31:46.540 18.117024: Stopping maxscale 235: Redirecting to /bin/systemctl stop maxscale.service 235: 03:31:57.478 29.055019: Connecting to all backend nodes 235: 03:31:57.495 29.071203: Stopping everything 235: 03:31:58.112 29.688644: Master server version '10.2.17-MariaDB-log' 235: 03:31:58.112 29.688674: Removing all binlog data from Maxscale node 235: 03:31:58.634 30.210793: Creating binlog dir 235: 03:31:59.165 30.741242: Set 'maxscale' as a owner of binlog dir 235: 03:31:59.675 31.252068: Testing binlog when MariaDB is started with ' ' option 235: 03:31:59.675 31.252091: ls binlog data dir on Maxscale node 235: 03:32:00.198 31.774483: show master status 235: 03:32:00.198 31.774999: Real master file: mar-bin.000001 235: 03:32:00.198 31.775010: Real master pos : 326 235: 03:32:00.198 31.775014: Stopping first slave (node 1) 235: 03:32:00.199 31.775256: Configure first backend slave node to be slave of real master 235: 03:32:00.366 31.942463: Starting back Maxscale 235: Redirecting to /bin/systemctl start maxscale.service 235: 03:32:01.110 32.686546: Connecting to MaxScale binlog router (with any DB) 235: 03:32:01.116 32.692384: configuring Maxscale binlog router 235: 03:32:01.158 32.734794: show master status 235: 03:32:01.159 32.735996: Maxscale binlog master file: mar-bin.000001 235: 03:32:01.159 32.736007: Maxscale binlog master pos : 4 235: 03:32:01.159 32.736011: Setup all backend nodes except first one to be slaves of binlog Maxscale node 235: 03:32:01.959 33.535969: Starting query thread 235: 03:32:01.959 33.536052: Sleeping 235: Creating test table 235: 03:32:01.963 33.539912: Flush logs on master 235: 03:32:17.710 49.287083: Flush logs on master 235: 03:34:03.007 154.583724: Blocking master 235: 03:34:06.182 157.758858: Done! Waiting for thread 235: 03:34:06.182 157.758897: Done! 235: 03:34:06.182 157.758903: Checking data on the node3 (slave) 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: WARNING: Slave has not caught up in 240 seconds. Test will most likely fail. Test #235: binlog_change_master ............. Passed 494.00 sec Start 235: binlog_change_master 235: Test command: /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/binlog_change_master "binlog_change_master" 235: Test timeout computed to be: 900 235: Redirecting to /bin/systemctl stop maxscale.service 235: Redirecting to /bin/systemctl stop maxscale.service 235: Number of slave hosts is 2 when it should be 3 235: Checking nodes... 235: Number of slave hosts is 2 when it should be 3 235: 03:39:39.451 29.821434: Template file is /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/cnf/maxscale.cnf.template.setup_binlog_tx_safe 235: maxscale: no process found 235: Redirecting to /bin/systemctl restart maxscale.service 235: 03:39:43.057 33.427910: Starting test 235: 03:39:49.752 6.694670: Starting binlog configuration 235: Stopping node 0 235: Stopping node 1 235: Stopping node 2 235: Stopping node 3 235: 03:40:01.166 18.108864: Stopping maxscale 235: Redirecting to /bin/systemctl stop maxscale.service 235: 03:40:12.134 29.076612: Connecting to all backend nodes 235: 03:40:12.149 29.092220: Stopping everything 235: 03:40:12.787 29.730159: Master server version '10.2.17-MariaDB-log' 235: 03:40:12.787 29.730187: Removing all binlog data from Maxscale node 235: 03:40:13.335 30.278244: Creating binlog dir 235: 03:40:13.867 30.810226: Set 'maxscale' as a owner of binlog dir 235: 03:40:14.374 31.316521: Testing binlog when MariaDB is started with ' ' option 235: 03:40:14.374 31.316555: ls binlog data dir on Maxscale node 235: 03:40:14.920 31.862678: show master status 235: 03:40:14.920 31.863181: Real master file: mar-bin.000001 235: 03:40:14.920 31.863191: Real master pos : 326 235: 03:40:14.920 31.863194: Stopping first slave (node 1) 235: 03:40:14.921 31.863504: Configure first backend slave node to be slave of real master 235: 03:40:15.099 32.042329: Starting back Maxscale 235: Redirecting to /bin/systemctl start maxscale.service 235: 03:40:15.868 32.810767: Connecting to MaxScale binlog router (with any DB) 235: 03:40:15.872 32.814526: configuring Maxscale binlog router 235: 03:40:15.893 32.836191: show master status 235: 03:40:15.897 32.839891: Maxscale binlog master file: mar-bin.000001 235: 03:40:15.897 32.839913: Maxscale binlog master pos : 4 235: 03:40:15.897 32.839918: Setup all backend nodes except first one to be slaves of binlog Maxscale node 235: 03:40:16.701 33.643850: Starting query thread 235: 03:40:16.701 33.643937: Sleeping 235: Creating test table 235: 03:40:16.704 33.647279: Flush logs on master 235: 03:40:32.194 49.137404: Flush logs on master 235: 03:42:02.828 139.770553: Blocking master 235: 03:42:05.914 142.856762: Done! Waiting for thread 235: 03:42:05.914 142.856793: Done! 235: 03:42:05.914 142.856796: Checking data on the node3 (slave) 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: WARNING: Slave has not caught up in 240 seconds. Test will most likely fail. Test #235: binlog_change_master ............. Passed 479.57 sec Start 235: binlog_change_master 235: Test command: /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/binlog_change_master "binlog_change_master" 235: Test timeout computed to be: 900 235: Redirecting to /bin/systemctl stop maxscale.service 235: Redirecting to /bin/systemctl stop maxscale.service 235: Number of slave hosts is 2 when it should be 3 235: Checking nodes... 235: Number of slave hosts is 2 when it should be 3 235: 03:47:37.965 28.769614: Template file is /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/cnf/maxscale.cnf.template.setup_binlog_tx_safe 235: maxscale: no process found 235: Redirecting to /bin/systemctl restart maxscale.service 235: 03:47:41.276 32.080163: Starting test 235: 03:47:47.733 6.457041: Starting binlog configuration 235: Stopping node 0 235: Stopping node 1 235: Stopping node 2 235: Stopping node 3 235: 03:47:58.454 17.178807: Stopping maxscale 235: Redirecting to /bin/systemctl stop maxscale.service 235: 03:48:09.347 28.070958: Connecting to all backend nodes 235: 03:48:09.363 28.086986: Stopping everything 235: 03:48:10.246 28.970584: Master server version '10.2.17-MariaDB-log' 235: 03:48:10.246 28.970609: Removing all binlog data from Maxscale node 235: 03:48:10.766 29.489934: Creating binlog dir 235: 03:48:11.288 30.012019: Set 'maxscale' as a owner of binlog dir 235: 03:48:11.810 30.534334: Testing binlog when MariaDB is started with ' ' option 235: 03:48:11.810 30.534360: ls binlog data dir on Maxscale node 235: 03:48:12.349 31.072982: show master status 235: 03:48:12.349 31.073701: Real master file: mar-bin.000001 235: 03:48:12.349 31.073711: Real master pos : 326 235: 03:48:12.349 31.073714: Stopping first slave (node 1) 235: 03:48:12.350 31.074066: Configure first backend slave node to be slave of real master 235: 03:48:12.461 31.185684: Starting back Maxscale 235: Redirecting to /bin/systemctl start maxscale.service 235: 03:48:13.147 31.871160: Connecting to MaxScale binlog router (with any DB) 235: 03:48:13.151 31.875850: configuring Maxscale binlog router 235: 03:48:13.191 31.915395: show master status 235: 03:48:13.192 31.916633: Maxscale binlog master file: mar-bin.000001 235: 03:48:13.192 31.916649: Maxscale binlog master pos : 4 235: 03:48:13.192 31.916653: Setup all backend nodes except first one to be slaves of binlog Maxscale node 235: 03:48:13.859 32.583540: Starting query thread 235: 03:48:13.859 32.583622: Sleeping 235: Creating test table 235: 03:48:13.863 32.587638: Flush logs on master 235: 03:48:29.322 48.046597: Flush logs on master 235: 03:50:13.191 151.915703: Blocking master 235: 03:50:16.248 154.971934: Done! Waiting for thread 235: 03:50:16.248 154.971966: Done! 235: 03:50:16.248 154.971971: Checking data on the node3 (slave) 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: WARNING: Slave has not caught up in 240 seconds. Test will most likely fail. Test #235: binlog_change_master ............. Passed 490.66 sec Start 235: binlog_change_master 235: Test command: /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/binlog_change_master "binlog_change_master" 235: Test timeout computed to be: 900 235: Redirecting to /bin/systemctl stop maxscale.service 235: Redirecting to /bin/systemctl stop maxscale.service 235: Number of slave hosts is 2 when it should be 3 235: Checking nodes... 235: Number of slave hosts is 2 when it should be 3 235: 03:55:49.367 29.510724: Template file is /home/turenko/Maxscale-newgcc/MaxScale/maxscale-system-test/cnf/maxscale.cnf.template.setup_binlog_tx_safe 235: maxscale: no process found 235: Redirecting to /bin/systemctl restart maxscale.service 235: 03:55:53.222 33.365254: Starting test 235: 03:56:00.181 6.959638: Starting binlog configuration 235: Stopping node 0 235: Stopping node 1 235: Stopping node 2 235: Stopping node 3 235: 03:56:12.691 19.469059: Stopping maxscale 235: Redirecting to /bin/systemctl stop maxscale.service 235: 03:56:23.465 30.243798: Connecting to all backend nodes 235: 03:56:23.480 30.258475: Stopping everything 235: 03:56:24.239 31.017016: Master server version '10.2.17-MariaDB-log' 235: 03:56:24.239 31.017045: Removing all binlog data from Maxscale node 235: 03:56:24.785 31.563040: Creating binlog dir 235: 03:56:25.334 32.112207: Set 'maxscale' as a owner of binlog dir 235: 03:56:25.863 32.640987: Testing binlog when MariaDB is started with ' ' option 235: 03:56:25.863 32.641010: ls binlog data dir on Maxscale node 235: 03:56:26.392 33.169970: show master status 235: 03:56:26.392 33.170702: Real master file: mar-bin.000001 235: 03:56:26.392 33.170716: Real master pos : 326 235: 03:56:26.392 33.170719: Stopping first slave (node 1) 235: 03:56:26.393 33.170967: Configure first backend slave node to be slave of real master 235: 03:56:26.784 33.562459: Starting back Maxscale 235: Redirecting to /bin/systemctl start maxscale.service 235: 03:56:27.516 34.294776: Connecting to MaxScale binlog router (with any DB) 235: 03:56:27.520 34.298537: configuring Maxscale binlog router 235: 03:56:27.543 34.320950: show master status 235: 03:56:27.544 34.322908: Maxscale binlog master file: mar-bin.000001 235: 03:56:27.545 34.322927: Maxscale binlog master pos : 4 235: 03:56:27.545 34.322931: Setup all backend nodes except first one to be slaves of binlog Maxscale node 235: 03:56:28.208 34.986283: Starting query thread 235: 03:56:28.208 34.986375: Sleeping 235: Creating test table 235: 03:56:28.212 34.990039: Flush logs on master 235: 03:56:43.824 50.601954: Flush logs on master 235: 03:57:56.276 123.054278: Blocking master 235: Error: can't execute SQL-query: START TRANSACTION 235: Lost connection to MySQL server during query 235: 235: 03:58:19.407 146.185778: START TRANSACTION Failed 235: 03:58:19.407 146.185803: Transaction 203 failed, doing master failover 235: 03:58:19.407 146.185808: Closing connection 235: 03:58:19.407 146.185821: Waiting for replication 235: 03:58:34.407 161.185888: Calling select_new_master() 235: 03:58:34.408 161.185927: Connection to backend 235: 03:59:24.444 211.222242: 'stop slave' to node2 235: 03:59:24.492 211.270786: 'reset slave all' to node2 235: 03:59:24.494 211.272349: show master status 235: 03:59:24.496 211.274263: Real master file: mar-bin.000001 235: 03:59:24.496 211.274296: Real master pos : 414912 235: 03:59:24.496 211.274302: Connecting to MaxScale binlog router (with any DB) 235: 03:59:24.497 211.275375: show master status on maxscale 235: 03:59:24.498 211.276053: Real master file: mar-bin.000003 235: 03:59:24.498 211.276069: Real master pos : 246091 235: 03:59:24.498 211.276078: log file num 000003 235: 03:59:24.498 211.276090: log file num (d) 3 235: 03:59:24.498 211.276093: log file name mar-bin. 235: 03:59:24.498 211.276099: Flush logs 4 times 235: 03:59:25.107 211.885052: configuring Maxscale binlog router 235: 03:59:25.107 211.885094: reconnect to binlog 235: 03:59:25.108 211.886711: Configuring new master with FILE and POS 235: 03:59:25.108 211.886738: change master query: change master to MASTER_HOST='192.168.121.247', MASTER_USER='repl', MASTER_PASSWORD='repl', MASTER_LOG_FILE='mar-bin.000004', MASTER_LOG_POS=4, MASTER_PORT=3306 235: 03:59:25.211 211.988940: start slave 235: 03:59:25.253 212.031552: start slave one more 235: 03:59:25.253 212.031880: slave started! 235: 03:59:25.255 212.033215: Retrying transaction 203 235: 03:59:44.388 231.166699: Done! Waiting for thread 235: 03:59:44.388 231.166736: Done! 235: 03:59:44.388 231.166740: Checking data on the node3 (slave) 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: cp: cannot stat ‘/tmp/core*’: No such file or directory 235: ls: cannot access /tmp/core*: No such file or directory 235: 239.249591: TEST_FAILED! Transaction 287 did not put data into slave 235: 239.259698: TEST_FAILED! Transaction 288 did not put data into slave 235: 239.269529: TEST_FAILED! Transaction 289 did not put data into slave 235: 239.279264: TEST_FAILED! Transaction 290 did not put data into slave 235: 239.288942: TEST_FAILED! Transaction 291 did not put data into slave 235: 239.298516: TEST_FAILED! Transaction 292 did not put data into slave 235: 239.308136: TEST_FAILED! Transaction 293 did not put data into slave 235: 239.318541: TEST_FAILED! Transaction 294 did not put data into slave 235: 239.328505: TEST_FAILED! Transaction 295 did not put data into slave 235: 239.338292: TEST_FAILED! Transaction 296 did not put data into slave 235: 239.347964: TEST_FAILED! Transaction 297 did not put data into slave 235: 239.357639: TEST_FAILED! Transaction 298 did not put data into slave Test #235: binlog_change_master .............***Failed 282.73 sec 0% tests passed, 1 tests failed out of 1 Label Time Summary: BREAKS_REPL = 282.73 sec (1 test) binlogrouter = 282.73 sec (1 test) Total Test time (real) = 5239.42 sec The following tests FAILED: 235 - binlog_change_master (Failed)