[MDEV-13549] Galera 3 test failures Created: 2017-08-16  Updated: 2019-12-18  Resolved: 2019-07-03

Status: Closed
Project: MariaDB Server
Component/s: Galera
Affects Version/s: 10.1, 10.2, 10.3, 10.4
Fix Version/s: N/A

Type: Bug Priority: Critical
Reporter: Jan Lindström (Inactive) Assignee: Jan Lindström (Inactive)
Resolution: Duplicate Votes: 0
Labels: contribution, foundation

Attachments: Zip Archive galera_gra_log.zip    
Issue Links:
Blocks
is blocked by MDEV-12234 wsrep.pool_of_threads fails in buildb... Closed
is blocked by MDEV-13738 galera_applier_ftwrl_table_alter may ... Closed
is blocked by MDEV-13743 galera_toi_truncate may fail with: qu... Closed
is blocked by MDEV-13871 galera.galera_unicode_identifiers fai... Closed
is blocked by MDEV-13873 galera.galera_suspend_slave failed in... Closed
is blocked by MDEV-13874 galera.galera_wsrep_log_conficts fail... Closed
is blocked by MDEV-13876 galera.MW-328A failed in buildbot wit... Closed
is blocked by MDEV-13877 galera.galera_gcs_fc_limit fails in b... Closed
is blocked by MDEV-13879 galera.galera_wan fails in buildbot w... Closed
is blocked by MDEV-13880 galera.galera_var_node_address fails ... Closed
is blocked by MDEV-13881 galera.partition failed in buildbot w... Closed
is blocked by MDEV-13883 galera.query_cache fails in buildbot ... Closed
is blocked by MDEV-14142 galera.galera_transaction_read_only f... Closed
is blocked by MDEV-14143 galera.galera_kill_smallchanges, gale... Closed
is blocked by MDEV-14145 galera.MW-328B failed in buildbot wit... Closed
is blocked by MDEV-14149 galera.MW-328A, galera.MW-328C failed... Closed
is blocked by MDEV-14381 galera.galera_as_master_large, galera... Closed
is blocked by MDEV-14496 galera.galera_var_node_address failed... Closed
is blocked by MDEV-15522 Change galera suite MTR tests to use ... Closed
is blocked by MDEV-15540 Galera suite MTR tests issuing wsrep_... Closed
is blocked by MDEV-15792 Fix mtr to be able to wait for >1 exi... Closed
is blocked by MDEV-15794 Test failure on galera.galera_var_ret... Closed
is blocked by MDEV-15803 Test failure on galera.galera_var_aut... Closed
is blocked by MDEV-15804 Test failure on galera.pxc-421 Closed
is blocked by MDEV-15806 Test failure on galera.galera_paralle... Closed
is blocked by MDEV-15807 Test failure on galera.galera_lock_table Closed
is blocked by MDEV-15808 Test failure on galera.galera_gra_log Closed
is blocked by MDEV-15809 Test failure on galera.MW-44 Closed
is blocked by MDEV-15810 Test failure on galera.lp1376747 and ... Closed
is blocked by MDEV-15811 Test failure on galera.galera_pc_igno... Closed
is blocked by MDEV-15823 Test failure on galera.galera_var_sla... Closed
is blocked by MDEV-15845 Test failure on galera.galera_concurr... Closed
is blocked by MDEV-16008 galera mtr test galera_many_tables_no... Closed
is blocked by MDEV-16890 Galera test failure on galera_sst_mys... Closed
is blocked by MDEV-17058 Test failure on wsrep.variables Closed
is blocked by MDEV-17060 Test failure on galera.galera_var_sla... Closed
is blocked by MDEV-17061 Test failure on galera.galera_gcs_fc_... Closed
is blocked by MDEV-17108 Test failure on galera.galera_kill_ddl Closed
is blocked by MDEV-17143 Galera test failure on galera.MW-44 Closed
is blocked by MDEV-17151 Galera test failure on galera.galera_... Closed
is blocked by MDEV-17206 Test failure on galera.galera_ist_* i... Closed
is blocked by MDEV-17208 Test failure on galera.MW-286 Closed
is blocked by MDEV-17210 InnoDB: Assertion failure in file loc... Closed
is blocked by MDEV-17403 Test failure on galera.galera_enum Closed
Relates
relates to MDEV-13743 galera_toi_truncate may fail with: qu... Closed
relates to MDEV-13876 galera.MW-328A failed in buildbot wit... Closed
relates to MDEV-14069 galera_sst_mysqldump.test fails with:... Closed
relates to MDEV-14324 Add wsrep_cluster_name=%worker_id%tes... Closed
relates to MDEV-13738 galera_applier_ftwrl_table_alter may ... Closed

 Description   

Collection of known Galera 3 test failures on MariaDB 10.1-10.3.



 Comments   
Comment by Jan Lindström (Inactive) [ 2017-08-16 ]

galera.MW-336 'innodb_plugin'            w2 [ fail ]
        Test ended at 2017-08-16 08:54:27
 
CURRENT_TEST: galera.MW-336
--- /usr/share/mysql/mysql-test/suite/galera/r/MW-336.result	2017-08-16 07:21:05.000000000 +0000
+++ /run/shm/var/2/log/MW-336.reject	2017-08-16 08:54:27.424495545 +0000
@@ -5,11 +5,11 @@
 SET GLOBAL wsrep_slave_threads = 10;
 SELECT COUNT(*) = 11 FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER = 'system user';
 COUNT(*) = 11
-1
+0
 SET GLOBAL wsrep_slave_threads = 20;
 SELECT COUNT(*) = 21 FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER = 'system user';
 COUNT(*) = 21
-1
+0
 SET GLOBAL wsrep_slave_threads = 1;
 INSERT INTO t1 VALUES (1);
 INSERT INTO t1 VALUES (2);
@@ -37,6 +37,6 @@
 INSERT INTO t1 VALUES (20);
 SELECT COUNT(*) = 2 FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER = 'system user';
 COUNT(*) = 2
-1
+0
 SET GLOBAL wsrep_slave_threads = 1;
 DROP TABLE t1;
 
mysqltest: Result content mismatch
 
 - skipping '/run/shm/var/2/log/galera.MW-336-innodb_plugin/'
 
Retrying test galera.MW-336, attempt(2/3)...
 
worker[2] > Restart  - not started
worker[2] > Restart  - not started
***Warnings generated in error logs during shutdown after running tests: galera.MW-258 galera.MW-252 galera.MW-292 galera.GAL-382 galera.MW-328A galera.MW-328D galera.GAL-401 galera.MW-328C galera.MW-285 galera.MW-309 galera.MW-328E galera.MW-336 galera.MW-328B galera.MW-259
 
170816  8:54:25 [Warning] WSREP: Refusing exit for the last slave thread.

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

EDIT: test should be run with wsrep_replicate_myisam

diff --git a/mysql-test/suite/galera/include/have_wsrep_replicate_myisam.inc b/mysql-test/suite/galera/include/have_wsrep_replicate_myisam.inc
new file mode 100644
index 00000000000..726fc6e2b18
--- /dev/null
+++ b/mysql-test/suite/galera/include/have_wsrep_replicate_myisam.inc
@@ -0,0 +1,4 @@
+--require suite/galera/r/have_wsrep_replicate_myisam.require
+disable_query_log;
+SHOW VARIABLES LIKE 'wsrep_replicate_myisam';
+enable_query_log;
diff --git a/mysql-test/suite/galera/r/have_wsrep_replicate_myisam.require b/mysql-test/suite/galera/r/have_wsrep_replicate_myisam.require
new file mode 100644
index 00000000000..c55610fd049
--- /dev/null
+++ b/mysql-test/suite/galera/r/have_wsrep_replicate_myisam.require
@@ -0,0 +1,2 @@
+Variable_name  Value
+wsrep_replicate_myisam ON
diff --git a/mysql-test/suite/galera/t/galera_flush_local.opt b/mysql-test/suite/galera/t/galera_flush_local.opt
index 5a1fb6748d9..65b8006909a 100644
--- a/mysql-test/suite/galera/t/galera_flush_local.opt
+++ b/mysql-test/suite/galera/t/galera_flush_local.opt
@@ -1 +1 @@
---query_cache_type=1 --query_cache_size=1000000
+--query_cache_type=1 --query_cache_size=1000000 --wsrep_replicate_myisam
diff --git a/mysql-test/suite/galera/t/galera_flush_local.test b/mysql-test/suite/galera/t/galera_flush_local.test
index 768f4ea4f1b..24acd9ec4ff 100644
--- a/mysql-test/suite/galera/t/galera_flush_local.test
+++ b/mysql-test/suite/galera/t/galera_flush_local.test
@@ -5,6 +5,7 @@
 --source include/galera_cluster.inc
 --source include/have_innodb.inc
 --source include/have_query_cache.inc
+--source include/have_wsrep_replicate_myisam.inc
 
 --disable_warnings
 DROP TABLE IF EXISTS t1, t2, x1, x2;

CURRENT_TEST: galera.galera_flush_local
--- /usr/share/mysql/mysql-test/suite/galera/r/galera_flush_local.result	2017-08-16 10:40:23.000000000 +0000
+++ /run/shm/var/2/log/galera_flush_local.reject	2017-08-16 14:57:27.731510623 +0000
@@ -55,19 +55,19 @@
 test.x1	repair	status	OK
 test.x2	repair	status	OK
 wsrep_last_committed_diff
-1
+0
 SELECT COUNT(*) = 10 FROM t1;
 COUNT(*) = 10
 1
 SELECT COUNT(*) = 10 FROM x1;
 COUNT(*) = 10
-1
+0
 SELECT COUNT(*) = 10000 FROM t2;
 COUNT(*) = 10000
 1
 SELECT COUNT(*) = 10 FROM x2;
 COUNT(*) = 10
-1
+0
 DROP TABLE t1, t2, x1, x2;
 CREATE TABLE t1 (f1 INTEGER);
 CREATE TABLE t2 (f1 INT PRIMARY KEY AUTO_INCREMENT, f2 INTEGER);
@@ -128,18 +128,18 @@
 wsrep_last_committed_diff
 1
 wsrep_last_committed_diff2
-1
+0
 SELECT COUNT(*) = 10 FROM t1;
 COUNT(*) = 10
 1
 SELECT COUNT(*) = 10 FROM x1;
 COUNT(*) = 10
-1
+0
 SELECT COUNT(*) = 10000 FROM t2;
 COUNT(*) = 10000
 1
 SELECT COUNT(*) = 10 FROM x2;
 COUNT(*) = 10
-1
+0
 set wsrep_on=1;
 DROP TABLE t1, t2, x1, x2;
 
mysqltest: Result content mismatch
 
 - skipping '/run/shm/var/2/log/galera.galera_flush_local-innodb_plugin/'
 
Retrying test galera.galera_flush_local, attempt(2/3)...
 
worker[2] > Restart  - not started
worker[2] > Restart  - not started

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

Edit: MariaDB doesn't have thread_statistics; suggested patch:

diff --git a/mysql-test/include/have_thread_statistics.inc b/mysql-test/include/have_thread_statistics.inc
new file mode 100644
index 00000000000..ade156ce56e
--- /dev/null
+++ b/mysql-test/include/have_thread_statistics.inc
@@ -0,0 +1,4 @@
+-- require r/have_thread_statistics.require
+disable_query_log;
+show variables like 'thread_statistics';
+enable_query_log;
diff --git a/mysql-test/include/have_userstat.inc b/mysql-test/include/have_userstat.inc
new file mode 100644
index 00000000000..2fa5947806b
--- /dev/null
+++ b/mysql-test/include/have_userstat.inc
@@ -0,0 +1,4 @@
+-- require r/have_userstat.require
+disable_query_log;
+show variables like 'userstat';
+enable_query_log;
diff --git a/mysql-test/r/have_thread_statistics.require b/mysql-test/r/have_thread_statistics.require
new file mode 100644
index 00000000000..2d4fb9870bb
--- /dev/null
+++ b/mysql-test/r/have_thread_statistics.require
@@ -0,0 +1,2 @@
+Variable_name  Value
+thread_statistics      ON
diff --git a/mysql-test/r/have_userstat.require b/mysql-test/r/have_userstat.require
new file mode 100644
index 00000000000..e37ec675ebd
--- /dev/null
+++ b/mysql-test/r/have_userstat.require
@@ -0,0 +1,2 @@
+Variable_name  Value
+userstat       ON
diff --git a/mysql-test/suite/galera/disabled.def b/mysql-test/suite/galera/disabled.def
index f9909914089..93f6d66c04d 100644
--- a/mysql-test/suite/galera/disabled.def
+++ b/mysql-test/suite/galera/disabled.def
@@ -12,7 +12,7 @@
 MW-336 : MDEV-13549 Galera test failures
 galera_gra_log : MDEV-13549 Galera test failures
 galera_flush_local : MDEV-13549 Galera test failures
-galera_flush :  MDEV-13549 Galera test failures
+galera_flush :  Test must be rewritten without usage of thread_statistics
 MW-329 : MDEV-13549 Galera test failures
 galera_account_management : MariaDB 10.0 does not support ALTER USER
 galera_binlog_row_image : MariaDB 10.0 does not support binlog_row_image
diff --git a/mysql-test/suite/galera/t/galera_flush.opt b/mysql-test/suite/galera/t/galera_flush.opt
new file mode 100644
index 00000000000..7bcda1b7498
--- /dev/null
+++ b/mysql-test/suite/galera/t/galera_flush.opt
@@ -0,0 +1 @@
+--userstat=1 
diff --git a/mysql-test/suite/galera/t/galera_flush.test b/mysql-test/suite/galera/t/galera_flush.test
index 2a0fdc694fc..5b871790a2f 100644
--- a/mysql-test/suite/galera/t/galera_flush.test
+++ b/mysql-test/suite/galera/t/galera_flush.test
@@ -6,6 +6,8 @@
 --source include/have_innodb.inc
 --source include/have_perfschema.inc
 --source include/have_query_cache.inc
+--source include/have_userstat.inc
+--source include/have_thread_statistics.inc
 
 --disable_warnings
 DROP TABLE IF EXISTS t1, t2; 

galera.galera_flush 'innodb_plugin'      w4 [ fail ]
        Test ended at 2017-08-16 14:57:31
 
CURRENT_TEST: galera.galera_flush
mysqltest: At line 177: query 'SET @old_thread_statistics= @@global.thread_statistics' failed: 1193: Unknown system variable 'thread_statistics'
 
The result from queries just before the failure was:
< snip >
wsrep_last_committed_diff
1
FLUSH GENERAL LOGS;
wsrep_last_committed_diff
1
FLUSH ENGINE LOGS;
wsrep_last_committed_diff
1
FLUSH RELAY LOGS;
wsrep_last_committed_diff
1
SET @userstat_old= @@userstat;
SET GLOBAL userstat=ON;
FLUSH CLIENT_STATISTICS;
FLUSH INDEX_STATISTICS;
FLUSH TABLE_STATISTICS;
FLUSH USER_STATISTICS;
wsrep_last_committed_diff
1
SET @old_thread_statistics= @@global.thread_statistics;
 
More results from queries before failure can be found in /run/shm/var/4/log/galera_flush.log
 
 - skipping '/run/shm/var/4/log/galera.galera_flush-innodb_plugin/'
 
Retrying test galera.galera_flush, attempt(2/3)...

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

CURRENT_TEST: galera.MW-329
--- /usr/share/mysql/mysql-test/suite/galera/r/MW-329.result	2017-08-16 10:40:23.000000000 +0000
+++ /run/shm/var/2/log/MW-329.reject	2017-08-16 14:57:35.659472451 +0000
@@ -15,7 +15,7 @@
 CALL proc_insert();;
 SELECT VARIABLE_VALUE > 0 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_local_replays';
 VARIABLE_VALUE > 0
-1
+0
 DROP PROCEDURE proc_insert;
 DROP TABLE t1;
 CALL mtr.add_suppression("conflict state 3 after post commit");
 
mysqltest: Result content mismatch
 
 - skipping '/run/shm/var/2/log/galera.MW-329-innodb_plugin/'
 
Retrying test galera.MW-329, attempt(2/3)...
 
worker[2] > Restart  - not started
worker[2] > Restart  - not started
galera.MW-329 'innodb_plugin'            w2 [ retry-pass ]     31

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

EDIT: provided log is expected and it looks that actual problem is that nothing happens after second node creates file tmp/mysqld.2.expect
Following patch makes test pass (as well as galera_gcache_recover which had identical problem)

diff --git a/mysql-test/mysql-test-run.pl b/mysql-test/mysql-test-run.pl
index ed7dd020e5b..b0d3b24193e 100755
--- a/mysql-test/mysql-test-run.pl
+++ b/mysql-test/mysql-test-run.pl
@@ -4385,6 +4385,7 @@ sub run_testcase ($$) {
   my $test= $tinfo->{suite}->start_test($tinfo);
   # Set only when we have to keep waiting after expectedly died server
   my $keep_waiting_proc = 0;
+  my $keep_waiting_proc_old = 0;
   my $print_timeout= start_timer($print_freq * 60);
 
   while (1)
@@ -4396,6 +4397,7 @@ sub run_testcase ($$) {
       $proc = My::SafeProcess->check_any();
       if ($proc)
       {
+        $keep_waiting_proc_old = $keep_waiting_proc;
        mtr_verbose ("Found exited process $proc");
       }
       else
@@ -4553,7 +4555,8 @@ sub run_testcase ($$) {
     if ($check_crash)
     {
       # Keep waiting if it returned 2, if 1 don't wait or stop waiting.
-      $keep_waiting_proc = 0 if $check_crash == 1;
+      $keep_waiting_proc = $keep_waiting_proc_old if $check_crash == 1;
+      $keep_waiting_proc_old = 0 if $check_crash == 1;
       $keep_waiting_proc = $proc if $check_crash == 2;
       next;
     }

Original failure below:

galera.GAL-419 'xtradb'                  w1 [ fail ]
        Test ended at 2017-08-17 08:30:00
 
CURRENT_TEST: galera.GAL-419
170817  8:21:32 [Warning] WSREP: option --wsrep-causal-reads is deprecated
170817  8:21:32 [Note] /home/jan/mysql/10.0-galera/sql/mysqld (mysqld 10.0.32-MariaDB-wsrep-debug) starting as process 4163 ...
170817  8:21:32 [Note] WSREP: Read nil XID from storage engines, skipping position init
170817  8:21:32 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/libgalera_smm.so'
170817  8:21:32 [Note] WSREP: wsrep_load(): Galera 3.21(r8678538) by Codership Oy <info@codership.com> loaded successfully.
170817  8:21:32 [Note] WSREP: CRC-32C: using hardware acceleration.
170817  8:21:32 [Note] WSREP: Found saved state: e9b3818a-830b-11e7-8e87-6fcbe87151dc:-1, safe_to_bootsrap: 0
170817  8:21:32 [Note] WSREP: Passing config to GCS: base_dir = /dev/shm/1/mysqld.1/data/; base_host = 127.0.0.1; base_port = 16002; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT10S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /dev/shm/1/mysqld.1/data/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /dev/shm/1/mysqld.1/data//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 10M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3
170817  8:21:32 [Note] WSREP: GCache history reset: e9b3818a-830b-11e7-8e87-6fcbe87151dc:0 -> e9b3818a-830b-11e7-8e87-6fcbe87151dc:-1
170817  8:21:32 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
170817  8:21:32 [Note] WSREP: wsrep_sst_grab()
170817  8:21:32 [Note] WSREP: Start replication
170817  8:21:32 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
170817  8:21:32 [ERROR] WSREP: It may not be safe to bootstrap the cluster from this node. It was not the last one to leave the cluster and may not contain all the updates. To force cluster bootstrap with this node, edit the grastate.dat file manually and set safe_to_bootstrap to 1 .
170817  8:21:32 [ERROR] WSREP: wsrep::connect(gcomm://) failed: 7
170817  8:21:32 [ERROR] Aborting
 
170817  8:21:32 [Note] WSREP: Service disconnected.
170817  8:21:33 [Note] WSREP: Some threads may fail to exit.
170817  8:21:33 [Note] /home/jan/mysql/10.0-galera/sql/mysqld: Shutdown complete
 
mysqltest: In included file "./include/wait_until_connected_again.inc": 
included from /home/jan/mysql/10.0-galera/mysql-test/suite/galera/include/start_mysqld.inc at line 20:
included from /home/jan/mysql/10.0-galera/mysql-test/suite/galera/t/GAL-419.test at line 36:
At line 26: Server failed to restart
 
The result from queries just before the failure was:
SET SESSION wsrep_sync_wait = 0;
Killing server ...
SET SESSION wsrep_sync_wait = 0;
Killing server ...
 
 - saving '/dev/shm/1/log/galera.GAL-419-xtradb/' to '/dev/shm/log/galera.GAL-419-xtradb/'

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

galera.galera_toi_ddl_fk_insert 'innodb_plugin' w2 [ fail ]
        Test ended at 2017-08-17 08:45:35
 
CURRENT_TEST: galera.galera_toi_ddl_fk_insert
mysqltest: At line 54: query 'reap' failed: 1452: Cannot add or update a child row: a foreign key constraint fails (`test`.`child`, CONSTRAINT `child_ibfk_1` FOREIGN KEY (`parent_id`) REFERENCES `parent` (`id`))
 
The result from queries just before the failure was:
< snip >
INSERT INTO ten VALUES (1),(2),(3),(4),(5),(6),(7),(8),(9),(10);
CREATE TABLE parent (
id INT PRIMARY KEY AUTO_INCREMENT,
f2 INTEGER,
KEY (id)
) ENGINE=InnoDB;
CREATE TABLE child (
id INT PRIMARY KEY AUTO_INCREMENT,
parent_id INT
) ENGINE=InnoDB;
INSERT INTO parent VALUES (DEFAULT, 0);
INSERT INTO child (parent_id) SELECT 1 FROM ten AS a1, ten AS a2, ten AS a3, ten AS a4;;
INSERT INTO parent (f2) SELECT 1 FROM ten AS a1, ten AS a2, ten AS a3, ten AS a4;;
INSERT INTO parent (f2) SELECT 2 FROM ten AS a1, ten AS a2, ten AS a3, ten AS a4;;
ALTER TABLE child ADD FOREIGN KEY (parent_id) REFERENCES parent(id);;
 
More results from queries before failure can be found in /dev/shm/2/log/galera_toi_ddl_fk_insert.log

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

EDIT: it looks we just shouldn't drop object:

diff --git a/mysql-test/suite/galera/r/galera_var_notify_cmd.result b/mysql-test/suite/galera/r/galera_var_notify_cmd.result
index e9e4605e1bc..b7ae4af336d 100644
--- a/mysql-test/suite/galera/r/galera_var_notify_cmd.result
+++ b/mysql-test/suite/galera/r/galera_var_notify_cmd.result
@@ -7,4 +7,3 @@ MAX(size) = 2
 SELECT COUNT(DISTINCT idx) = 2 FROM mtr_wsrep_notify.status;
 COUNT(DISTINCT idx) = 2
 1
-DROP SCHEMA mtr_wsrep_notify;
diff --git a/mysql-test/suite/galera/t/galera_var_notify_cmd.test b/mysql-test/suite/galera/t/galera_var_notify_cmd.test
index 4fea69f62bb..55b162f7786 100644
--- a/mysql-test/suite/galera/t/galera_var_notify_cmd.test
+++ b/mysql-test/suite/galera/t/galera_var_notify_cmd.test
@@ -11,4 +11,4 @@ SELECT COUNT(DISTINCT uuid) = 2 FROM mtr_wsrep_notify.membership;
 SELECT MAX(size) = 2 FROM mtr_wsrep_notify.status;
 SELECT COUNT(DISTINCT idx) = 2 FROM mtr_wsrep_notify.status;
 
-DROP SCHEMA mtr_wsrep_notify;
+# DROP SCHEMA mtr_wsrep_notify;

galera.galera_var_notify_cmd 'innodb_plugin' [ pass ]     14
 
MTR's internal check of the test case 'galera.galera_var_notify_cmd' failed.
This means that the test case does not preserve the state that existed
before the test case was executed.  Most likely the test case did not
do a proper clean-up. It could also be caused by the previous test run
by this thread, if the server wasn't restarted.
This is the diff of the states of the servers before and after the
test case was executed:
mysqltest: Logging to '/dev/shm/tmp/check-mysqld_1.log'.
mysqltest: Results saved in '/dev/shm/tmp/check-mysqld_1.result'.
mysqltest: Connecting to server localhost:16000 (socket /dev/shm/tmp/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...
mysqltest: ... Connected.
mysqltest: Start processing test commands from './include/check-testcase.test' ...
mysqltest: ... Done processing test commands.
--- /dev/shm/tmp/check-mysqld_1.result	2017-08-17 08:52:10.277797568 +0300
+++ /dev/shm/tmp/check-mysqld_1.reject	2017-08-17 08:52:10.421797957 +0300
@@ -577,7 +577,6 @@
 CATALOG_NAME	SCHEMA_NAME	DEFAULT_CHARACTER_SET_NAME	DEFAULT_COLLATION_NAME	SQL_PATH
 def	information_schema	utf8	utf8_general_ci	NULL
 def	mtr	latin1	latin1_swedish_ci	NULL
-def	mtr_wsrep_notify	latin1	latin1_swedish_ci	NULL
 def	mysql	latin1	latin1_swedish_ci	NULL
 def	performance_schema	utf8	utf8_general_ci	NULL
 def	test	latin1	latin1_swedish_ci	NULL
 
mysqltest: Result length mismatch
 
not ok
 
galera.galera_var_notify_cmd 'xtradb'    [ fail ]
        Test ended at 2017-08-17 08:52:26
 
CURRENT_TEST: galera.galera_var_notify_cmd
--- /home/jan/mysql/10.0-galera/mysql-test/suite/galera/r/galera_var_notify_cmd.result	2017-08-08 12:16:55.894763479 +0300
+++ /home/jan/mysql/10.0-galera/mysql-test/suite/galera/r/galera_var_notify_cmd.reject	2017-08-17 08:52:25.969839817 +0300
@@ -6,5 +6,5 @@
 1
 SELECT COUNT(DISTINCT idx) = 2 FROM mtr_wsrep_notify.status;
 COUNT(DISTINCT idx) = 2
-1
+0
 DROP SCHEMA mtr_wsrep_notify;
 
mysqltest: Result content mismatch
 
 - saving '/dev/shm/log/galera.galera_var_notify_cmd-xtradb/' to '/dev/shm/log/galera.galera_var_notify_cmd-xtradb/'

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

galera.galera_var_slave_threads 'innodb_plugin' w2 [ fail ]  Found warnings/errors in server log file!
        Test ended at 2017-08-17 09:08:36
line
170817  9:08:36 [Warning] WSREP: Refusing exit for the last slave thread.
^ Found warnings in /dev/shm/2/log/mysqld.2.err
ok

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

galera.mysql-wsrep#90 'innodb_plugin'    w2 [ fail ]
        Test ended at 2017-08-17 09:10:24
 
CURRENT_TEST: galera.mysql-wsrep#90
--- /home/jan/mysql/10.0-galera/mysql-test/suite/galera/r/mysql-wsrep#90.result2017-08-08 12:16:55.894763479 +0300
+++ /home/jan/mysql/10.0-galera/mysql-test/suite/galera/r/mysql-wsrep#90.reject2017-08-17 09:10:24.388625983 +0300
@@ -17,6 +17,13 @@
 SET DEBUG_SYNC = 'alter_table_before_open_tables WAIT_FOR continue';
 ALTER TABLE t1 ADD COLUMN f2 INTEGER;;
 SET SESSION wsrep_sync_wait = 0;
+Timeout in wait_condition.inc for SELECT COUNT(*) = 1 FROM INFORMATION_SCHEMA.PROCESSLIST WHERE STATE = 'debug sync point: alter_table_before_open_tables'
+Id	User	Host	db	Command	Time	State	Info	Progress
+1	system user		NULL	Sleep	106	wsrep aborter idle	NULL	0.000
+2	system user		NULL	Sleep	70	committed 102	NULL	0.000
+106	root	localhost	test	Sleep	31		NULL	0.000
+107	root	localhost:48848	test	Sleep	31		NULL	0.000
+108	root	localhost:48852	test	Query	0	init	show full processlist	0.000
 SET GLOBAL wsrep_OSU_method = "RSU";
 SET DEBUG_SYNC= 'now SIGNAL continue';
 SELECT COUNT(*) = 2 FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = 't1';
 
mysqltest: Result length mismatch

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

galera.galera_as_slave_replication_bundle 'xtradb' w4 [ fail ]
        Test ended at 2017-08-17 09:42:18
 
CURRENT_TEST: galera.galera_as_slave_preordered
CURRENT_TEST: galera.galera_as_slave_replication_bundle
mysqltest: In included file "./include/wait_condition.inc": 
included from /home/jan/mysql/10.0-galera/mysql-test/suite/galera/t/galera_as_slave_replication_bundle.test at line 30:
At line 54: query 'let $success= `$wait_condition`' failed with wrong errno 2013: 'Lost connection to MySQL server during query', instead of 0...
 
The result from queries just before the failure was:
START SLAVE;
CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB;
INSERT INTO t1 VALUES(1);
INSERT INTO t1 VALUES(2);
INSERT INTO t1 VALUES(3);
INSERT INTO t1 VALUES(4);
INSERT INTO t1 VALUES(5);
CURRENT_TEST: galera.galera_as_slave_replication_bundle
mysqltest: In included file "./include/wait_condition.inc": 
included from /home/jan/mysql/10.0-galera/mysql-test/suite/galera/t/galera_as_slave_replication_bundle.test at line 33:
At line 54: query 'let $success= `$wait_condition`' failed with wrong errno 2013: 'Lost connection to MySQL server during query', instead of 0...
 
The result from queries just before the failure was:
START SLAVE;
CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB;
INSERT INTO t1 VALUES(1);
INSERT INTO t1 VALUES(2);
INSERT INTO t1 VALUES(3);
INSERT INTO t1 VALUES(4);
INSERT INTO t1 VALUES(5);
 
 
Server [mysqld.2 - pid: 15866, winpid: 15866, exit: 256] failed during test run
Server log from this test:
----------SERVER LOG START-----------
170817  9:41:58 [Warning] WSREP: option --wsrep-causal-reads is deprecated
170817  9:41:58 [Note] /home/jan/mysql/10.0-galera/sql/mysqld (mysqld 10.0.32-MariaDB-wsrep-debug) starting as process 15868 ...
170817  9:41:58 [Note] WSREP: Read nil XID from storage engines, skipping position init
170817  9:41:58 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/libgalera_smm.so'
170817  9:41:58 [Note] WSREP: wsrep_load(): Galera 3.21(r8678538) by Codership Oy <info@codership.com> loaded successfully.
170817  9:41:58 [Note] WSREP: CRC-32C: using hardware acceleration.
170817  9:41:58 [Warning] WSREP: Could not open state file for reading: '/dev/shm/4/mysqld.2/data//grastate.dat'
170817  9:41:58 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1, safe_to_bootsrap: 1
170817  9:41:58 [Note] WSREP: Passing config to GCS: base_dir = /dev/shm/4/mysqld.2/data/; base_host = 127.0.0.1; base_port = 16043; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /dev/shm/4/mysqld.2/data/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /dev/shm/4/mysqld.2/data//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 10M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S
170817  9:41:58 [Note] WSREP: GCache history reset: 00000000-0000-0000-0000-000000000000:0 -> 00000000-0000-0000-0000-000000000000:-1
170817  9:41:58 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
170817  9:41:58 [Note] WSREP: wsrep_sst_grab()
170817  9:41:58 [Note] WSREP: Start replication
170817  9:41:58 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
170817  9:41:58 [Note] WSREP: protonet asio version 0
170817  9:41:58 [Note] WSREP: Using CRC-32C for message checksums.
170817  9:41:58 [Note] WSREP: backend: asio
170817  9:41:58 [Note] WSREP: gcomm thread scheduling priority set to other:0 
170817  9:41:58 [Warning] WSREP: access file(/dev/shm/4/mysqld.2/data//gvwstate.dat) failed(No such file or directory)
170817  9:41:58 [Note] WSREP: restore pc from disk failed
170817  9:41:58 [Note] WSREP: GMCast version 0
170817  9:41:58 [Note] WSREP: (2ac3acf5, 'tcp://0.0.0.0:16043') listening at tcp://0.0.0.0:16043
170817  9:41:58 [Note] WSREP: (2ac3acf5, 'tcp://0.0.0.0:16043') multicast: , ttl: 1
170817  9:41:58 [Note] WSREP: EVS version 0
170817  9:41:58 [Note] WSREP: gcomm: connecting to group 'my_wsrep_cluster', peer ''
170817  9:41:58 [Note] WSREP: start_prim is enabled, turn off pc_recovery
170817  9:41:58 [Note] WSREP: Node 2ac3acf5 state prim
170817  9:41:58 [Note] WSREP: view(view_id(PRIM,2ac3acf5,1) memb {
	2ac3acf5,0
} joined {
} left {
} partitioned {
})
170817  9:41:58 [Note] WSREP: save pc into disk
170817  9:41:58 [Note] WSREP: gcomm: connected
170817  9:41:58 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
170817  9:41:58 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
170817  9:41:58 [Note] WSREP: Opened channel 'my_wsrep_cluster'
170817  9:41:58 [Note] WSREP: Waiting for SST to complete.
170817  9:41:58 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1
170817  9:41:58 [Note] WSREP: Starting new group from scratch: 2ac44089-8317-11e7-9645-82bf5c67e922
170817  9:41:58 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 2ac443d4-8317-11e7-8bf7-8f54815e8be4
170817  9:41:58 [Note] WSREP: STATE EXCHANGE: sent state msg: 2ac443d4-8317-11e7-8bf7-8f54815e8be4
170817  9:41:58 [Note] WSREP: STATE EXCHANGE: got state msg: 2ac443d4-8317-11e7-8bf7-8f54815e8be4 from 0 (jan-laptop-asus)
170817  9:41:58 [Note] WSREP: Quorum results:
	version    = 4,
	component  = PRIMARY,
	conf_id    = 0,
	members    = 1/1 (joined/total),
	act_id     = 0,
	last_appl. = -1,
	protocols  = 0/7/3 (gcs/repl/appl),
	group UUID = 2ac44089-8317-11e7-9645-82bf5c67e922
170817  9:41:58 [Note] WSREP: Flow-control interval: [16, 16]
170817  9:41:58 [Note] WSREP: Trying to continue unpaused monitor
170817  9:41:58 [Note] WSREP: Restored state OPEN -> JOINED (0)
170817  9:41:58 [Note] WSREP: Member 0.0 (jan-laptop-asus) synced with group.
170817  9:41:58 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 0)
170817  9:41:58 [Note] WSREP: New cluster view: global state: 2ac44089-8317-11e7-9645-82bf5c67e922:0, view# 1: Primary, number of nodes: 1, my index: 0, protocol version 3
170817  9:41:58 [Note] WSREP: SST complete, seqno: 0
170817  9:41:58 [Note] InnoDB: innodb_empty_free_list_algorithm has been changed to legacy because of small buffer pool size. In order to use backoff, increase buffer pool at least up to 20MB.
 
170817  9:41:58 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2017-08-17 09:41:58 7f06205a5f40 InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!!
2017-08-17 09:41:58 7f06205a5f40 InnoDB: !!!!!!!! UNIV_SYNC_DEBUG switched on !!!!!!!!!
170817  9:41:58 [Note] InnoDB: The InnoDB memory heap is disabled
170817  9:41:58 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
170817  9:41:58 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
170817  9:41:58 [Note] InnoDB: Compressed tables use zlib 1.2.8
170817  9:41:58 [Note] InnoDB: Using Linux native AIO
170817  9:41:58 [Note] InnoDB: Using CPU crc32 instructions
170817  9:41:58 [Note] InnoDB: Initializing buffer pool, size = 8.0M
170817  9:41:58 [Note] InnoDB: Completed initialization of buffer pool
170817  9:41:58 [Note] InnoDB: Highest supported file format is Barracuda.
170817  9:41:58 [Note] WSREP: (2ac3acf5, 'tcp://0.0.0.0:16043') connection established to 2ad6015a tcp://127.0.0.1:16046
170817  9:41:58 [Note] WSREP: (2ac3acf5, 'tcp://0.0.0.0:16043') turning message relay requesting on, nonlive peers: 
170817  9:41:58 [Note] WSREP: declaring 2ad6015a at tcp://127.0.0.1:16046 stable
170817  9:41:58 [Note] WSREP: Node 2ac3acf5 state prim
170817  9:41:58 [Note] WSREP: view(view_id(PRIM,2ac3acf5,2) memb {
	2ac3acf5,0
	2ad6015a,0
} joined {
} left {
} partitioned {
})
170817  9:41:58 [Note] WSREP: save pc into disk
170817  9:41:58 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2
170817  9:41:58 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 2b22e74e-8317-11e7-be5b-eb3ea40963e0
170817  9:41:58 [Note] WSREP: STATE EXCHANGE: sent state msg: 2b22e74e-8317-11e7-be5b-eb3ea40963e0
170817  9:41:58 [Note] WSREP: STATE EXCHANGE: got state msg: 2b22e74e-8317-11e7-be5b-eb3ea40963e0 from 0 (jan-laptop-asus)
170817  9:41:59 [Note] WSREP: STATE EXCHANGE: got state msg: 2b22e74e-8317-11e7-be5b-eb3ea40963e0 from 1 (jan-laptop-asus)
170817  9:41:59 [Note] WSREP: Quorum results:
	version    = 4,
	component  = PRIMARY,
	conf_id    = 1,
	members    = 1/2 (joined/total),
	act_id     = 0,
	last_appl. = 0,
	protocols  = 0/7/3 (gcs/repl/appl),
	group UUID = 2ac44089-8317-11e7-9645-82bf5c67e922
170817  9:41:59 [Note] WSREP: Flow-control interval: [23, 23]
170817  9:41:59 [Note] WSREP: Trying to continue unpaused monitor
170817  9:41:59 [Note] InnoDB: 128 rollback segment(s) are active.
170817  9:41:59 [Note] InnoDB: Waiting for purge to start
170817  9:41:59 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.36-82.1 started; log sequence number 1629444
170817  9:41:59 [Note] Plugin 'XTRADB_READ_VIEW' is disabled.
170817  9:41:59 [Note] Plugin 'XTRADB_INTERNAL_HASH_TABLES' is disabled.
170817  9:41:59 [Note] Plugin 'XTRADB_RSEG' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_LOCK_WAITS' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_CMP_RESET' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_CMPMEM_RESET' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_CMP_PER_INDEX_RESET' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_FT_DEFAULT_STOPWORD' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_FT_DELETED' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_FT_BEING_DELETED' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_FT_CONFIG' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_FT_INDEX_CACHE' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_FT_INDEX_TABLE' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_SYS_TABLESTATS' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_SYS_INDEXES' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_SYS_COLUMNS' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_SYS_FIELDS' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_SYS_TABLESPACES' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_SYS_DATAFILES' is disabled.
170817  9:41:59 [Note] Plugin 'INNODB_CHANGED_PAGES' is disabled.
170817  9:41:59 [Note] Plugin 'partition' is disabled.
170817  9:41:59 [Warning] /home/jan/mysql/10.0-galera/sql/mysqld: unknown variable 'loose-feedback-user-info=mysql-test'
170817  9:41:59 [Warning] /home/jan/mysql/10.0-galera/sql/mysqld: unknown variable 'loose-feedback-debug-startup-interval=20'
170817  9:41:59 [Warning] /home/jan/mysql/10.0-galera/sql/mysqld: unknown variable 'loose-feedback-debug-first-interval=60'
170817  9:41:59 [Warning] /home/jan/mysql/10.0-galera/sql/mysqld: unknown variable 'loose-feedback-debug-interval=60'
170817  9:41:59 [Note] Server socket created on IP: '127.0.0.1'.
170817  9:42:00 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
170817  9:42:00 [Note] WSREP: REPL Protocols: 7 (3, 2)
170817  9:42:00 [Note] WSREP: Assign initial position for certification: 0, protocol version: 3
170817  9:42:00 [Note] WSREP: Service thread queue flushed.
170817  9:42:00 [Note] WSREP: GCache history reset: 00000000-0000-0000-0000-000000000000:0 -> 2ac44089-8317-11e7-9645-82bf5c67e922:0
170817  9:42:00 [Note] WSREP: Synchronized with group, ready for connections
170817  9:42:00 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
170817  9:42:00 [Note] WSREP: New cluster view: global state: 2ac44089-8317-11e7-9645-82bf5c67e922:0, view# 2: Primary, number of nodes: 2, my index: 0, protocol version 3
170817  9:42:00 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
170817  9:42:00 [Note] WSREP: REPL Protocols: 7 (3, 2)
170817  9:42:00 [Note] WSREP: Assign initial position for certification: 0, protocol version: 3
170817  9:42:00 [Note] WSREP: Service thread queue flushed.
170817  9:42:00 [Note] /home/jan/mysql/10.0-galera/sql/mysqld: ready for connections.
Version: '10.0.32-MariaDB-wsrep-debug'  socket: '/dev/shm/tmp/4/mysqld.2.sock'  port: 16041  Source distribution, wsrep_25.20.rc3fc46e
170817  9:42:00 [Note] WSREP: Member 1.0 (jan-laptop-asus) requested state transfer from '*any*'. Selected 0.0 (jan-laptop-asus)(SYNCED) as donor.
170817  9:42:00 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 0)
170817  9:42:00 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
170817  9:42:00 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'donor' --address '127.0.0.1:16048/rsync_sst' --socket '/dev/shm/tmp/4/mysqld.2.sock' --datadir '/dev/shm/4/mysqld.2/data/' --defaults-file '/dev/shm/4/my.cnf' --defaults-group-suffix '.2'  --binlog 'mysqld-bin' --gtid '2ac44089-8317-11e7-9645-82bf5c67e922:0''
170817  9:42:00 [Note] WSREP: sst_donor_thread signaled with 0
170817  9:42:00 [Note] WSREP: Flushing tables for SST...
170817  9:42:00 [Note] WSREP: Provider paused at 2ac44089-8317-11e7-9645-82bf5c67e922:0 (5)
170817  9:42:00 [Note] WSREP: Tables flushed.
WSREP_SST: [INFO] Preparing binlog files for transfer: (20170817 09:42:00.873)
mysqld-bin.000002
170817  9:42:01 [Note] WSREP: (2ac3acf5, 'tcp://0.0.0.0:16043') turning message relay requesting off
170817  9:42:01 [Note] WSREP: resuming provider at 5
170817  9:42:01 [Note] WSREP: Provider resumed.
170817  9:42:02 [Note] WSREP: 0.0 (jan-laptop-asus): State transfer to 1.0 (jan-laptop-asus) complete.
170817  9:42:02 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 0)
170817  9:42:02 [Note] WSREP: Member 0.0 (jan-laptop-asus) synced with group.
170817  9:42:02 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 0)
170817  9:42:02 [Note] WSREP: Synchronized with group, ready for connections
170817  9:42:02 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
170817  9:42:04 [Note] WSREP: 1.0 (jan-laptop-asus): State transfer from 0.0 (jan-laptop-asus) complete.
170817  9:42:04 [Note] WSREP: Member 1.0 (jan-laptop-asus) synced with group.
170817  9:42:05 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='', master_port='3306', master_log_file='', master_log_pos='4'. New state master_host='127.0.0.1', master_port='16040', master_log_file='', master_log_pos='4'.
170817  9:42:05 [Note] WSREP: ready state reached
170817  9:42:05 [Note] Slave SQL thread initialized, starting replication in log 'FIRST' at position 0, relay log './mysqld-relay-bin.000001' position: 4
170817  9:42:05 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16040',replication started in log 'FIRST' at position 4
mysqld: /home/jan/mysql/10.0-galera/sql/log_event.cc:6639: virtual int Gtid_log_event::do_apply_event(rpl_group_info*): Assertion `(thd->variables.option_bits & (1ULL << 25)) == 0' failed.
170817  9:42:05 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 10.0.32-MariaDB-wsrep-debug
key_buffer_size=1048576
read_buffer_size=131072
max_used_connections=2
max_threads=153
thread_count=4
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 62944 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x7f0607475070
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f061e142de8 thread_stack 0x48000
/home/jan/mysql/10.0-galera/sql/mysqld(my_print_stacktrace+0x35)[0x5570c7d4f11e]
/home/jan/mysql/10.0-galera/sql/mysqld(handle_fatal_signal+0x352)[0x5570c774f225]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x110c0)[0x7f06201a90c0]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcf)[0x7f061e1bffcf]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7f061e1c13fa]
/lib/x86_64-linux-gnu/libc.so.6(+0x2be37)[0x7f061e1b8e37]
/lib/x86_64-linux-gnu/libc.so.6(+0x2bee2)[0x7f061e1b8ee2]
/home/jan/mysql/10.0-galera/sql/mysqld(_ZN14Gtid_log_event14do_apply_eventEP14rpl_group_info+0xfc)[0x5570c7841f10]
/home/jan/mysql/10.0-galera/sql/mysqld(_ZN9Log_event11apply_eventEP14rpl_group_info+0x2e)[0x5570c74b366a]
/home/jan/mysql/10.0-galera/sql/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14rpl_group_infoP19rpl_parallel_thread+0x397)[0x5570c74a962f]
/home/jan/mysql/10.0-galera/sql/mysqld(+0x3f100b)[0x5570c74aa00b]
/home/jan/mysql/10.0-galera/sql/mysqld(handle_slave_sql+0xe00)[0x5570c74ad0f7]
/home/jan/mysql/10.0-galera/sql/mysqld(+0x99d0f0)[0x5570c7a560f0]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7494)[0x7f062019f494]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f061e275aff]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x0): 
Connection ID (thread ID): 9
Status: NOT_KILLED
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on
 
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Writing a core file
----------SERVER LOG END-------------
 
 
 - saving '/dev/shm/4/log/galera.galera_as_slave_replication_bundle-xtradb/' to '/dev/shm/log/galera.galera_as_slave_replication_bundle-xtradb/'
 - found 'core' (1/5)
 
Trying 'dbx' to get a backtrace
 
Trying 'gdb' to get a backtrace from coredump /dev/shm/log/galera.galera_as_slave_replication_bundle-xtradb/mysqld.2/data/core
Core generated by '/home/jan/mysql/10.0-galera/sql/mysqld'
Output from gdb follows. The first stack trace is from the failing thread.
The following stack traces are from all threads (so the failing one is
duplicated).
--------------------------
[New LWP 16317]
[New LWP 15888]
[New LWP 16054]
[New LWP 15892]
[New LWP 15891]
[New LWP 15893]
[New LWP 16033]
[New LWP 15900]
[New LWP 15894]
[New LWP 15895]
[New LWP 15901]
[New LWP 15907]
[New LWP 15902]
[New LWP 16011]
[New LWP 15903]
[New LWP 16012]
[New LWP 15904]
[New LWP 15905]
[New LWP 16013]
[New LWP 16014]
[New LWP 16313]
[New LWP 16015]
[New LWP 16016]
[New LWP 16316]
[New LWP 16034]
[New LWP 16035]
[New LWP 16052]
[New LWP 16314]
[New LWP 15868]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/home/jan/mysql/10.0-galera/sql/mysqld --defaults-group-suffix=.2 --defaults-fi'.
Program terminated with signal SIGABRT, Aborted.
#0  __pthread_kill (threadid=<optimized out>, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:57
57	../sysdeps/unix/sysv/linux/pthread_kill.c: No such file or directory.
[Current thread is 1 (Thread 0x7f061e143700 (LWP 16317))]
#0  __pthread_kill (threadid=<optimized out>, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:57
#1  0x00005570c7d4f1f5 in my_write_core (sig=6) at /home/jan/mysql/10.0-galera/mysys/stacktrace.c:478
#2  0x00005570c774f5a6 in handle_fatal_signal (sig=6) at /home/jan/mysql/10.0-galera/sql/signal_handler.cc:285
#3  <signal handler called>
#4  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#5  0x00007f061e1c13fa in __GI_abort () at abort.c:89
#6  0x00007f061e1b8e37 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x5570c7e94748 "(thd->variables.option_bits & (1ULL << 25)) == 0", file=file@entry=0x5570c7e91c78 "/home/jan/mysql/10.0-galera/sql/log_event.cc", line=line@entry=6639, function=function@entry=0x5570c7e96f00 <Gtid_log_event::do_apply_event(rpl_group_info*)::__PRETTY_FUNCTION__> "virtual int Gtid_log_event::do_apply_event(rpl_group_info*)") at assert.c:92
#7  0x00007f061e1b8ee2 in __GI___assert_fail (assertion=0x5570c7e94748 "(thd->variables.option_bits & (1ULL << 25)) == 0", file=0x5570c7e91c78 "/home/jan/mysql/10.0-galera/sql/log_event.cc", line=6639, function=0x5570c7e96f00 <Gtid_log_event::do_apply_event(rpl_group_info*)::__PRETTY_FUNCTION__> "virtual int Gtid_log_event::do_apply_event(rpl_group_info*)") at assert.c:101
#8  0x00005570c7841f10 in Gtid_log_event::do_apply_event (this=0x7f0607490670, rgi=0x7f060742b800) at /home/jan/mysql/10.0-galera/sql/log_event.cc:6639
#9  0x00005570c74b366a in Log_event::apply_event (this=0x7f0607490670, rgi=0x7f060742b800) at /home/jan/mysql/10.0-galera/sql/log_event.h:1360
#10 0x00005570c74a962f in apply_event_and_update_pos (ev=0x7f0607490670, thd=0x7f0607475070, rgi=0x7f060742b800, rpt=0x0) at /home/jan/mysql/10.0-galera/sql/slave.cc:3317
#11 0x00005570c74aa00b in exec_relay_log_event (thd=0x7f0607475070, rli=0x7f0611bd1cf8, serial_rgi=0x7f060742b800) at /home/jan/mysql/10.0-galera/sql/slave.cc:3659
#12 0x00005570c74ad0f7 in handle_slave_sql (arg=0x7f0611bd0000) at /home/jan/mysql/10.0-galera/sql/slave.cc:4797
#13 0x00005570c7a560f0 in pfs_spawn_thread (arg=0x7f0610897370) at /home/jan/mysql/10.0-galera/storage/perfschema/pfs.cc:1860
#14 0x00007f062019f494 in start_thread (arg=0x7f061e143700) at pthread_create.c:333
#15 0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 29 (Thread 0x7f06205a5f40 (LWP 15868)):
#0  0x00007f061e26c6ad in poll () at ../sysdeps/unix/syscall-template.S:84
#1  0x00005570c7486806 in handle_connections_sockets () at /home/jan/mysql/10.0-galera/sql/mysqld.cc:6886
#2  0x00005570c748582a in mysqld_main (argc=144, argv=0x7f061d88aac0) at /home/jan/mysql/10.0-galera/sql/mysqld.cc:6312
#3  0x00005570c7479b30 in main (argc=16, argv=0x7ffff3c10ab8) at /home/jan/mysql/10.0-galera/sql/main.cc:25
 
Thread 28 (Thread 0x7f0620435700 (LWP 16314)):
#0  0x00007f061e26c6ad in poll () at ../sysdeps/unix/syscall-template.S:84
#1  0x00005570c7d9c6e5 in vio_io_wait (vio=0x7f060c2e6bf0, event=VIO_IO_EVENT_READ, timeout=28800000) at /home/jan/mysql/10.0-galera/vio/viosocket.c:992
#2  0x00005570c7d9b038 in vio_socket_io_wait (vio=0x7f060c2e6bf0, event=VIO_IO_EVENT_READ) at /home/jan/mysql/10.0-galera/vio/viosocket.c:108
#3  0x00005570c7d9b1eb in vio_read (vio=0x7f060c2e6bf0, buf=0x7f0611bf7070 "\001", size=4) at /home/jan/mysql/10.0-galera/vio/viosocket.c:184
#4  0x00005570c748e658 in my_real_read (net=0x7f060c3f7328, complen=0x7f0620434930, header=1 '\001') at /home/jan/mysql/10.0-galera/sql/net_serv.cc:874
#5  0x00005570c748eea8 in my_net_read_packet (net=0x7f060c3f7328, read_from_server=1 '\001') at /home/jan/mysql/10.0-galera/sql/net_serv.cc:1137
#6  0x00005570c753df49 in do_command (thd=0x7f060c3f7070) at /home/jan/mysql/10.0-galera/sql/sql_parse.cc:953
#7  0x00005570c76649d7 in do_handle_one_connection (thd_arg=0x7f060c3f7070) at /home/jan/mysql/10.0-galera/sql/sql_connect.cc:1394
#8  0x00005570c766471d in handle_one_connection (arg=0x7f060c3f7070) at /home/jan/mysql/10.0-galera/sql/sql_connect.cc:1306
#9  0x00005570c7a560f0 in pfs_spawn_thread (arg=0x7f060c2e6e70) at /home/jan/mysql/10.0-galera/storage/perfschema/pfs.cc:1860
#10 0x00007f062019f494 in start_thread (arg=0x7f0620435700) at pthread_create.c:333
#11 0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 27 (Thread 0x7f0620510700 (LWP 16052)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00005570c7d548cf in safe_cond_wait (cond=0x5570c86e5e28 <mysql_bin_log+4072>, mp=0x5570c86e5d78 <mysql_bin_log+3896>, file=0x5570c7e8b260 "/home/jan/mysql/10.0-galera/include/mysql/psi/mysql_thread.h", line=1154) at /home/jan/mysql/10.0-galera/mysys/thr_mutex.c:493
#2  0x00005570c78182f3 in inline_mysql_cond_wait (that=0x5570c86e5e28 <mysql_bin_log+4072>, mutex=0x5570c86e5d78 <mysql_bin_log+3896>, src_file=0x5570c7e8b6d8 "/home/jan/mysql/10.0-galera/sql/log.cc", src_line=9468) at /home/jan/mysql/10.0-galera/include/mysql/psi/mysql_thread.h:1154
#3  0x00005570c782fa5f in binlog_background_thread (arg=0x0) at /home/jan/mysql/10.0-galera/sql/log.cc:9468
#4  0x00005570c7a560f0 in pfs_spawn_thread (arg=0x7f0611bbb670) at /home/jan/mysql/10.0-galera/storage/perfschema/pfs.cc:1860
#5  0x00007f062019f494 in start_thread (arg=0x7f0620510700) at pthread_create.c:333
#6  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 26 (Thread 0x7f0605ffe700 (LWP 16035)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00005570c7b031e7 in os_cond_wait (cond=0x7f06193ab3c8, fast_mutex=0x7f06193ab390) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0sync.cc:196
#2  0x00005570c7b0355d in os_event_wait_low (event=0x7f06193ab390, reset_sig_count=1) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0sync.cc:477
#3  0x00005570c7c564d5 in buf_dump_thread (arg=0x0) at /home/jan/mysql/10.0-galera/storage/xtradb/buf/buf0dump.cc:707
#4  0x00007f062019f494 in start_thread (arg=0x7f0605ffe700) at pthread_create.c:333
#5  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 25 (Thread 0x7f06067ff700 (LWP 16034)):
#0  0x00007f061e26e3f3 in select () at ../sysdeps/unix/syscall-template.S:84
#1  0x00005570c7b03dd6 in os_thread_sleep (tm=1000000) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0thread.cc:311
#2  0x00005570c7c5e53a in page_cleaner_sleep_if_needed (next_loop_time=1502952125821) at /home/jan/mysql/10.0-galera/storage/xtradb/buf/buf0flu.cc:2587
#3  0x00005570c7c5ee57 in buf_flush_lru_manager_thread (arg=0x0) at /home/jan/mysql/10.0-galera/storage/xtradb/buf/buf0flu.cc:2889
#4  0x00007f062019f494 in start_thread (arg=0x7f06067ff700) at pthread_create.c:333
#5  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 24 (Thread 0x7f061e18c700 (LWP 16316)):
#0  0x00007f061e26c6ad in poll () at ../sysdeps/unix/syscall-template.S:84
#1  0x00005570c7d9c6e5 in vio_io_wait (vio=0x7f06078800f0, event=VIO_IO_EVENT_READ, timeout=120000) at /home/jan/mysql/10.0-galera/vio/viosocket.c:992
#2  0x00005570c7d9b038 in vio_socket_io_wait (vio=0x7f06078800f0, event=VIO_IO_EVENT_READ) at /home/jan/mysql/10.0-galera/vio/viosocket.c:108
#3  0x00005570c7d9b1eb in vio_read (vio=0x7f06078800f0, buf=0x7f0607888070 "'", size=16384) at /home/jan/mysql/10.0-galera/vio/viosocket.c:184
#4  0x00005570c7d9b40b in vio_read_buff (vio=0x7f06078800f0, buf=0x7f060788d070 "", size=4) at /home/jan/mysql/10.0-galera/vio/viosocket.c:232
#5  0x00005570c748e658 in my_real_read (net=0x7f0607865470, complen=0x7f061e18bc40, header=0 '\000') at /home/jan/mysql/10.0-galera/sql/net_serv.cc:874
#6  0x00005570c748eea8 in my_net_read_packet (net=0x7f0607865470, read_from_server=0 '\000') at /home/jan/mysql/10.0-galera/sql/net_serv.cc:1137
#7  0x00005570c7717196 in cli_safe_read (mysql=0x7f0607865470) at /home/jan/mysql/10.0-galera/sql-common/client.c:581
#8  0x00005570c74a9032 in read_event (mysql=0x7f0607865470, mi=0x7f0611bd0000, suppress_warnings=0x7f061e18be0b) at /home/jan/mysql/10.0-galera/sql/slave.cc:3125
#9  0x00005570c74ab43a in handle_slave_io (arg=0x7f0611bd0000) at /home/jan/mysql/10.0-galera/sql/slave.cc:4129
#10 0x00005570c7a560f0 in pfs_spawn_thread (arg=0x7f06108970f0) at /home/jan/mysql/10.0-galera/storage/perfschema/pfs.cc:1860
#11 0x00007f062019f494 in start_thread (arg=0x7f061e18c700) at pthread_create.c:333
#12 0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 23 (Thread 0x7f06087f8700 (LWP 16016)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00005570c7b031e7 in os_cond_wait (cond=0x7f06193ab278, fast_mutex=0x7f06193ab240) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0sync.cc:196
#2  0x00005570c7b0355d in os_event_wait_low (event=0x7f06193ab240, reset_sig_count=1) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0sync.cc:477
#3  0x00005570c7bbc49a in srv_resume_thread (slot=0x5570c8f01e58 <srv_sys+536>, sig_count=1, wait=true, timeout_usec=0) at /home/jan/mysql/10.0-galera/storage/xtradb/srv/srv0srv.cc:1033
#4  0x00005570c7bc1237 in srv_purge_coordinator_suspend (slot=0x5570c8f01e58 <srv_sys+536>, rseg_history_len=6) at /home/jan/mysql/10.0-galera/storage/xtradb/srv/srv0srv.cc:3405
#5  0x00005570c7bc15e9 in srv_purge_coordinator_thread (arg=0x0) at /home/jan/mysql/10.0-galera/storage/xtradb/srv/srv0srv.cc:3496
#6  0x00007f062019f494 in start_thread (arg=0x7f06087f8700) at pthread_create.c:333
#7  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 22 (Thread 0x7f0608ff9700 (LWP 16015)):
#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1  0x00005570c7b030fb in os_cond_wait_timed (cond=0x7f06193aca88, fast_mutex=0x7f06193aca50, abstime=0x7f0608ff8de0) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0sync.cc:159
#2  0x00005570c7b0371f in os_event_wait_time_low (event=0x7f06193aca50, time_in_usec=5000000, reset_sig_count=1) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0sync.cc:591
#3  0x00005570c7c042be in ib_wqueue_timedwait (wq=0x7f0611bc54f8, wait_in_usecs=5000000) at /home/jan/mysql/10.0-galera/storage/xtradb/ut/ut0wqueue.cc:154
#4  0x00005570c7ce97fa in fts_optimize_thread (arg=0x7f0611bc54f8) at /home/jan/mysql/10.0-galera/storage/xtradb/fts/fts0opt.cc:3060
#5  0x00007f062019f494 in start_thread (arg=0x7f0608ff9700) at pthread_create.c:333
#6  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 21 (Thread 0x7f062047e700 (LWP 16313)):
#0  0x00007f061e26c6ad in poll () at ../sysdeps/unix/syscall-template.S:84
#1  0x00005570c7d9c6e5 in vio_io_wait (vio=0x7f060c2e66f0, event=VIO_IO_EVENT_READ, timeout=28800000) at /home/jan/mysql/10.0-galera/vio/viosocket.c:992
#2  0x00005570c7d9b038 in vio_socket_io_wait (vio=0x7f060c2e66f0, event=VIO_IO_EVENT_READ) at /home/jan/mysql/10.0-galera/vio/viosocket.c:108
#3  0x00005570c7d9b1eb in vio_read (vio=0x7f060c2e66f0, buf=0x7f0611bcb070 "\001", size=4) at /home/jan/mysql/10.0-galera/vio/viosocket.c:184
#4  0x00005570c748e658 in my_real_read (net=0x7f060c3f1328, complen=0x7f062047d930, header=1 '\001') at /home/jan/mysql/10.0-galera/sql/net_serv.cc:874
#5  0x00005570c748eea8 in my_net_read_packet (net=0x7f060c3f1328, read_from_server=1 '\001') at /home/jan/mysql/10.0-galera/sql/net_serv.cc:1137
#6  0x00005570c753df49 in do_command (thd=0x7f060c3f1070) at /home/jan/mysql/10.0-galera/sql/sql_parse.cc:953
#7  0x00005570c76649d7 in do_handle_one_connection (thd_arg=0x7f060c3f1070) at /home/jan/mysql/10.0-galera/sql/sql_connect.cc:1394
#8  0x00005570c766471d in handle_one_connection (arg=0x7f060c3f1070) at /home/jan/mysql/10.0-galera/sql/sql_connect.cc:1306
#9  0x00005570c7a560f0 in pfs_spawn_thread (arg=0x7f060c2e6970) at /home/jan/mysql/10.0-galera/storage/perfschema/pfs.cc:1860
#10 0x00007f062019f494 in start_thread (arg=0x7f062047e700) at pthread_create.c:333
#11 0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 20 (Thread 0x7f06097fa700 (LWP 16014)):
#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1  0x00005570c7b030fb in os_cond_wait_timed (cond=0x7f06193ac858, fast_mutex=0x7f06193ac820, abstime=0x7f06097f9e90) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0sync.cc:159
#2  0x00005570c7b0371f in os_event_wait_time_low (event=0x7f06193ac820, time_in_usec=10000000, reset_sig_count=1) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0sync.cc:591
#3  0x00005570c7ca9da9 in dict_stats_thread (arg=0x0) at /home/jan/mysql/10.0-galera/storage/xtradb/dict/dict0stats_bg.cc:359
#4  0x00007f062019f494 in start_thread (arg=0x7f06097fa700) at pthread_create.c:333
#5  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 19 (Thread 0x7f0609ffb700 (LWP 16013)):
#0  0x00007f061e26e3f3 in select () at ../sysdeps/unix/syscall-template.S:84
#1  0x00005570c7b03dd6 in os_thread_sleep (tm=1000000) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0thread.cc:311
#2  0x00005570c7bc0702 in srv_master_sleep () at /home/jan/mysql/10.0-galera/storage/xtradb/srv/srv0srv.cc:3039
#3  0x00005570c7bc0846 in srv_master_thread (arg=0x0) at /home/jan/mysql/10.0-galera/storage/xtradb/srv/srv0srv.cc:3089
#4  0x00007f062019f494 in start_thread (arg=0x7f0609ffb700) at pthread_create.c:333
#5  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 18 (Thread 0x7f060cffa700 (LWP 15905)):
#0  0x00007f061ff966ca in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
#1  0x00005570c7b00ba5 in os_aio_linux_collect (array=0x7f0619be56a0, segment=1, seg_size=256) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0file.cc:5138
#2  0x00005570c7b00f14 in os_aio_linux_handle (global_seg=5, message1=0x7f060cff9e70, message2=0x7f060cff9e68, type=0x7f060cff9e60, space_id=0x7f060cff9e58) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0file.cc:5284
#3  0x00005570c7cbb864 in fil_aio_wait (segment=5) at /home/jan/mysql/10.0-galera/storage/xtradb/fil/fil0fil.cc:5778
#4  0x00005570c7bc37bc in io_handler_thread (arg=0x5570c8f06548 <n+40>) at /home/jan/mysql/10.0-galera/storage/xtradb/srv/srv0start.cc:542
#5  0x00007f062019f494 in start_thread (arg=0x7f060cffa700) at pthread_create.c:333
#6  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 17 (Thread 0x7f060d7fb700 (LWP 15904)):
#0  0x00007f061ff966ca in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
#1  0x00005570c7b00ba5 in os_aio_linux_collect (array=0x7f0619be56a0, segment=0, seg_size=256) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0file.cc:5138
#2  0x00005570c7b00f14 in os_aio_linux_handle (global_seg=4, message1=0x7f060d7fae70, message2=0x7f060d7fae68, type=0x7f060d7fae60, space_id=0x7f060d7fae58) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0file.cc:5284
#3  0x00005570c7cbb864 in fil_aio_wait (segment=4) at /home/jan/mysql/10.0-galera/storage/xtradb/fil/fil0fil.cc:5778
#4  0x00005570c7bc37bc in io_handler_thread (arg=0x5570c8f06540 <n+32>) at /home/jan/mysql/10.0-galera/storage/xtradb/srv/srv0start.cc:542
#5  0x00007f062019f494 in start_thread (arg=0x7f060d7fb700) at pthread_create.c:333
#6  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 16 (Thread 0x7f060a7fc700 (LWP 16012)):
#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1  0x00005570c7b030fb in os_cond_wait_timed (cond=0x7f06193ab2e8, fast_mutex=0x7f06193ab2b0, abstime=0x7f060a7fbe30) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0sync.cc:159
#2  0x00005570c7b0371f in os_event_wait_time_low (event=0x7f06193ab2b0, time_in_usec=1000000, reset_sig_count=1) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0sync.cc:591
#3  0x00005570c7bbf2bc in srv_error_monitor_thread (arg=0x0) at /home/jan/mysql/10.0-galera/storage/xtradb/srv/srv0srv.cc:2319
#4  0x00007f062019f494 in start_thread (arg=0x7f060a7fc700) at pthread_create.c:333
#5  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 15 (Thread 0x7f060dffc700 (LWP 15903)):
#0  0x00007f061ff966ca in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
#1  0x00005570c7b00ba5 in os_aio_linux_collect (array=0x7f0619be5560, segment=1, seg_size=256) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0file.cc:5138
#2  0x00005570c7b00f14 in os_aio_linux_handle (global_seg=3, message1=0x7f060dffbe70, message2=0x7f060dffbe68, type=0x7f060dffbe60, space_id=0x7f060dffbe58) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0file.cc:5284
#3  0x00005570c7cbb864 in fil_aio_wait (segment=3) at /home/jan/mysql/10.0-galera/storage/xtradb/fil/fil0fil.cc:5778
#4  0x00005570c7bc37bc in io_handler_thread (arg=0x5570c8f06538 <n+24>) at /home/jan/mysql/10.0-galera/storage/xtradb/srv/srv0start.cc:542
#5  0x00007f062019f494 in start_thread (arg=0x7f060dffc700) at pthread_create.c:333
#6  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 14 (Thread 0x7f060affd700 (LWP 16011)):
#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1  0x00005570c7b030fb in os_cond_wait_timed (cond=0x7f06193ac5b8, fast_mutex=0x7f06193ac580, abstime=0x7f060affce70) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0sync.cc:159
#2  0x00005570c7b0371f in os_event_wait_time_low (event=0x7f06193ac580, time_in_usec=1000000, reset_sig_count=1) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0sync.cc:591
#3  0x00005570c7ad928d in lock_wait_timeout_thread (arg=0x0) at /home/jan/mysql/10.0-galera/storage/xtradb/lock/lock0wait.cc:540
#4  0x00007f062019f494 in start_thread (arg=0x7f060affd700) at pthread_create.c:333
#5  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 13 (Thread 0x7f060e7fd700 (LWP 15902)):
#0  0x00007f061ff966ca in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
#1  0x00005570c7b00ba5 in os_aio_linux_collect (array=0x7f0619be5560, segment=0, seg_size=256) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0file.cc:5138
#2  0x00005570c7b00f14 in os_aio_linux_handle (global_seg=2, message1=0x7f060e7fce70, message2=0x7f060e7fce68, type=0x7f060e7fce60, space_id=0x7f060e7fce58) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0file.cc:5284
#3  0x00005570c7cbb864 in fil_aio_wait (segment=2) at /home/jan/mysql/10.0-galera/storage/xtradb/fil/fil0fil.cc:5778
#4  0x00005570c7bc37bc in io_handler_thread (arg=0x5570c8f06530 <n+16>) at /home/jan/mysql/10.0-galera/storage/xtradb/srv/srv0start.cc:542
#5  0x00007f062019f494 in start_thread (arg=0x7f060e7fd700) at pthread_create.c:333
#6  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 12 (Thread 0x7f060bfff700 (LWP 15907)):
#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1  0x00005570c7b030fb in os_cond_wait_timed (cond=0x7f06193ab358, fast_mutex=0x7f06193ab320, abstime=0x7f060bffee40) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0sync.cc:159
#2  0x00005570c7b0371f in os_event_wait_time_low (event=0x7f06193ab320, time_in_usec=5000000, reset_sig_count=1) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0sync.cc:591
#3  0x00005570c7bbec67 in srv_monitor_thread (arg=0x0) at /home/jan/mysql/10.0-galera/storage/xtradb/srv/srv0srv.cc:2073
#4  0x00007f062019f494 in start_thread (arg=0x7f060bfff700) at pthread_create.c:333
#5  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 11 (Thread 0x7f060effe700 (LWP 15901)):
#0  0x00007f061ff966ca in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
#1  0x00005570c7b00ba5 in os_aio_linux_collect (array=0x7f0619be5600, segment=0, seg_size=256) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0file.cc:5138
#2  0x00005570c7b00f14 in os_aio_linux_handle (global_seg=1, message1=0x7f060effde70, message2=0x7f060effde68, type=0x7f060effde60, space_id=0x7f060effde58) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0file.cc:5284
#3  0x00005570c7cbb864 in fil_aio_wait (segment=1) at /home/jan/mysql/10.0-galera/storage/xtradb/fil/fil0fil.cc:5778
#4  0x00005570c7bc37bc in io_handler_thread (arg=0x5570c8f06528 <n+8>) at /home/jan/mysql/10.0-galera/storage/xtradb/srv/srv0start.cc:542
#5  0x00007f062019f494 in start_thread (arg=0x7f060effe700) at pthread_create.c:333
#6  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 10 (Thread 0x7f06117ff700 (LWP 15895)):
#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1  0x00005570c7d54baa in safe_cond_timedwait (cond=0x5570c8ee8de0 <COND_checkpoint>, mp=0x5570c8ee8d20 <LOCK_checkpoint>, abstime=0x7f06117fede0, file=0x5570c7ebfff0 "/home/jan/mysql/10.0-galera/include/mysql/psi/mysql_thread.h", line=1191) at /home/jan/mysql/10.0-galera/mysys/thr_mutex.c:547
#2  0x00005570c79432b9 in inline_mysql_cond_timedwait (that=0x5570c8ee8de0 <COND_checkpoint>, mutex=0x5570c8ee8d20 <LOCK_checkpoint>, abstime=0x7f06117fede0, src_file=0x5570c7ec0030 "/home/jan/mysql/10.0-galera/storage/maria/ma_servicethread.c", src_line=116) at /home/jan/mysql/10.0-galera/include/mysql/psi/mysql_thread.h:1191
#3  0x00005570c794379b in my_service_thread_sleep (control=0x5570c8502200 <checkpoint_control>, sleep_time=29000000000) at /home/jan/mysql/10.0-galera/storage/maria/ma_servicethread.c:115
#4  0x00005570c7938b7e in ma_checkpoint_background (arg=0x1e) at /home/jan/mysql/10.0-galera/storage/maria/ma_checkpoint.c:708
#5  0x00005570c7a560f0 in pfs_spawn_thread (arg=0x7f061d838570) at /home/jan/mysql/10.0-galera/storage/perfschema/pfs.cc:1860
#6  0x00007f062019f494 in start_thread (arg=0x7f06117ff700) at pthread_create.c:333
#7  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 9 (Thread 0x7f0620559700 (LWP 15894)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f0618bb00fe in fifo_lock_get (q=0x7f0619397000) at galerautils/src/gu_fifo.c:261
#2  gu_fifo_get_head (q=0x7f0619397000, err=err@entry=0x7f0620558184) at galerautils/src/gu_fifo.c:329
#3  0x00007f0618cfe5c5 in gcs_recv (conn=0x7f061d869400, action=action@entry=0x7f06205581d0) at gcs/src/gcs.cpp:1864
#4  0x00007f0618d4f44a in galera::Gcs::recv (act=..., this=<optimized out>) at galera/src/galera_gcs.hpp:118
#5  galera::GcsActionSource::process (this=0x7f0619bed730, recv_ctx=0x7f0612c37070, exit_loop=@0x7f062055825f: false) at galera/src/gcs_action_source.cpp:175
#6  0x00007f0618d7659b in galera::ReplicatorSMM::async_recv (this=0x7f0619bed000, recv_ctx=0x7f0612c37070) at galera/src/replicator_smm.cpp:360
#7  0x00007f0618d900b8 in galera_recv (gh=<optimized out>, recv_ctx=<optimized out>) at galera/src/wsrep_provider.cpp:244
#8  0x00005570c76df36e in wsrep_replication_process (thd=0x7f0612c37070) at /home/jan/mysql/10.0-galera/sql/wsrep_thd.cc:359
#9  0x00005570c7483a41 in start_wsrep_THD (arg=0x5570c76df2bc <wsrep_replication_process(THD*)>) at /home/jan/mysql/10.0-galera/sql/mysqld.cc:5417
#10 0x00007f062019f494 in start_thread (arg=0x7f0620559700) at pthread_create.c:333
#11 0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 8 (Thread 0x7f060f7ff700 (LWP 15900)):
#0  0x00007f061ff966ca in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
#1  0x00005570c7b00ba5 in os_aio_linux_collect (array=0x7f0619be5650, segment=0, seg_size=256) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0file.cc:5138
#2  0x00005570c7b00f14 in os_aio_linux_handle (global_seg=0, message1=0x7f060f7fee70, message2=0x7f060f7fee68, type=0x7f060f7fee60, space_id=0x7f060f7fee58) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0file.cc:5284
#3  0x00005570c7cbb864 in fil_aio_wait (segment=0) at /home/jan/mysql/10.0-galera/storage/xtradb/fil/fil0fil.cc:5778
#4  0x00005570c7bc37bc in io_handler_thread (arg=0x5570c8f06520 <n>) at /home/jan/mysql/10.0-galera/storage/xtradb/srv/srv0start.cc:542
#5  0x00007f062019f494 in start_thread (arg=0x7f060f7ff700) at pthread_create.c:333
#6  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 7 (Thread 0x7f060b7fe700 (LWP 16033)):
#0  0x00007f061e26e3f3 in select () at ../sysdeps/unix/syscall-template.S:84
#1  0x00005570c7b03dd6 in os_thread_sleep (tm=1000000) at /home/jan/mysql/10.0-galera/storage/xtradb/os/os0thread.cc:311
#2  0x00005570c7c5e53a in page_cleaner_sleep_if_needed (next_loop_time=1502952125848) at /home/jan/mysql/10.0-galera/storage/xtradb/buf/buf0flu.cc:2587
#3  0x00005570c7c5e88b in buf_flush_page_cleaner_thread (arg=0x0) at /home/jan/mysql/10.0-galera/storage/xtradb/buf/buf0flu.cc:2728
#4  0x00007f062019f494 in start_thread (arg=0x7f060b7fe700) at pthread_create.c:333
#5  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 6 (Thread 0x7f06205a2700 (LWP 15893)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00005570c7d548cf in safe_cond_wait (cond=0x5570c86bc840 <COND_wsrep_rollback>, mp=0x5570c86bc780 <LOCK_wsrep_rollback>, file=0x5570c7e0e840 "/home/jan/mysql/10.0-galera/include/mysql/psi/mysql_thread.h", line=1154) at /home/jan/mysql/10.0-galera/mysys/thr_mutex.c:493
#2  0x00005570c76ddd40 in inline_mysql_cond_wait (that=0x5570c86bc840 <COND_wsrep_rollback>, mutex=0x5570c86bc780 <LOCK_wsrep_rollback>, src_file=0x5570c7e0ea18 "/home/jan/mysql/10.0-galera/sql/wsrep_thd.cc", src_line=453) at /home/jan/mysql/10.0-galera/include/mysql/psi/mysql_thread.h:1154
#3  0x00005570c76df958 in wsrep_rollback_process (thd=0x7f0612816070) at /home/jan/mysql/10.0-galera/sql/wsrep_thd.cc:453
#4  0x00005570c7483a41 in start_wsrep_THD (arg=0x5570c76df85d <wsrep_rollback_process(THD*)>) at /home/jan/mysql/10.0-galera/sql/mysqld.cc:5417
#5  0x00007f062019f494 in start_thread (arg=0x7f06205a2700) at pthread_create.c:333
#6  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 5 (Thread 0x7f06147ff700 (LWP 15891)):
#0  0x00007f061e2760f3 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84
#1  0x00007f0618cd5dfc in asio::detail::epoll_reactor::run (this=0x7f06197d8f00, block=block@entry=true, ops=...) at /usr/include/asio/detail/impl/epoll_reactor.ipp:391
#2  0x00007f0618cf0dc9 in asio::detail::task_io_service::do_run_one (ec=..., this_thread=..., lock=..., this=0x7f061d8af8e0) at /usr/include/asio/detail/impl/task_io_service.ipp:355
#3  asio::detail::task_io_service::run (ec=..., this=0x7f061d8af8e0) at /usr/include/asio/detail/impl/task_io_service.ipp:148
#4  asio::io_service::run (this=<optimized out>) at /usr/include/asio/impl/io_service.ipp:58
#5  gcomm::AsioProtonet::event_loop (this=<optimized out>, period=...) at gcomm/src/asio_protonet.cpp:121
#6  0x00007f0618d07631 in GCommConn::run (this=0x7f061d82c900) at gcs/src/gcs_gcomm.cpp:498
#7  0x00007f0618d0f299 in GCommConn::run_fn (arg=<optimized out>) at gcs/src/gcs_gcomm.cpp:209
#8  0x00007f062019f494 in start_thread (arg=0x7f06147ff700) at pthread_create.c:333
#9  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 4 (Thread 0x7f0613ffe700 (LWP 15892)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f0618d05b1f in gu::Lock::wait (this=0x7f0613ffd080, cond=...) at galerautils/src/gu_lock.hpp:59
#2  RecvBuf::front (timeout=..., this=0x7f061d82cb40) at gcs/src/gcs_gcomm.cpp:124
#3  gcomm_recv (backend=<optimized out>, msg=0x7f061d869660, timeout=9223372035999999999) at gcs/src/gcs_gcomm.cpp:673
#4  0x00007f0618cfb00a in core_msg_recv (timeout=9223372035999999999, recv_msg=0x7f061d869660, backend=0x7f061d869778) at gcs/src/gcs_core.cpp:468
#5  gcs_core_recv (conn=0x7f061d869600, recv_act=recv_act@entry=0x7f0613ffde70, timeout=9223372035999999999) at gcs/src/gcs_core.cpp:1073
#6  0x00007f0618d00e46 in gcs_recv_thread (arg=0x7f061d869400) at gcs/src/gcs.cpp:1268
#7  0x00007f062019f494 in start_thread (arg=0x7f0613ffe700) at pthread_create.c:333
#8  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 3 (Thread 0x7f06204c7700 (LWP 16054)):
#0  do_sigwait (set=<optimized out>, set@entry=0x7f06204c6df0, sig=sig@entry=0x7f06204c6dec) at ../sysdeps/unix/sysv/linux/sigwait.c:64
#1  0x00007f06201a8ced in __sigwait (set=0x7f06204c6df0, sig=0x7f06204c6dec) at ../sysdeps/unix/sysv/linux/sigwait.c:96
#2  0x00005570c747fc44 in signal_hand (arg=0x0) at /home/jan/mysql/10.0-galera/sql/mysqld.cc:3435
#3  0x00005570c7a560f0 in pfs_spawn_thread (arg=0x7f0611bbbb70) at /home/jan/mysql/10.0-galera/storage/perfschema/pfs.cc:1860
#4  0x00007f062019f494 in start_thread (arg=0x7f06204c7700) at pthread_create.c:333
#5  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 2 (Thread 0x7f06177ff700 (LWP 15888)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007f0618d3e1a5 in gu::Lock::wait (this=0x7f06177fec80, cond=...) at galerautils/src/gu_lock.hpp:59
#2  galera::ServiceThd::thd_func (arg=0x7f0619bed5f8) at galera/src/galera_service_thd.cpp:30
#3  0x00007f062019f494 in start_thread (arg=0x7f06177ff700) at pthread_create.c:333
#4  0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
 
Thread 1 (Thread 0x7f061e143700 (LWP 16317)):
#0  __pthread_kill (threadid=<optimized out>, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:57
#1  0x00005570c7d4f1f5 in my_write_core (sig=6) at /home/jan/mysql/10.0-galera/mysys/stacktrace.c:478
#2  0x00005570c774f5a6 in handle_fatal_signal (sig=6) at /home/jan/mysql/10.0-galera/sql/signal_handler.cc:285
#3  <signal handler called>
#4  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#5  0x00007f061e1c13fa in __GI_abort () at abort.c:89
#6  0x00007f061e1b8e37 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x5570c7e94748 "(thd->variables.option_bits & (1ULL << 25)) == 0", file=file@entry=0x5570c7e91c78 "/home/jan/mysql/10.0-galera/sql/log_event.cc", line=line@entry=6639, function=function@entry=0x5570c7e96f00 <Gtid_log_event::do_apply_event(rpl_group_info*)::__PRETTY_FUNCTION__> "virtual int Gtid_log_event::do_apply_event(rpl_group_info*)") at assert.c:92
#7  0x00007f061e1b8ee2 in __GI___assert_fail (assertion=0x5570c7e94748 "(thd->variables.option_bits & (1ULL << 25)) == 0", file=0x5570c7e91c78 "/home/jan/mysql/10.0-galera/sql/log_event.cc", line=6639, function=0x5570c7e96f00 <Gtid_log_event::do_apply_event(rpl_group_info*)::__PRETTY_FUNCTION__> "virtual int Gtid_log_event::do_apply_event(rpl_group_info*)") at assert.c:101
#8  0x00005570c7841f10 in Gtid_log_event::do_apply_event (this=0x7f0607490670, rgi=0x7f060742b800) at /home/jan/mysql/10.0-galera/sql/log_event.cc:6639
#9  0x00005570c74b366a in Log_event::apply_event (this=0x7f0607490670, rgi=0x7f060742b800) at /home/jan/mysql/10.0-galera/sql/log_event.h:1360
#10 0x00005570c74a962f in apply_event_and_update_pos (ev=0x7f0607490670, thd=0x7f0607475070, rgi=0x7f060742b800, rpt=0x0) at /home/jan/mysql/10.0-galera/sql/slave.cc:3317
#11 0x00005570c74aa00b in exec_relay_log_event (thd=0x7f0607475070, rli=0x7f0611bd1cf8, serial_rgi=0x7f060742b800) at /home/jan/mysql/10.0-galera/sql/slave.cc:3659
#12 0x00005570c74ad0f7 in handle_slave_sql (arg=0x7f0611bd0000) at /home/jan/mysql/10.0-galera/sql/slave.cc:4797
#13 0x00005570c7a560f0 in pfs_spawn_thread (arg=0x7f0610897370) at /home/jan/mysql/10.0-galera/storage/perfschema/pfs.cc:1860
#14 0x00007f062019f494 in start_thread (arg=0x7f061e143700) at pthread_create.c:333
#15 0x00007f061e275aff in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

galera.galera_many_tables_pk 'innodb_plugin' w3 [ fail ]
        Test ended at 2017-08-17 10:27:24
 
CURRENT_TEST: galera.galera_many_tables_pk
mysqltest: At line 95: query 'COMMIT' succeeded - should have failed with errno 1213...
 
The result from queries just before the failure was:
< snip >
SET AUTOCOMMIT=OFF;
START TRANSACTION;
COMMIT;
CREATE TABLE sum_table (f1 INTEGER);
SELECT SUM(f1) = 100 FROM sum_table;
SUM(f1) = 100
1
SET AUTOCOMMIT=OFF;
START TRANSACTION;
SET AUTOCOMMIT=OFF;
START TRANSACTION;
UPDATE t100 SET f1 = 3;
COMMIT;
COMMIT;

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

galera.galera_gcs_fragment 'innodb_plugin' [ fail ]
        Test ended at 2017-08-17 11:39:52
 
CURRENT_TEST: galera.galera_gcs_fragment
mysqltest: In included file "./include/galera_set_sync_point.inc": 
included from /home/jan/mysql/10.0-galera/mysql-test/suite/galera/t/galera_gcs_fragment.test at line 23:
At line 1: query 'SET GLOBAL wsrep_provider_options = 'dbug=d,$galera_sync_point'' failed: 1210: Incorrect arguments to SET
 
The result from queries just before the failure was:
CREATE TABLE t1 (f1 INT PRIMARY KEY, f2 TEXT);
SET GLOBAL wsrep_cluster_address='';
SET SESSION wsrep_sync_wait=0;
SET GLOBAL wsrep_provider_options = 'dbug=d,gcs_core_after_frag_send';

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

Patch for test GAL-419 above resolves this problem as well

galera.galera_gcache_recover 'xtradb'    w2 [ fail ]
        Test ended at 2017-08-17 11:55:24
 
CURRENT_TEST: galera.galera_gcache_recover
mysqltest: In included file "./include/wait_until_connected_again.inc": 
included from /home/jan/mysql/10.0-galera/mysql-test/suite/galera/include/start_mysqld.inc at line 20:
included from /home/jan/mysql/10.0-galera/mysql-test/suite/galera/t/galera_gcache_recover.test at line 44:
At line 26: Server failed to restart
 
The result from queries just before the failure was:
CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB;
INSERT INTO t1 VALUES (1);
SET SESSION wsrep_sync_wait = 0;
SET SESSION wsrep_sync_wait = 0;
Killing server ...
INSERT INTO t1 VALUES (2);
Killing server ...
Performing --wsrep-recover ...
Using --wsrep-start-position when starting mysqld ...
INSERT INTO t1 VALUES (3);
Performing --wsrep-recover ...
Using --wsrep-start-position when starting mysqld ...
 
 - saving '/dev/shm/2/log/galera.galera_gcache_recover-xtradb/' to '/dev/shm/log/galera.galera_gcache_recover-xtradb/'

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

EDIT: this requires patch for GAL-419 and following (cnff file has configuration for innodb log size):

diff --git a/mysql-test/suite/galera/r/galera_gcache_recover_full_gcache.result b/mysql-test/suite/galera/r/galera_gcache_recover_full_gcache.result
index 30ee5772411..6424fa34e0d 100644
--- a/mysql-test/suite/galera/r/galera_gcache_recover_full_gcache.result
+++ b/mysql-test/suite/galera/r/galera_gcache_recover_full_gcache.result
@@ -15,5 +15,11 @@ Using --wsrep-start-position when starting mysqld ...
 include/diff_servers.inc [servers=1 2]
 DROP TABLE t1;
 CALL mtr.add_suppression("Skipped GCache ring buffer recovery");
+call mtr.add_suppression("Resizing redo log from *");
+call mtr.add_suppression("Starting to delete and rewrite log files.");
+call mtr.add_suppression("New log files created, LSN=*");
 include/assert_grep.inc [IST first seqno 2 not found from cache, falling back to SST]
 CALL mtr.add_suppression("Skipped GCache ring buffer recovery");
+call mtr.add_suppression("Resizing redo log from *");
+call mtr.add_suppression("Starting to delete and rewrite log files.");
+call mtr.add_suppression("New log files created, LSN=*");
diff --git a/mysql-test/suite/galera/t/galera_gcache_recover_full_gcache.test b/mysql-test/suite/galera/t/galera_gcache_recover_full_gcache.test
index b7fd9cf3aed..e3d09f99862 100644
--- a/mysql-test/suite/galera/t/galera_gcache_recover_full_gcache.test
+++ b/mysql-test/suite/galera/t/galera_gcache_recover_full_gcache.test
@@ -46,6 +46,9 @@ DROP TABLE t1;
 
 # Warning always happens when the cluster is started for the first time
 CALL mtr.add_suppression("Skipped GCache ring buffer recovery");
+call mtr.add_suppression("Resizing redo log from *");
+call mtr.add_suppression("Starting to delete and rewrite log files.");
+call mtr.add_suppression("New log files created, LSN=*");
 
 # Confirm that IST did not take place
 --let $assert_text = IST first seqno 2 not found from cache, falling back to SST
@@ -57,3 +60,6 @@ CALL mtr.add_suppression("Skipped GCache ring buffer recovery");
 
 --connection node_2
 CALL mtr.add_suppression("Skipped GCache ring buffer recovery");
+call mtr.add_suppression("Resizing redo log from *");
+call mtr.add_suppression("Starting to delete and rewrite log files.");
+call mtr.add_suppression("New log files created, LSN=*");

galera.galera_gcache_recover_full_gcache 'innodb_plugin' [ fail ]
        Test ended at 2017-08-17 12:07:04
 
CURRENT_TEST: galera.galera_gcache_recover_full_gcache
mysqltest: In included file "./include/wait_until_connected_again.inc": 
included from /home/jan/mysql/10.0-galera/mysql-test/suite/galera/include/start_mysqld.inc at line 20:
included from /home/jan/mysql/10.0-galera/mysql-test/suite/galera/t/galera_gcache_recover_full_gcache.test at line 35:
At line 26: Server failed to restart
 
The result from queries just before the failure was:
SET SESSION wsrep_sync_wait = 0;
CREATE TABLE t1 (f1 INTEGER PRIMARY KEY AUTO_INCREMENT, f2 LONGBLOB) ENGINE=InnoDB;
SET SESSION wsrep_sync_wait = 0;
Killing server ...
INSERT INTO t1 (f2) VALUES (REPEAT('x', 1024 * 1024 * 10));
INSERT INTO t1 (f2) VALUES (REPEAT('x', 1024 * 1024 * 10));
INSERT INTO t1 (f2) VALUES (REPEAT('x', 1024 * 1024 * 10));
INSERT INTO t1 (f2) VALUES (REPEAT('x', 1024 * 1024 * 10));
INSERT INTO t1 (f2) VALUES (REPEAT('x', 1024 * 1024 * 10));
Killing server ...
Performing --wsrep-recover ...
Using --wsrep-start-position when starting mysqld ...
Performing --wsrep-recover ...
Using --wsrep-start-position when starting mysqld 

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

galera.galera_gcache_recover_manytrx 'innodb_plugin' [ fail ]
        Test ended at 2017-08-17 12:27:03
 
CURRENT_TEST: galera.galera_gcache_recover_manytrx
mysqltest: In included file "./include/wait_until_connected_again.inc": 
included from /home/jan/mysql/10.0-galera/mysql-test/suite/galera/include/start_mysqld.inc at line 20:
included from /home/jan/mysql/10.0-galera/mysql-test/suite/galera/t/galera_gcache_recover_manytrx.test at line 173:
At line 26: Server failed to restart
 
The result from queries just before the failure was:
< snip >
CALL insert_multi();;
CALL insert_transaction ();;
CALL update_simple ();;
CALL insert_1k ();;
CALL insert_1m ();;
CALL insert_10m ();;
SET SESSION wsrep_sync_wait = 0;
Killing server ...
Killing server ...
ERROR HY000: Lost connection to MySQL server during query
ERROR HY000: Lost connection to MySQL server during query
ERROR HY000: Lost connection to MySQL server during query
ERROR HY000: Lost connection to MySQL server during query
ERROR HY000: Lost connection to MySQL server during query
ERROR HY000: Lost connection to MySQL server during query
ERROR HY000: Lost connection to MySQL server during query
Performing --wsrep-recover ...
Using --wsrep-start-position when starting mysqld ...
Performing --wsrep-recover ...
Using --wsrep-start-position when starting mysqld ...
 
More results from queries before failure can be found in /dev/shm/log/galera_gcache_recover_manytrx.log
 
 - saving '/dev/shm/log/galera.galera_gcache_recover_manytrx-innodb_plugin/' to '/dev/shm/log/galera.galera_gcache_recover_manytrx-innodb_plugin/'

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

EDIT: should be solved after patch from MDEV-14069

galera.galera_ist_mysqldump 'xtradb'     [ fail ]
        Test ended at 2017-08-17 14:29:00
 
CURRENT_TEST: galera.galera_ist_mysqldump
mysqltest: In included file "./include/wait_until_connected_again.inc": 
included from /home/jan/mysql/10.0-galera/mysql-test/suite/galera/include/start_mysqld.inc at line 20:
included from ./suite/galera/include/galera_st_shutdown_slave.inc at line 59:
included from /home/jan/mysql/10.0-galera/mysql-test/suite/galera/t/galera_ist_mysqldump.test at line 14:
At line 26: query 'show status' failed with wrong errno 1205: 'Lock wait timeout exceeded; try restarting transaction', instead of 0...
 
The result from queries just before the failure was:
< snip >
INSERT INTO t1 VALUES ('node1_committed_during');
INSERT INTO t1 VALUES ('node1_committed_during');
INSERT INTO t1 VALUES ('node1_committed_during');
INSERT INTO t1 VALUES ('node1_committed_during');
INSERT INTO t1 VALUES ('node1_committed_during');
COMMIT;
START TRANSACTION;
INSERT INTO t1 VALUES ('node1_to_be_committed_after');
INSERT INTO t1 VALUES ('node1_to_be_committed_after');
INSERT INTO t1 VALUES ('node1_to_be_committed_after');
INSERT INTO t1 VALUES ('node1_to_be_committed_after');
INSERT INTO t1 VALUES ('node1_to_be_committed_after');
SET AUTOCOMMIT=OFF;
START TRANSACTION;
INSERT INTO t1 VALUES ('node1_to_be_rollbacked_after');
INSERT INTO t1 VALUES ('node1_to_be_rollbacked_after');
INSERT INTO t1 VALUES ('node1_to_be_rollbacked_after');
INSERT INTO t1 VALUES ('node1_to_be_rollbacked_after');
INSERT INTO t1 VALUES ('node1_to_be_rollbacked_after');
Starting server ...
 
More results from queries before failure can be found in /dev/shm/log/galera_ist_mysqldump.log

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

galera.mysql-wsrep#31 'innodb_plugin'    [ fail ]
        Test ended at 2017-08-17 15:49:52
 
CURRENT_TEST: galera.mysql-wsrep#31
mysqltest: At line 30: (Expected position: $expected_position, found $galera_wsrep_start_position)
 
The result from queries just before the failure was:
CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB;
INSERT INTO t1 VALUES('test');
CREATE DATABASE db;
Shutting down server 2 ...
Recovering server 2 ...
Performing --wsrep-recover ...

Comment by Jan Lindström (Inactive) [ 2017-08-17 ]

EDIT: following patch makes test pass

--- a/mysql-test/suite/galera/t/galera_concurrent_ctas.test
+++ b/mysql-test/suite/galera/t/galera_concurrent_ctas.test
@@ -43,9 +43,9 @@ let $run=10;
 while($run)
 {
   --error 0,1
-  exec $MYSQL --user=root --host=127.0.0.1 --port=$NODE_MYPORT_1 test
-         < $MYSQLTEST_VARDIR/tmp/galera_concurrent.sql &
-       $MYSQL --user=root --host=127.0.0.1 --port=$NODE_MYPORT_2 test
+  exec $MYSQL --user=root --host=127.0.0.1 --port=$NODE_MYPORT_1 test \
+         < $MYSQLTEST_VARDIR/tmp/galera_concurrent.sql & \
+       $MYSQL --user=root --host=127.0.0.1 --port=$NODE_MYPORT_2 test \
          < $MYSQLTEST_VARDIR/tmp/galera_concurrent.sql;
   dec $run;

galera.galera_concurrent_ctas 'innodb_plugin' w1 [ fail ]  timeout after 900 seconds

Comment by Jan Lindström (Inactive) [ 2017-08-18 ]

galera.galera_bf_abort_for_update 'xtradb' w1 [ fail ]
        Test ended at 2017-08-17 19:55:50
 
CURRENT_TEST: galera.galera_bf_abort_for_update
mysqltest: At line 21: query 'SELECT * FROM t1 FOR UPDATE' succeeded - should have failed with errno 1213...
 
The result from queries just before the failure was:
CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB;
SET AUTOCOMMIT=OFF;
START TRANSACTION;
INSERT INTO t1 VALUES (1);
INSERT INTO t1 VALUES (1);
SELECT * FROM t1 FOR UPDATE;

Comment by Jan Lindström (Inactive) [ 2017-08-18 ]

galera.galera_wsrep_desync_wsrep_on 'innodb_plugin' w4 [ fail ]
        Test ended at 2017-08-17 19:52:23
 
CURRENT_TEST: galera.galera_wsrep_desync_wsrep_on
mysqltest: At line 23: query 'ALTER TABLE t1 ADD PRIMARY KEY (f1)' failed: 1317: Query execution was interrupted
 
The result from queries just before the failure was:
CREATE TABLE ten (f1 INTEGER);
INSERT INTO ten VALUES (0),(1),(2),(3),(4),(5),(6),(7),(8),(9);
CREATE TABLE t1 (f1 INTEGER) Engine=InnoDB;
INSERT INTO t1 (f1) SELECT 000000 + (10000 * a1.f1) + (1000 * a2.f1) + (100 * a3.f1) + (10 * a4.f1) + a5.f1 FROM ten AS a1, ten AS a2, ten AS a3, ten AS a4, ten AS a5;
SET GLOBAL wsrep_desync = TRUE;
SET SESSION wsrep_on = FALSE;
ALTER TABLE t1 ADD PRIMARY KEY (f1);

Comment by Jan Lindström (Inactive) [ 2017-08-18 ]

galera.galera_ssl_upgrade 'innodb_plugin' [ fail ]
        Test ended at 2017-08-18 07:51:02
 
CURRENT_TEST: galera.galera_ssl_upgrade
 
 
Server [mysqld.1 - pid: 3046, winpid: 3046, exit: 256] failed during test run
Server log from this test:
----------SERVER LOG START-----------
170818  7:50:14 [Warning] WSREP: option --wsrep-causal-reads is deprecated
170818  7:50:14 [Note] /home/jan/mysql/10.0-galera/sql/mysqld (mysqld 10.0.32-MariaDB-wsrep-debug) starting as process 2664 ...
170818  7:50:14 [Note] WSREP: Read nil XID from storage engines, skipping position init
170818  7:50:14 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/libgalera_smm.so'
170818  7:50:14 [Note] WSREP: wsrep_load(): Galera 3.21(r8678538) by Codership Oy <info@codership.com> loaded successfully.
170818  7:50:14 [Note] WSREP: CRC-32C: using hardware acceleration.
170818  7:50:14 [Warning] WSREP: Could not open state file for reading: '/dev/shm/mysqld.1/data//grastate.dat'
170818  7:50:14 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1, safe_to_bootsrap: 1
170818  7:50:14 [Note] WSREP: Passing config to GCS: base_dir = /dev/shm/mysqld.1/data/; base_host = 127.0.0.1; base_port = 16002; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /dev/shm/mysqld.1/data/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /dev/shm/mysqld.1/data//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.
170818  7:50:14 [Note] WSREP: GCache history reset: 00000000-0000-0000-0000-000000000000:0 -> 00000000-0000-0000-0000-000000000000:-1
170818  7:50:14 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
170818  7:50:14 [Note] WSREP: wsrep_sst_grab()
170818  7:50:14 [Note] WSREP: Start replication
170818  7:50:14 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
170818  7:50:14 [Note] WSREP: protonet asio version 0
170818  7:50:14 [Note] WSREP: Using CRC-32C for message checksums.
170818  7:50:14 [Note] WSREP: initializing ssl context
170818  7:50:14 [Note] WSREP: backend: asio
170818  7:50:14 [Note] WSREP: gcomm thread scheduling priority set to other:0 
170818  7:50:14 [Warning] WSREP: access file(/dev/shm/mysqld.1/data//gvwstate.dat) failed(No such file or directory)
170818  7:50:14 [Note] WSREP: restore pc from disk failed
170818  7:50:14 [Note] WSREP: GMCast version 0
170818  7:50:14 [Note] WSREP: (b9417633, 'ssl://0.0.0.0:16002') listening at ssl://0.0.0.0:16002
170818  7:50:14 [Note] WSREP: (b9417633, 'ssl://0.0.0.0:16002') multicast: , ttl: 1
170818  7:50:14 [Note] WSREP: EVS version 0
170818  7:50:14 [Note] WSREP: gcomm: connecting to group 'my_wsrep_cluster', peer ''
170818  7:50:14 [Note] WSREP: start_prim is enabled, turn off pc_recovery
170818  7:50:14 [Note] WSREP: Node b9417633 state prim
170818  7:50:14 [Note] WSREP: view(view_id(PRIM,b9417633,1) memb {
	b9417633,0
} joined {
} left {
} partitioned {
})
170818  7:50:14 [Note] WSREP: save pc into disk
170818  7:50:14 [Note] WSREP: gcomm: connected
170818  7:50:14 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
170818  7:50:14 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
170818  7:50:14 [Note] WSREP: Opened channel 'my_wsrep_cluster'
170818  7:50:14 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1
170818  7:50:14 [Note] WSREP: Starting new group from scratch: b9419a01-83d0-11e7-9ccc-167bbd401adf
170818  7:50:14 [Note] WSREP: STATE_EXCHANGE: sent state UUID: b9419b8f-83d0-11e7-8fa1-077ae0da96e8
170818  7:50:14 [Note] WSREP: STATE EXCHANGE: sent state msg: b9419b8f-83d0-11e7-8fa1-077ae0da96e8
170818  7:50:14 [Note] WSREP: STATE EXCHANGE: got state msg: b9419b8f-83d0-11e7-8fa1-077ae0da96e8 from 0 (jan-laptop-asus)
170818  7:50:14 [Note] WSREP: Quorum results:
	version    = 4,
	component  = PRIMARY,
	conf_id    = 0,
	members    = 1/1 (joined/total),
	act_id     = 0,
	last_appl. = -1,
	protocols  = 0/7/3 (gcs/repl/appl),
	group UUID = b9419a01-83d0-11e7-9ccc-167bbd401adf
170818  7:50:14 [Note] WSREP: Flow-control interval: [16, 16]
170818  7:50:14 [Note] WSREP: Trying to continue unpaused monitor
170818  7:50:14 [Note] WSREP: Restored state OPEN -> JOINED (0)
170818  7:50:14 [Note] WSREP: Member 0.0 (jan-laptop-asus) synced with group.
170818  7:50:14 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 0)
170818  7:50:14 [Note] WSREP: Waiting for SST to complete.
170818  7:50:14 [Note] WSREP: New cluster view: global state: b9419a01-83d0-11e7-9ccc-167bbd401adf:0, view# 1: Primary, number of nodes: 1, my index: 0, protocol version 3
170818  7:50:14 [Note] WSREP: SST complete, seqno: 0
170818  7:50:14 [Note] Plugin 'XTRADB_READ_VIEW' is disabled.
170818  7:50:14 [Note] Plugin 'XTRADB_INTERNAL_HASH_TABLES' is disabled.
170818  7:50:14 [Note] Plugin 'XTRADB_RSEG' is disabled.
170818  7:50:14 [Note] Plugin 'partition' is disabled.
170818  7:50:14 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2017-08-18 07:50:14 7f95aae34f40 InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!!
2017-08-18 07:50:14 7f95aae34f40 InnoDB: !!!!!!!! UNIV_SYNC_DEBUG switched on !!!!!!!!!
170818  7:50:14 [Note] InnoDB: The InnoDB memory heap is disabled
170818  7:50:14 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
170818  7:50:14 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
170818  7:50:14 [Note] InnoDB: Compressed tables use zlib 1.2.8
170818  7:50:14 [Note] InnoDB: Using Linux native AIO
170818  7:50:14 [Note] InnoDB: Using CPU crc32 instructions
170818  7:50:14 [Note] InnoDB: Initializing buffer pool, size = 8.0M
170818  7:50:14 [Note] InnoDB: Completed initialization of buffer pool
170818  7:50:14 [Note] InnoDB: Highest supported file format is Barracuda.
170818  7:50:14 [Note] WSREP: SSL handshake successful, remote endpoint ssl://127.0.0.1:49006 local endpoint ssl://127.0.0.1:16002 cipher: AES128-SHA compression: 
170818  7:50:14 [Note] WSREP: (b9417633, 'ssl://0.0.0.0:16002') connection established to b9455d22 ssl://127.0.0.1:16005
170818  7:50:14 [Note] WSREP: (b9417633, 'ssl://0.0.0.0:16002') turning message relay requesting on, nonlive peers: 
170818  7:50:14 [Note] WSREP: declaring b9455d22 at ssl://127.0.0.1:16005 stable
170818  7:50:14 [Note] WSREP: Node b9417633 state prim
170818  7:50:14 [Note] WSREP: view(view_id(PRIM,b9417633,2) memb {
	b9417633,0
	b9455d22,0
} joined {
} left {
} partitioned {
})
170818  7:50:14 [Note] WSREP: save pc into disk
170818  7:50:14 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2
170818  7:50:14 [Note] WSREP: STATE_EXCHANGE: sent state UUID: b99209ad-83d0-11e7-90f0-ba68aa7358f5
170818  7:50:14 [Note] WSREP: STATE EXCHANGE: sent state msg: b99209ad-83d0-11e7-90f0-ba68aa7358f5
170818  7:50:14 [Note] WSREP: STATE EXCHANGE: got state msg: b99209ad-83d0-11e7-90f0-ba68aa7358f5 from 0 (jan-laptop-asus)
170818  7:50:15 [Note] WSREP: STATE EXCHANGE: got state msg: b99209ad-83d0-11e7-90f0-ba68aa7358f5 from 1 (jan-laptop-asus)
170818  7:50:15 [Note] WSREP: Quorum results:
	version    = 4,
	component  = PRIMARY,
	conf_id    = 1,
	members    = 1/2 (joined/total),
	act_id     = 0,
	last_appl. = 0,
	protocols  = 0/7/3 (gcs/repl/appl),
	group UUID = b9419a01-83d0-11e7-9ccc-167bbd401adf
170818  7:50:15 [Note] WSREP: Flow-control interval: [23, 23]
170818  7:50:15 [Note] WSREP: Trying to continue unpaused monitor
170818  7:50:15 [Note] InnoDB: 128 rollback segment(s) are active.
170818  7:50:15 [Note] InnoDB: Waiting for purge to start
170818  7:50:15 [Note] WSREP: Member 1.0 (jan-laptop-asus) requested state transfer from '*any*'. Selected 0.0 (jan-laptop-asus)(SYNCED) as donor.
170818  7:50:15 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 0)
170818  7:50:15 [Note] InnoDB: 5.6.37 started; log sequence number 1629444
170818  7:50:15 [Note] Plugin 'INNODB_LOCK_WAITS' is disabled.
170818  7:50:15 [Note] Plugin 'INNODB_CMP_RESET' is disabled.
170818  7:50:15 [Note] Plugin 'INNODB_CMPMEM_RESET' is disabled.
170818  7:50:15 [Note] Plugin 'INNODB_CMP_PER_INDEX_RESET' is disabled.
170818  7:50:15 [Note] Plugin 'INNODB_FT_DEFAULT_STOPWORD' is disabled.
170818  7:50:15 [Note] Plugin 'INNODB_FT_DELETED' is disabled.
170818  7:50:15 [Note] Plugin 'INNODB_FT_BEING_DELETED' is disabled.
170818  7:50:15 [Note] Plugin 'INNODB_FT_CONFIG' is disabled.
170818  7:50:15 [Note] Plugin 'INNODB_FT_INDEX_CACHE' is disabled.
170818  7:50:15 [Note] Plugin 'INNODB_FT_INDEX_TABLE' is disabled.
170818  7:50:15 [Note] Plugin 'INNODB_SYS_TABLESTATS' is disabled.
170818  7:50:15 [Note] Plugin 'INNODB_SYS_INDEXES' is disabled.
170818  7:50:15 [Note] Plugin 'INNODB_SYS_COLUMNS' is disabled.
170818  7:50:15 [Note] Plugin 'INNODB_SYS_FIELDS' is disabled.
170818  7:50:15 [Note] Plugin 'INNODB_SYS_TABLESPACES' is disabled.
170818  7:50:15 [Note] Plugin 'INNODB_SYS_DATAFILES' is disabled.
170818  7:50:15 [Warning] /home/jan/mysql/10.0-galera/sql/mysqld: unknown variable 'loose-feedback-user-info=mysql-test'
170818  7:50:15 [Warning] /home/jan/mysql/10.0-galera/sql/mysqld: unknown variable 'loose-feedback-debug-startup-interval=20'
170818  7:50:15 [Warning] /home/jan/mysql/10.0-galera/sql/mysqld: unknown variable 'loose-feedback-debug-first-interval=60'
170818  7:50:15 [Warning] /home/jan/mysql/10.0-galera/sql/mysqld: unknown variable 'loose-feedback-debug-interval=60'
170818  7:50:15 [Warning] /home/jan/mysql/10.0-galera/sql/mysqld: unknown option '--loose-skip-plugin-innodb-changed-pages'
170818  7:50:15 [Note] Server socket created on IP: '127.0.0.1'.
170818  7:50:15 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
170818  7:50:15 [Note] WSREP: REPL Protocols: 7 (3, 2)
170818  7:50:15 [Note] WSREP: Assign initial position for certification: 0, protocol version: 3
170818  7:50:15 [Note] WSREP: Service thread queue flushed.
170818  7:50:15 [Note] WSREP: GCache history reset: 00000000-0000-0000-0000-000000000000:0 -> b9419a01-83d0-11e7-9ccc-167bbd401adf:0
170818  7:50:15 [Note] WSREP: Synchronized with group, ready for connections
170818  7:50:15 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
170818  7:50:15 [Note] WSREP: New cluster view: global state: b9419a01-83d0-11e7-9ccc-167bbd401adf:0, view# 2: Primary, number of nodes: 2, my index: 0, protocol version 3
170818  7:50:15 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
170818  7:50:15 [Note] WSREP: REPL Protocols: 7 (3, 2)
170818  7:50:15 [Note] WSREP: Assign initial position for certification: 0, protocol version: 3
170818  7:50:15 [Note] WSREP: Service thread queue flushed.
170818  7:50:15 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
170818  7:50:15 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'donor' --address '127.0.0.1:16007/rsync_sst' --socket '/dev/shm/tmp/mysqld.1.sock' --datadir '/dev/shm/mysqld.1/data/' --defaults-file '/dev/shm/my.cnf' --defaults-group-suffix '.1'   '' --gtid 'b9419a01-83d0-11e7-9ccc-167bbd401adf:0''
170818  7:50:15 [Note] WSREP: sst_donor_thread signaled with 0
170818  7:50:15 [Note] /home/jan/mysql/10.0-galera/sql/mysqld: ready for connections.
Version: '10.0.32-MariaDB-wsrep-debug'  socket: '/dev/shm/tmp/mysqld.1.sock'  port: 16000  Source distribution, wsrep_25.20.rc3fc46e
170818  7:50:15 [Note] WSREP: Flushing tables for SST...
170818  7:50:15 [Note] WSREP: Provider paused at b9419a01-83d0-11e7-9ccc-167bbd401adf:0 (5)
170818  7:50:15 [Note] WSREP: Tables flushed.
170818  7:50:16 [Note] WSREP: resuming provider at 5
170818  7:50:16 [Note] WSREP: Provider resumed.
170818  7:50:16 [Note] WSREP: 0.0 (jan-laptop-asus): State transfer to 1.0 (jan-laptop-asus) complete.
170818  7:50:16 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 0)
170818  7:50:16 [Note] WSREP: Member 0.0 (jan-laptop-asus) synced with group.
170818  7:50:16 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 0)
170818  7:50:16 [Note] WSREP: Synchronized with group, ready for connections
170818  7:50:16 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
170818  7:50:17 [Note] WSREP: (b9417633, 'ssl://0.0.0.0:16002') turning message relay requesting off
170818  7:50:19 [Note] WSREP: 1.0 (jan-laptop-asus): State transfer from 0.0 (jan-laptop-asus) complete.
170818  7:50:19 [Note] WSREP: Member 1.0 (jan-laptop-asus) synced with group.
170818  7:50:19 [Note] /home/jan/mysql/10.0-galera/sql/mysqld: Normal shutdown
 
170818  7:50:19 [Note] WSREP: Stop replication
170818  7:50:19 [Note] WSREP: Closing send monitor...
170818  7:50:19 [Note] WSREP: Closed send monitor.
170818  7:50:19 [Note] WSREP: gcomm: terminating thread
170818  7:50:19 [Note] WSREP: gcomm: joining thread
170818  7:50:19 [Note] WSREP: gcomm: closing backend
170818  7:50:22 [Note] WSREP: (b9417633, 'ssl://0.0.0.0:16002') connection to peer b9455d22 with addr ssl://127.0.0.1:16005 timed out, no messages seen in PT3S
170818  7:50:22 [Note] WSREP: (b9417633, 'ssl://0.0.0.0:16002') turning message relay requesting on, nonlive peers: ssl://127.0.0.1:16005 
170818  7:50:24 [Note] WSREP: (b9417633, 'ssl://0.0.0.0:16002') reconnecting to b9455d22 (ssl://127.0.0.1:16005), attempt 0
170818  7:50:24 [Note] WSREP: SSL handshake successful, remote endpoint ssl://127.0.0.1:16005 local endpoint ssl://127.0.0.1:34166 cipher: AES128-SHA compression: 
170818  7:50:24 [Note] WSREP: evs::proto(b9417633, LEAVING, view_id(REG,b9417633,2)) suspecting node: b9455d22
170818  7:50:24 [Note] WSREP: evs::proto(b9417633, LEAVING, view_id(REG,b9417633,2)) suspected node without join message, declaring inactive
170818  7:50:24 [Note] WSREP: view(view_id(NON_PRIM,b9417633,2) memb {
	b9417633,0
} joined {
} left {
} partitioned {
	b9455d22,0
})
170818  7:50:24 [Note] WSREP: view((empty))
170818  7:50:24 [Note] WSREP: gcomm: closed
170818  7:50:24 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
170818  7:50:24 [Note] WSREP: Flow-control interval: [16, 16]
170818  7:50:24 [Note] WSREP: Trying to continue unpaused monitor
170818  7:50:24 [Note] WSREP: Received NON-PRIMARY.
170818  7:50:24 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 0)
170818  7:50:24 [Note] WSREP: Received self-leave message.
170818  7:50:24 [Note] WSREP: Flow-control interval: [0, 0]
170818  7:50:24 [Note] WSREP: Trying to continue unpaused monitor
170818  7:50:24 [Note] WSREP: Received SELF-LEAVE. Closing connection.
170818  7:50:24 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 0)
170818  7:50:24 [Note] WSREP: RECV thread exiting 0: Success
170818  7:50:24 [Note] WSREP: New cluster view: global state: b9419a01-83d0-11e7-9ccc-167bbd401adf:0, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version 3
170818  7:50:24 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
170818  7:50:24 [Note] WSREP: New cluster view: global state: b9419a01-83d0-11e7-9ccc-167bbd401adf:0, view# -1: non-Primary, number of nodes: 0, my index: -1, protocol version 3
170818  7:50:24 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
170818  7:50:24 [Note] WSREP: applier thread exiting (code:0)
170818  7:50:24 [Note] WSREP: recv_thread() joined.
170818  7:50:24 [Note] WSREP: Closing replication queue.
170818  7:50:24 [Note] WSREP: Closing slave action queue.
170818  7:50:26 [Note] WSREP: rollbacker thread exiting
170818  7:50:26 [Note] Event Scheduler: Purging the queue. 0 events
170818  7:50:26 [Note] WSREP: dtor state: CLOSED
170818  7:50:26 [Note] WSREP: apply mon: entered 0
170818  7:50:26 [Note] WSREP: apply mon: entered 0
170818  7:50:26 [Note] WSREP: mon: entered 9 oooe fraction 0 oool fraction 0
170818  7:50:26 [Note] WSREP: cert index usage at exit 0
170818  7:50:26 [Note] WSREP: cert trx map usage at exit 0
170818  7:50:26 [Note] WSREP: deps set usage at exit 0
170818  7:50:26 [Note] WSREP: avg deps dist 0
170818  7:50:26 [Note] WSREP: avg cert interval 0
170818  7:50:26 [Note] WSREP: cert index size 0
170818  7:50:26 [Note] WSREP: Service thread queue flushed.
170818  7:50:26 [Note] WSREP: wsdb trx map usage 0 conn query map usage 0
170818  7:50:26 [Note] WSREP: MemPool(LocalTrxHandle): hit ratio: 0, misses: 0, in use: 0, in pool: 0
170818  7:50:26 [Note] WSREP: MemPool(SlaveTrxHandle): hit ratio: 0, misses: 0, in use: 0, in pool: 0
170818  7:50:26 [Note] WSREP: Shifting CLOSED -> DESTROYED (TO: 0)
170818  7:50:26 [Note] WSREP: Flushing memory map to disk...
170818  7:50:26 [Note] InnoDB: FTS optimize thread exiting.
170818  7:50:26 [Note] InnoDB: Starting shutdown...
170818  7:50:28 [Note] InnoDB: Shutdown completed; log sequence number 1629618
170818  7:50:28 [Note] Debug sync points hit:                   1387
170818  7:50:28 [Note] Debug sync points executed:              0
170818  7:50:28 [Note] Debug sync points max active per thread: 0
170818  7:50:28 [Note] /home/jan/mysql/10.0-galera/sql/mysqld: Shutdown complete
 
170818  7:50:28 [Warning] WSREP: option --wsrep-causal-reads is deprecated
170818  7:50:28 [Note] /home/jan/mysql/10.0-galera/sql/mysqld (mysqld 10.0.32-MariaDB-wsrep-debug) starting as process 3047 ...
170818  7:50:28 [Note] WSREP: Read nil XID from storage engines, skipping position init
170818  7:50:28 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/libgalera_smm.so'
170818  7:50:28 [Note] WSREP: wsrep_load(): Galera 3.21(r8678538) by Codership Oy <info@codership.com> loaded successfully.
170818  7:50:28 [Note] WSREP: CRC-32C: using hardware acceleration.
170818  7:50:28 [Note] WSREP: Found saved state: b9419a01-83d0-11e7-9ccc-167bbd401adf:0, safe_to_bootsrap: 0
170818  7:50:28 [Note] WSREP: Passing config to GCS: base_dir = /dev/shm/mysqld.1/data/; base_host = 127.0.0.1; base_port = ; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /dev/shm/mysqld.1/data/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /dev/shm/mysqld.1/data//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.check
170818  7:50:28 [Note] WSREP: Assign initial position for certification: 0, protocol version: -1
170818  7:50:28 [Note] WSREP: wsrep_sst_grab()
170818  7:50:28 [Note] WSREP: Start replication
170818  7:50:28 [Note] WSREP: Setting initial position to b9419a01-83d0-11e7-9ccc-167bbd401adf:0
170818  7:50:28 [Note] WSREP: protonet asio version 0
170818  7:50:28 [Note] WSREP: Using CRC-32C for message checksums.
170818  7:50:28 [Note] WSREP: initializing ssl context
170818  7:50:28 [Note] WSREP: backend: asio
170818  7:50:28 [Note] WSREP: gcomm thread scheduling priority set to other:0 
170818  7:50:28 [Warning] WSREP: access file(/dev/shm/mysqld.1/data//gvwstate.dat) failed(No such file or directory)
170818  7:50:28 [Note] WSREP: restore pc from disk failed
170818  7:50:28 [Note] WSREP: GMCast version 0
170818  7:50:28 [Note] WSREP: (c1ed4525, 'ssl://0.0.0.0:0') listening at ssl://0.0.0.0:0
170818  7:50:28 [Note] WSREP: (c1ed4525, 'ssl://0.0.0.0:0') multicast: , ttl: 1
170818  7:50:28 [Note] WSREP: EVS version 0
170818  7:50:28 [Note] WSREP: gcomm: connecting to group 'my_wsrep_cluster', peer '127.0.0.1:'
170818  7:50:31 [Warning] WSREP: no nodes coming from prim view, prim not possible
170818  7:50:31 [Note] WSREP: view(view_id(NON_PRIM,c1ed4525,1) memb {
	c1ed4525,0
} joined {
} left {
} partitioned {
})
170818  7:50:32 [Warning] WSREP: last inactive check more than PT1.5S ago (PT3.50199S), skipping check
170818  7:51:01 [Note] WSREP: view((empty))
170818  7:51:01 [ERROR] WSREP: failed to open gcomm backend connection: 110: failed to reach primary view: 110 (Connection timed out)
	 at gcomm/src/pc.cpp:connect():158
170818  7:51:01 [ERROR] WSREP: gcs/src/gcs_core.cpp:gcs_core_open():208: Failed to open backend connection: -110 (Connection timed out)
170818  7:51:01 [ERROR] WSREP: gcs/src/gcs.cpp:gcs_open():1457: Failed to open channel 'my_wsrep_cluster' at 'gcomm://127.0.0.1:': -110 (Connection timed out)
170818  7:51:01 [ERROR] WSREP: gcs connect failed: Connection timed out
170818  7:51:01 [ERROR] WSREP: wsrep::connect(gcomm://127.0.0.1:) failed: 7
170818  7:51:01 [ERROR] Aborting
 
170818  7:51:01 [Note] WSREP: Service disconnected.
170818  7:51:02 [Note] WSREP: Some threads may fail to exit.
170818  7:51:02 [Note] Debug sync points hit:                   0
170818  7:51:02 [Note] Debug sync points executed:              0
170818  7:51:02 [Note] Debug sync points max active per thread: 0
170818  7:51:02 [Note] /home/jan/mysql/10.0-galera/sql/mysqld: Shutdown complete
 
----------SERVER LOG END-------------
mysqltest failed but provided no output
The result from queries just before the failure was:
< snip >
SELECT VARIABLE_VALUE = 'Synced' FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_local_state_comment';
VARIABLE_VALUE = 'Synced'
1
SELECT VARIABLE_VALUE = 2 FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size';
VARIABLE_VALUE = 2
1
 
 
 
 - saving '/dev/shm/log/galera.galera_ssl_upgrade-innodb_plugin/' to '/dev/shm/log/galera.galera_ssl_upgrade-innodb_plugin/'

Comment by Jan Lindström (Inactive) [ 2017-08-18 ]

galera.mysql-wsrep#33 'innodb_plugin'    [ fail ]
        Test ended at 2017-08-18 08:13:53
 
CURRENT_TEST: galera.mysql-wsrep#33
mysqltest: In included file "./include/wait_until_connected_again.inc": 
included from ./suite/galera/include/galera_load_provider.inc at line 10:
included from ./suite/galera/include/galera_st_disconnect_slave.inc at line 57:
included from /home/jan/mysql/10.0-galera/mysql-test/suite/galera/t/mysql-wsrep#33.test at line 9:
At line 18: query 'show status' failed with wrong errno 1205: 'Lock wait timeout exceeded; try restarting transaction', instead of 0...
 
The result from queries just before the failure was:
< snip >
INSERT INTO t1 VALUES ('node1_committed_during');
INSERT INTO t1 VALUES ('node1_committed_during');
INSERT INTO t1 VALUES ('node1_committed_during');
INSERT INTO t1 VALUES ('node1_committed_during');
INSERT INTO t1 VALUES ('node1_committed_during');
COMMIT;
START TRANSACTION;
INSERT INTO t1 VALUES ('node1_to_be_committed_after');
INSERT INTO t1 VALUES ('node1_to_be_committed_after');
INSERT INTO t1 VALUES ('node1_to_be_committed_after');
INSERT INTO t1 VALUES ('node1_to_be_committed_after');
INSERT INTO t1 VALUES ('node1_to_be_committed_after');
SET AUTOCOMMIT=OFF;
START TRANSACTION;
INSERT INTO t1 VALUES ('node1_to_be_rollbacked_after');
INSERT INTO t1 VALUES ('node1_to_be_rollbacked_after');
INSERT INTO t1 VALUES ('node1_to_be_rollbacked_after');
INSERT INTO t1 VALUES ('node1_to_be_rollbacked_after');
INSERT INTO t1 VALUES ('node1_to_be_rollbacked_after');
Loading wsrep provider ...
 
More results from queries before failure can be found in /dev/shm/log/mysql-wsrep#33.log
 
 - saving '/dev/shm/log/galera.mysql-wsrep#33-innodb_plugin/' to '/dev/shm/log/galera.mysql-wsrep#33-innodb_plugin/'

Comment by Jan Lindström (Inactive) [ 2017-08-18 ]

galera.galera_var_auto_inc_control_on 'innodb_plugin' w2 [ fail ]
        Test ended at 2017-08-18 05:50:50
 
CURRENT_TEST: galera.galera_var_auto_inc_control_on
--- /usr/share/mysql/mysql-test/suite/galera/r/galera_var_auto_inc_control_on.result	2017-08-18 05:19:49.000000000 +0000
+++ /run/shm/var/2/log/galera_var_auto_inc_control_on.reject	2017-08-18 05:50:50.509852958 +0000
@@ -4,7 +4,7 @@
 1
 SELECT @@global.auto_increment_offset = (SELECT VARIABLE_VALUE FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_local_index') + 1;
 @@global.auto_increment_offset = (SELECT VARIABLE_VALUE FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_local_index') + 1
-1
+0
 INSERT INTO t1 VALUES (DEFAULT, 'node1');;
 INSERT INTO t1 VALUES (DEFAULT, 'node2');;
 SELECT @@auto_increment_increment = (SELECT VARIABLE_VALUE FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_size');
@@ -12,7 +12,7 @@
 1
 SELECT @@global.auto_increment_offset = (SELECT VARIABLE_VALUE FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_local_index') + 1;
 @@global.auto_increment_offset = (SELECT VARIABLE_VALUE FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_local_index') + 1
-1
+0
 INSERT INTO t1 VALUES (DEFAULT, 'node2'), (DEFAULT, 'node2'), (DEFAULT, 'node2'), (DEFAULT, 'node2'), (DEFAULT, 'node2'), (DEFAULT, 'node2'), (DEFAULT, 'node2'), (DEFAULT, 'node2'), (DEFAULT, 'node2'), (DEFAULT, 'node2');;
 INSERT INTO t1 VALUES (DEFAULT, 'node1'), (DEFAULT, 'node1'), (DEFAULT, 'node1'), (DEFAULT, 'node1'), (DEFAULT, 'node1'), (DEFAULT, 'node1'), (DEFAULT, 'node1'), (DEFAULT, 'node1'), (DEFAULT, 'node1'), (DEFAULT, 'node1');;
 SELECT COUNT(*) = 22 FROM t1;
 
mysqltest: Result content mismatch

Comment by Jan Lindström (Inactive) [ 2017-08-18 ]

galera.MW-44 'innodb_plugin'             w4 [ fail ]
        Test ended at 2017-08-18 01:53:25
 
CURRENT_TEST: galera.MW-44
mysqltest: At line 15: query 'SET SESSION wsrep_osu_method=TOI' failed: 1213: Deadlock found when trying to get lock; try restarting transaction
 
The result from queries just before the failure was:
TRUNCATE TABLE mysql.general_log;
TRUNCATE TABLE mysql.general_log;
SET SESSION wsrep_osu_method=TOI;
 
 - saving '/dev/shm/var/4/log/galera.MW-44-innodb_plugin/' to '/dev/shm/var/log/galera.MW-44-innodb_plugin/'
 
Retrying test galera.MW-44, attempt(2/3)...
 
worker[4] > Restart  - not started
worker[4] > Restart  - not started
***Warnings generated in error logs during shutdown after running tests: galera.MW-44
 
170818  1:53:21 [Warning] Although a path was specified for the --log option, log tables are used. To enable logging to files use the --log-output option.
170818  1:53:21 [Warning] Although a path was specified for the --log-slow-queries option, log tables are used. To enable logging to files use the --log-output=file option.
170818  1:53:21 [Warning] Although a path was specified for the --log option, log tables are used. To enable logging to files use the --log-output option.
170818  1:53:21 [Warning] Although a path was specified for the --log-slow-queries option, log tables are used. To enable logging to files use the --log-output=file option.
 
galera.MW-44 'innodb_plugin'             w4 [ retry-pass ]     15

Comment by Jan Lindström (Inactive) [ 2017-08-18 ]

galera.galera_var_retry_autocommit 'innodb_plugin' w4 [ fail ]
        Test ended at 2017-08-18 06:28:51
 
CURRENT_TEST: galera.galera_var_retry_autocommit
mysqltest: At line 73: query 'INSERT INTO t1 (f1) VALUES (5),(6)' succeeded - should have failed with errno 1213...
 
The result from queries just before the failure was:
< snip >
SET SESSION wsrep_retry_autocommit = 1;
INSERT INTO t1 (f1) VALUES (3),(4);;
TRUNCATE TABLE t1;
SELECT * FROM test.t1;
f1	f2
3	0
4	0
CREATE PROCEDURE repeated_truncate ()
BEGIN
DECLARE i INT;
DECLARE CONTINUE HANDLER FOR SQLEXCEPTION BEGIN END;
SET i = 0;
WHILE i <= 1000 DO
TRUNCATE TABLE t1;
SET i = i + 1;
END WHILE;
END|
CALL repeated_truncate();
SET SESSION wsrep_retry_autocommit = 1;
INSERT INTO t1 (f1) VALUES (5),(6);

Comment by Jan Lindström (Inactive) [ 2017-08-19 ]

 

galera.pxc-421 'innodb_plugin' w4 [ fail ]
Test ended at 2017-08-18 06:51:31

CURRENT_TEST: galera.pxc-421
— /usr/share/mysql/mysql-test/suite/galera/r/pxc-421.result 2017-08-18 04:34:54.000000000 -0400
+++ /dev/shm/var/4/log/pxc-421.reject 2017-08-18 06:51:31.178182821 -0400
@@ -25,11 +25,11 @@
SET GLOBAL wsrep_slave_threads = 1;
SELECT COUNT FROM t1;
COUNT
-16
+13
SET GLOBAL auto_increment_offset = 2;
SELECT COUNT FROM t1;
COUNT
-15
+12
show global variables like 'wsrep_slave_threads';
Variable_name Value
wsrep_slave_threads 12
{noformat]

Comment by Jan Lindström (Inactive) [ 2017-08-19 ]

galera.galera_var_node_address 'xtradb'  w4 [ pass ]     42
worker[4] > Restart [mysqld.1 - pid: 23552, winpid: 23552] - using different config file
worker[4] > Restart [mysqld.2 - pid: 23559, winpid: 23559] - using different config file
worker[4] > Restart [mysqld.3 - pid: 23562, winpid: 23562] - using different config file
worker[4] > Restart [mysqld.4 - pid: 23569, winpid: 23569] - using different config file
***Warnings generated in error logs during shutdown after running tests: galera.galera_var_node_address
 
170818  6:08:02 [ERROR] WSREP: gcs/src/gcs_core.cpp:core_handle_uuid_msg():868: STATE EXCHANGE: failed for: 1e250f4c-83fd-11e7-87aa-cee9854ab148: -107 (Transport endpoint is not connected)

Comment by Jan Lindström (Inactive) [ 2017-08-19 ]

galera.lp1376747-2 'innodb_plugin'       w2 [ fail ]
        Test ended at 2017-08-18 09:39:33
 
CURRENT_TEST: galera.lp1376747-2
mysqltest: At line 8: query 'FLUSH TABLES t1 FOR EXPORT' failed: 1146: Table 'test.t1' doesn't exist
 
The result from queries just before the failure was:
CREATE TABLE t1 (id INT PRIMARY KEY) ENGINE=InnoDB;
INSERT INTO t1 VALUES (1);
FLUSH TABLES t1 FOR EXPORT;
 
 - saving '/run/shm/var/2/log/galera.lp1376747-2-innodb_plugin/' to '/run/shm/var/log/galera.lp1376747-2-innodb_plugin/'
 
Retrying test galera.lp1376747-2, attempt(2/3)...

Comment by Jan Lindström (Inactive) [ 2017-08-19 ]

galera.lp1376747 'innodb_plugin'         w2 [ fail ]
        Test ended at 2017-08-18 05:19:37
 
CURRENT_TEST: galera.lp1376747
mysqltest: At line 8: query 'FLUSH TABLES t1 WITH READ LOCK' failed: 1146: Table 'test.t1' doesn't exist
 
The result from queries just before the failure was:
CREATE TABLE t1 (id INT PRIMARY KEY) ENGINE=InnoDB;
INSERT INTO t1 VALUES (1);
FLUSH TABLES t1 WITH READ LOCK;
 
 - saving '/run/shm/var/2/log/galera.lp1376747-innodb_plugin/' to '/run/shm/var/log/galera.lp1376747-innodb_plugin/'
 
Retrying test galera.lp1376747, attempt(2/3)..

Comment by Jan Lindström (Inactive) [ 2017-08-20 ]

galera.galera_toi_ddl_nonconflicting 'xtradb' w4 [ fail ]
        Test ended at 2017-08-19 01:36:51
 
CURRENT_TEST: galera.galera_toi_ddl_nonconflicting
--- /usr/share/mysql/mysql-test/suite/galera/r/galera_toi_ddl_nonconflicting.result	2017-08-19 00:55:30.000000000 -0400
+++ /dev/shm/var/4/log/galera_toi_ddl_nonconflicting.reject	2017-08-19 01:36:51.012924061 -0400
@@ -7,7 +7,7 @@
 1
 SELECT COUNT(*) = 2 FROM INFORMATION_SCHEMA.STATISTICS WHERE TABLE_NAME = 't1';
 COUNT(*) = 2
-1
+0
 SELECT COUNT(*) = 2 FROM t1;
 COUNT(*) = 2
 1
 
mysqltest: Result content mismatch

Comment by Jan Lindström (Inactive) [ 2017-08-20 ]

galera.galera_parallel_simple 'xtradb'   w4 [ fail ]
        Test ended at 2017-08-19 01:37:04
 
CURRENT_TEST: galera.galera_parallel_simple
mysqltest: At line 16: query 'LOCK TABLE t1 WRITE' failed: 1146: Table 'test.t1' doesn't exist
 
The result from queries just before the failure was:
CREATE TABLE t1 (id INT) ENGINE=InnoDB;
CREATE TABLE t2 (id INT) ENGINE=InnoDB;
SET GLOBAL wsrep_slave_threads = 2;
LOCK TABLE t1 WRITE;
 
 - saving '/dev/shm/var/4/log/galera.galera_parallel_simple-xtradb/' to '/dev/shm/var/log/galera.galera_parallel_simple-xtradb/'
 
Retrying test galera.galera_parallel_simple, attempt(2/3)...

Comment by Jan Lindström (Inactive) [ 2017-08-21 ]

galera.galera_admin 'innodb_plugin'      w4 [ fail ]
        Test ended at 2017-08-20 06:20:04
 
CURRENT_TEST: galera.galera_admin
--- /usr/share/mysql/mysql-test/suite/galera/r/galera_admin.result	2017-08-20 04:52:12.000000000 +0000
+++ /run/shm/var/4/log/galera_admin.reject	2017-08-20 06:20:04.017868290 +0000
@@ -14,6 +14,11 @@
 Table	Op	Msg_type	Msg_text
 test.t1	analyze	status	OK
 test.t2	analyze	status	OK
+Timeout in wait_condition.inc for SELECT VARIABLE_VALUE = 1834 + 1 FROM INFORMATION_SCHEMA.SESSION_STATUS WHERE VARIABLE_NAME = 'wsrep_last_committed'
+Id	User	Host	db	Command	Time	State	Info	Progress
+1	system user		NULL	Sleep	30	committed 1836	NULL	0.000
+2	system user		NULL	Sleep	64	wsrep aborter idle	NULL	0.000
+87	root	localhost:58468	test	Query	0	init	show full processlist	0.000
 # OPTIMIZE test
 OPTIMIZE TABLE t1, t2;
 Table	Op	Msg_type	Msg_text

Comment by Jan Lindström (Inactive) [ 2017-08-21 ]

From this point on failing test cases are from mariadb-10.1 branch i.e. https://github.com/MariaDB/server/tree/10.1

galera.galera_var_max_ws_rows 'xtradb'   w4 [ fail ]
        Test ended at 2017-08-21 11:09:53
 
CURRENT_TEST: galera.galera_var_max_ws_rows
mysqltest: At line 17: query 'INSERT INTO t1 (f2) VALUES (3)' failed: 1180: wsrep_max_ws_rows exceeded
 
The result from queries just before the failure was:
CREATE TABLE ten (f1 INTEGER) ENGINE=InnoDB;
INSERT INTO ten VALUES (1),(2),(3),(4),(5),(6),(7),(8),(9),(10);
CREATE TABLE t1 (f1 INTEGER AUTO_INCREMENT PRIMARY KEY, f2 INTEGER) ENGINE=InnoDB;
SET GLOBAL wsrep_max_ws_rows = 4;
START TRANSACTION;
INSERT INTO t1 (f2) VALUES (1);
INSERT INTO t1 (f2) VALUES (2);
INSERT INTO t1 (f2) VALUES (3);

Comment by Jan Lindström (Inactive) [ 2017-08-21 ]

galera.MW-286 'innodb_plugin'            w1 [ fail ]
        Test ended at 2017-08-21 13:27:16
 
CURRENT_TEST: galera.MW-286
mysqltest: At line 29: query 'SET wsrep_on = TRUE' failed: 1213: Deadlock found when trying to get lock; try restarting transaction
 
The result from queries just before the failure was:
CREATE TABLE ten (f1 INTEGER);
INSERT INTO ten VALUES (0),(1),(2),(3),(4),(5),(6),(7),(8),(9);
CREATE TABLE t1 (f1 INTEGER) Engine=InnoDB;
INSERT INTO t1 (f1) SELECT 000000 + (10000 * a1.f1) + (1000 * a2.f1) + (100 * a3.f1) + (10 * a4.f1) + a5.f1 FROM ten AS a1, ten AS a2, ten AS a3, ten AS a4, ten AS a5;
INSERT INTO t1 (f1) SELECT 100000 + (10000 * a1.f1) + (1000 * a2.f1) + (100 * a3.f1) + (10 * a4.f1) + a5.f1 FROM ten AS a1, ten AS a2, ten AS a3, ten AS a4, ten AS a5;;
SET GLOBAL wsrep_desync = TRUE;
SET wsrep_on = FALSE;
ALTER TABLE t1 ADD PRIMARY KEY (f1);
ERROR 70100: Query execution was interrupted
SET SESSION wsrep_sync_wait = 0;
SET wsrep_on = TRUE;

Comment by Jan Lindström (Inactive) [ 2017-08-25 ]

galera.galera_as_master 'innodb_plugin'  w2 [ fail ]
        Test ended at 2017-08-25 10:34:35
 
CURRENT_TEST: galera.galera_as_master
--- /home/jan/mysql/10.1/mysql-test/suite/galera/r/galera_as_master.result	2017-08-24 11:17:58.286874543 +0300
+++ /home/jan/mysql/10.1/mysql-test/suite/galera/r/galera_as_master.reject	2017-08-25 10:34:34.850828345 +0300
@@ -30,7 +30,6 @@
 SHOW TABLES;
 Tables_in_test
 t1
-t4
 SELECT * FROM t1;
 f1
 1
 
mysqltest: Result length mismatch

Comment by Jan Lindström (Inactive) [ 2017-08-25 ]

galera.galera_pc_ignore_sb 'innodb_plugin' w1 [ fail ]
        Test ended at 2017-08-25 08:30:03
 
CURRENT_TEST: galera.galera_pc_ignore_sb
mysqltest: At line 24: query 'INSERT INTO t1 VALUES (1)' failed: 1205: Lock wait timeout exceeded; try restarting transaction
 
The result from queries just before the failure was:
SET GLOBAL wsrep_provider_options = 'pc.ignore_sb=true';
Killing server ...
CREATE TABLE t1 (f1 INTEGER) ENGINE=InnoDB;
INSERT INTO t1 VALUES (1);

Comment by Jan Lindström (Inactive) [ 2017-08-25 ]

galera.galera_lock_table 'xtradb'        w2 [ fail ]
        Test ended at 2017-08-25 05:03:43
 
CURRENT_TEST: galera.galera_lock_table
mysqltest: At line 15: query 'LOCK TABLE t1 READ' failed: 1146: Table 'test.t1' doesn't exist
 
The result from queries just before the failure was:
CREATE TABLE t1 (id INT PRIMARY KEY) ENGINE=InnoDB;
CREATE TABLE t2 (id INT PRIMARY KEY) ENGINE=InnoDB;
LOCK TABLE t1 READ;

Comment by Jan Lindström (Inactive) [ 2017-08-26 ]

galera.MW-284 'innodb_plugin'            w2 [ fail ]
        Test ended at 2017-08-25 12:01:31
 
CURRENT_TEST: galera.MW-284
mysqltest: At line 41: query 'INSERT INTO t1 VALUES (1)' failed: 1047: WSREP has not yet prepared node for application use
 
The result from queries just before the failure was:
CREATE TABLE t1 (f1 INTEGER) ENGINE=InnoDB;
SET GLOBAL wsrep_provider_options='gmcast.isolate=1';
SET SESSION wsrep_on = OFF;
SET SESSION wsrep_on = ON;
SET global wsrep_sync_wait=0;
START SLAVE;
include/wait_for_slave_param.inc [Slave_IO_Running]
SET GLOBAL wsrep_provider_options='gmcast.isolate=0';
include/wait_for_slave_to_start.inc
INSERT INTO t1 VALUES (1);

Comment by Jan Lindström (Inactive) [ 2017-08-28 ]

galera.galera_as_slave 'xtradb'          w2 [ fail ]
        Test ended at 2017-08-26 01:33:36
 
CURRENT_TEST: galera.galera_as_slave
mysqltest: In included file "./include/wait_condition.inc": 
included from /usr/share/mysql/mysql-test/suite/galera/t/galera_as_slave.test at line 45:
At line 43: Query 'SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1';' didn't return a result set
 
The result from queries just before the failure was:
START SLAVE;
CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB;
INSERT INTO t1 VALUES(1);
INSERT INTO t1 VALUES (2);
SELECT COUNT(*) = 2 FROM t1;
COUNT(*) = 2
1
INSERT INTO t1 VALUES (3);
SELECT COUNT(*) = 3 FROM t1;
COUNT(*) = 3
1
DROP TABLE t1;
 
Warnings from just before the error:
Error 1317 Query execution was interrupted

Comment by Jan Lindström (Inactive) [ 2017-08-31 ]

10.2 (maybe 10.1):

galera.galera_kill_applier 'innodb'      w2 [ fail ]
        Test ended at 2017-08-31 05:43:03
 
CURRENT_TEST: galera.galera_kill_applier
mysqltest: At line 14: query 'KILL $applier_thread' failed with wrong errno 1064: 'You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1', instead of 1095..

Comment by Alice Sherepa [ 2017-11-07 ]

10.1 http://buildbot.askmonty.org/buildbot/builders/kvm-rpm-fedora26-amd64/builds/501/steps/mtr/logs/stdio

galera.GAL-401 'xtradb'                  w1 [ fail ]
        Test ended at 2017-11-07 00:54:15
 
CURRENT_TEST: galera.GAL-401
--- /usr/share/mysql-test/suite/galera/r/GAL-401.result	2017-11-06 21:05:39.000000000 +0000
+++ /dev/shm/var/1/log/GAL-401.reject	2017-11-07 00:54:15.210133061 +0000
@@ -8,7 +8,7 @@
 SET wsrep_dirty_reads=0;
 SHOW STATUS LIKE 'wsrep_desync_count';
 Variable_name	Value
-wsrep_desync_count	0
+wsrep_desync_count	1
 SET @@global.wsrep_desync = 0;
 SET SESSION wsrep_sync_wait=15;
 SHOW CREATE TABLE t1;
 
mysqltest: Result content mismatch

galera.GAL-480 'xtradb'                  w1 [ fail ]
        Test ended at 2017-11-07 00:54:46
 
CURRENT_TEST: galera.GAL-480
mysqltest: At line 13: query 'ALTER TABLE t1 DROP COLUMN f1' failed: 2013: Lost connection to MySQL server during query
 
The result from queries just before the failure was:
CREATE TABLE t1 (f1 CHAR(10), f0 integer) ENGINE=InnoDB;
FLUSH TABLE t1 FOR EXPORT;
UNLOCK TABLES;
ALTER TABLE t1 DROP COLUMN f1;
SET SESSION wsrep_osu_method='RSU';
ALTER TABLE t1 ADD COLUMN f1 CHAR(10);
ALTER TABLE t1 DROP COLUMN f1;
 
2017-11-07  0:54:43 140211984481024 [ERROR] WSREP: FSM: no such a transition JOINED -> DONOR
171107  0:54:43 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 10.1.29-MariaDB
key_buffer_size=1048576
read_buffer_size=131072
max_used_connections=2
max_threads=153
thread_count=4
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 62914 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x7f85abccc308
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f85a5873038 thread_stack 0x48400
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55789ab15cee]
/usr/sbin/mysqld(handle_fatal_signal+0x53b)[0x55789a64663b]
/lib64/libpthread.so.0(+0x123b0)[0x7f85af4b53b0]
/lib64/libc.so.6(gsignal+0xcb)[0x7f85ad6cd9fb]
/lib64/libc.so.6(abort+0x1b0)[0x7f85ad6cf800]
/usr/lib64/galera/libgalera_smm.so(_ZN6galera3FSMINS_10Replicator5StateENS_13ReplicatorSMM10TransitionENS_10EmptyGuardENS_11EmptyActionEE8shift_toES2_+0x1bd)[0x7f85a617c70d]
/usr/lib64/galera/libgalera_smm.so(_ZN6galera13ReplicatorSMM6desyncEv+0xce)[0x7f85a6174b3e]
/usr/lib64/galera/libgalera_smm.so(galera_desync+0x2e)[0x7f85a618eaee]
/usr/sbin/mysqld(+0x547b5f)[0x55789a5e7b5f]
/usr/sbin/mysqld(_ZN19Sql_cmd_alter_table7executeEP3THD+0x48e)[0x55789a58f9ae]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x1302)[0x55789a4b22d2]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x368)[0x55789a4ba0a8]
/usr/sbin/mysqld(+0x41a965)[0x55789a4ba965]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1e73)[0x55789a4bcfa3]
/usr/sbin/mysqld(_Z10do_commandP3THD+0x162)[0x55789a4bde72]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x182)[0x55789a58cde2]
/usr/sbin/mysqld(handle_one_connection+0x40)[0x55789a58cf80]
/usr/sbin/mysqld(+0x81736a)[0x55789a8b736a]
/lib64/libpthread.so.0(+0x736d)[0x7f85af4aa36d]
/lib64/libc.so.6(clone+0x3f)[0x7f85ad7a7e1f]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7f85a1e2bc60): ALTER TABLE t1 DROP COLUMN f1
Connection ID (thread ID): 6
Status: NOT_KILLED

Comment by Jan Lindström (Inactive) [ 2018-02-20 ]

galera.galera_gtid 'xtradb'              w4 [ fail ]
        Test ended at 2018-02-19 07:34:43
 
CURRENT_TEST: galera.galera_gtid
 
 
Could not execute 'check-testcase' before testcase 'galera.galera_gtid' (res: 1):
mysqltest: Logging to '/dev/shm/var/4/tmp/check-mysqld_2.log'.
mysqltest: Results saved in '/dev/shm/var/4/tmp/check-mysqld_2.result'.
mysqltest: Connecting to server localhost:16021 (socket /dev/shm/var/tmp/4/mysqld.2.sock) as 'root', connection 'default', attempt 0 ...
mysqltest: ... Connected.
mysqltest: Start processing test commands from './include/check-testcase.test' ...
mysqltest: At line 78: query 'call mtr.check_testcase()' failed: 1047: WSREP has not yet prepared node for application use
not ok
mysqltest failed but provided no output

Comment by Jan Lindström (Inactive) [ 2018-02-20 ]

galera.galera_gtid_slave 'xtradb'        w1 [ fail ]
        Test ended at 2018-02-19 12:48:35
 
CURRENT_TEST: galera.galera_gtid_slave
--- /usr/share/mysql-test/suite/galera/r/galera_gtid_slave.result	2018-02-19 12:01:03.000000000 +0200
+++ /dev/shm/var/1/log/galera_gtid_slave.reject	2018-02-19 12:48:35.359131708 +0200
@@ -20,6 +20,19 @@
 1-1-4,2-2-2,2-3-3
 DROP TABLE t1,t2;
 reset master;
+Timeout in wait_condition.inc for SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1';
+Id	User	Host	db	Command	Time	State	Info	Progress
+1	system user		NULL	Sleep	40	wsrep aborter idle	NULL	0.000
+2	system user		NULL	Sleep	32	committed 7	NULL	0.000
+6	root	localhost:46393	test	Sleep	35		NULL	0.000
+7	root	localhost:46395	test	Query	0	init	show full processlist	0.000
+8	system user		NULL	Connect	33	Waiting for master to send event	NULL	0.000
+9	system user		NULL	Connect	33	Slave has read all relay log; waiting for the slave I/O thread to update it	NULL	0.000
+Timeout in wait_condition.inc for SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1';
+Id	User	Host	db	Command	Time	State	Info	Progress
+1	system user		NULL	Sleep	70	wsrep aborter idle	NULL	0.000
+2	system user		NULL	Sleep	63	committed 6	NULL	0.000
+6	root	localhost:46009	test	Query	0	init	show full processlist	0.000
 STOP SLAVE;
 RESET SLAVE ALL;
 reset master;

Comment by Jan Lindström (Inactive) [ 2018-02-20 ]

galera.galera_unicode_identifiers 'innodb_plugin' w2 [ fail ]
        Test ended at 2018-02-19 10:51:41
 
CURRENT_TEST: galera.galera_unicode_identifiers
mysqltest: At line 52: query 'USE `database with space`' failed: 1049: Unknown database 'database with space'
 
The result from queries just before the failure was:
< snip >
UNIQUE `index name with space` (`second column with space`)
);
INSERT INTO `table with space` VALUES (1, 1);
CREATE DATABASE `база`;
USE `база`;
CREATE TABLE `таблица` (
`първа_колона` INTEGER PRIMARY KEY,
`втора_колона` INTEGER,
UNIQUE `индекс` (`втора_колона`)
);
INSERT INTO `таблица` VALUES (1, 1);
CREATE DATABASE `втора база`;
USE `втора база`;
CREATE TABLE `втора таблица` (
`първа колона` INTEGER,
`втора колона` INTEGER,
KEY `първи индекс` (`първа колона`)
);
INSERT INTO `втора таблица` VALUES (1, 1);
USE `database with space`;

Comment by Jan Lindström (Inactive) [ 2018-02-22 ]

galera.galera_wsrep_log_conficts 'xtradb' w1 [ fail ]
        Test ended at 2018-02-21 18:41:12
 
CURRENT_TEST: galera.galera_wsrep_log_conficts
 
 
Could not execute 'check-testcase' before testcase 'galera.galera_wsrep_log_conficts' (res: 1):
mysqltest: Logging to '/dev/shm/var/1/tmp/check-mysqld_2.log'.
mysqltest: Results saved in '/dev/shm/var/1/tmp/check-mysqld_2.result'.
mysqltest: Connecting to server localhost:16061 (socket /dev/shm/var/tmp/1/mysqld.2.sock) as 'root', connection 'default', attempt 0 ...
mysqltest: ... Connected.
mysqltest: Start processing test commands from './include/check-testcase.test' ...
mysqltest: At line 78: query 'call mtr.check_testcase()' failed: 1047: WSREP has not yet prepared node for application use
not ok
mysqltest failed but provided no output

Comment by Marko Mäkelä [ 2019-03-06 ]

The test galera.MW-336 remains disabled on 10.4 due to result differences like this, on all platforms that run Galera tests:

10.4 2a791c53ad93c8bc1441dd227000234bd49c4990

CURRENT_TEST: galera.MW-336
--- /usr/share/mysql/mysql-test/suite/galera/r/MW-336.result	2019-03-05 10:04:26.000000000 -0500
+++ /dev/shm/var/1/log/MW-336.reject	2019-03-05 14:23:45.776472751 -0500
@@ -13,6 +13,27 @@
 COUNT(*)
 13
 # Wait 9 slave threads to exit 1
+Timeout in wait_condition.inc for SELECT COUNT(*) = 2 FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER = 'system user' AND (STATE IS NULL OR STATE NOT LIKE 'InnoDB%');
+Id	User	Host	db	Command	Time	State	Info	Progress
+1	system user		NULL	Sleep	99	wsrep aborter idle	NULL	0.000
+2	system user		NULL	Sleep	99	Waiting on cond	NULL	0.000
+3	system user		NULL	Sleep	63	wsrep applier committed	NULL	0.000
+4	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000
+5	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000
+6	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000
+7	system user		NULL	Daemon	NULL	InnoDB purge coordinator	NULL	0.000
+8	system user		NULL	Daemon	NULL	InnoDB shutdown handler	NULL	0.000
+95	root	localhost	test	Sleep	32		NULL	0.000
+96	root	localhost:42270	test	Query	0	Init	show full processlist	0.000
+97	system user		NULL	Sleep	30	wsrep applier idle	NULL	0.000
+98	system user		NULL	Sleep	30	wsrep applier idle	NULL	0.000
+99	system user		NULL	Sleep	30	wsrep applier committed	NULL	0.000
+100	system user		NULL	Sleep	30	wsrep applier committed	NULL	0.000
+101	system user		NULL	Sleep	30	wsrep applier committed	NULL	0.000
+102	system user		NULL	Sleep	30	wsrep applier idle	NULL	0.000
+103	system user		NULL	Sleep	30	wsrep applier idle	NULL	0.000
+104	system user		NULL	Sleep	30	wsrep applier committed	NULL	0.000
+105	system user		NULL	Sleep	30	wsrep applier committed	NULL	0.000
 SET GLOBAL wsrep_slave_threads = 10;
 # Wait 10 slave threads to start 2
 SET GLOBAL wsrep_slave_threads = 20;
@@ -25,11 +46,44 @@
 COUNT(*)
 53
 # Wait 10 slave threads to exit 3
+Timeout in wait_condition.inc for SELECT COUNT(*) = 2 FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER = 'system user' AND (STATE IS NULL OR STATE NOT LIKE 'InnoDB%');
+Id	User	Host	db	Command	Time	State	Info	Progress
+1	system user		NULL	Sleep	130	wsrep aborter idle	NULL	0.000
+2	system user		NULL	Sleep	130	Waiting on cond	NULL	0.000
+4	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000
+5	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000
+6	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000
+7	system user		NULL	Daemon	NULL	InnoDB purge coordinator	NULL	0.000
+8	system user		NULL	Daemon	NULL	InnoDB shutdown handler	NULL	0.000
+95	root	localhost	test	Sleep	63		NULL	0.000
+96	root	localhost:42270	test	Query	0	Init	show full processlist	0.000
+115	system user		NULL	Sleep	30	wsrep applier committed	NULL	0.000
 SET GLOBAL wsrep_slave_threads = 10;
 SET GLOBAL wsrep_slave_threads = 0;
 Warnings:
 Warning	1292	Truncated incorrect wsrep_slave_threads value: '0'
 # Wait 10 slave threads to start 3
+Timeout in wait_condition.inc for SELECT COUNT(*) = 11 FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER = 'system user' AND (STATE IS NULL OR STATE NOT LIKE 'InnoDB%');
+Id	User	Host	db	Command	Time	State	Info	Progress
+1	system user		NULL	Sleep	738	wsrep aborter idle	NULL	0.000
+2	system user		NULL	Sleep	738	Waiting on cond	NULL	0.000
+4	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000
+5	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000
+6	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000
+7	system user		NULL	Daemon	NULL	InnoDB purge coordinator	NULL	0.000
+8	system user		NULL	Daemon	NULL	InnoDB shutdown handler	NULL	0.000
+95	root	localhost	test	Sleep	671		NULL	0.000
+96	root	localhost:42270	test	Query	0	Init	show full processlist	0.000
+115	system user		NULL	Sleep	638	wsrep applier committed	NULL	0.000
+123	system user		NULL	Sleep	608	wsrep applier idle	NULL	0.000
+116	system user		NULL	Sleep	608	wsrep applier idle	NULL	0.000
+117	system user		NULL	Sleep	608	wsrep applier idle	NULL	0.000
+118	system user		NULL	Sleep	608	wsrep applier idle	NULL	0.000
+119	system user		NULL	Sleep	608	wsrep applier idle	NULL	0.000
+122	system user		NULL	Sleep	608	wsrep applier idle	NULL	0.000
+121	system user		NULL	Sleep	608	wsrep applier idle	NULL	0.000
+120	system user		NULL	Sleep	608	wsrep applier idle	NULL	0.000
+124	system user		NULL	Sleep	608	wsrep applier idle	NULL	0.000
 connection node_2;
 # Generate 12 replication events
 connection node_1;
@@ -37,5 +91,23 @@
 COUNT(*)
 65
 # Wait 10 slave threads to exit 4
+Timeout in wait_condition.inc for SELECT COUNT(*) = 2 FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER = 'system user' AND (STATE IS NULL OR STATE NOT LIKE 'InnoDB%');
+Id	User	Host	db	Command	Time	State	Info	Progress
+1	system user		NULL	Sleep	768	wsrep aborter idle	NULL	0.000
+2	system user		NULL	Sleep	768	Waiting on cond	NULL	0.000
+4	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000
+5	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000
+6	system user		NULL	Daemon	NULL	InnoDB purge worker	NULL	0.000
+7	system user		NULL	Daemon	NULL	InnoDB purge coordinator	NULL	0.000
+8	system user		NULL	Daemon	NULL	InnoDB shutdown handler	NULL	0.000
+95	root	localhost	test	Sleep	701		NULL	0.000
+96	root	localhost:42270	test	Query	0	Init	show full processlist	0.000
+117	system user		NULL	Sleep	30	wsrep applier committed	NULL	0.000
+118	system user		NULL	Sleep	30	wsrep applier committed	NULL	0.000
+119	system user		NULL	Sleep	30	wsrep applier committed	NULL	0.000
+122	system user		NULL	Sleep	638	wsrep applier idle	NULL	0.000
+121	system user		NULL	Sleep	30	wsrep applier committed	NULL	0.000
+120	system user		NULL	Sleep	638	wsrep applier idle	NULL	0.000
+124	system user		NULL	Sleep	30	wsrep applier committed	NULL	0.000
 connection node_1;
 DROP TABLE t1;
 
mysqltest: Result length mismatch

Comment by Marko Mäkelä [ 2019-03-06 ]

I had disabled galera.MW-336 in MariaDB 10.3.10 due to occasional failures, and recently re-enabled it after 10.3.13 in a merge. I think that the test or code may need to be improved in 10.1‥10.3 as well, in addition to fixing the deterministic failure on 10.4.

Comment by Marko Mäkelä [ 2019-03-08 ]

I disabled some more tests in 10.1 due to regular failures and plan to merge that all way up to 10.4.

galera.partition and galera.galera_binlog_stmt_autoinc regularly display mismatching values for AUTO_INCREMENT columns, possibly due to nondeterminism in the order in which the nodes are being started or selected for connections.

galera.MW-336 often times out while waiting for something in PROCESSLIST. This might be the same issue as with MDEV-17061.

Comment by Jan Lindström (Inactive) [ 2019-07-03 ]

Closing as we have a new MDEV-19937.

Generated at Thu Feb 08 08:06:27 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.