[MDEV-13738] galera_applier_ftwrl_table_alter may fail with '2006: MySQL server has gone away' Created: 2017-09-05  Updated: 2020-01-17  Resolved: 2018-09-13

Status: Closed
Project: MariaDB Server
Component/s: Galera, Tests
Affects Version/s: 10.1
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Andrii Nikitin (Inactive) Assignee: Jan Lindström (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Blocks
blocks MDEV-13549 Galera 3 test failures Closed
Relates
relates to MDEV-13549 Galera 3 test failures Closed
relates to MDEV-21525 galera.galera_ftwrl MTR failed: Crash... Closed

 Description   

I can reproduce the problem in at least two environments (both source distribution and hasky build) when running galera_applier_ftwrl_table_alter with few more tests, e.g.

./mtr --repeat=20 --suite=galera -par=12 --big-test galera_applier_ftwrl_table_alter galera_gtid  galera_applier_ftwrl_table query_cache

But with more tests it should be easier to reproduce, e.g.:

./mtr --repeat=20 --suite=galera -par=12 --big-test galera_applier_ftwrl_table_alter galera_applier_ftwrl_table galera_binlog_checksum galera_gtid galera_binlog_event_max_size_min galera_binlog_event_max_size_max

The failure usually occurs at start of run, so better just restart command if no error is shown within ~1 min
In my understanding one of nodes (or both) just may silently exit when first node executes :

FLUSH TABLE t1 WITH READ LOCK;
 
ALTER TABLE t1 ADD COLUMN f2 INTEGER;

Error logs of both nodes and system logs are just empty without messages past node initialization.

below are few examples:

[buildbot@centos73-amd64 mysql-test]$ ./mtr --repeat=20 --suite=galera -par=12 --big-test galera_applier_ftwrl_table_alter galera_gtid  galera_applier_ftwrl_table query_cache
 
Logging: ./mtr  --repeat=20 --suite=galera -par=12 --big-test galera_applier_ftwrl_table_alter galera_gtid galera_applier_ftwrl_table query_cache
 
vardir: /home/buildbot/mariadb-environs/_depot/m-tar/10.1-16005/mysql-test/var
 
Removing old var directory...
 
Creating var directory '/home/buildbot/mariadb-environs/_depot/m-tar/10.1-16005/mysql-test/var'...
 
Checking supported features...
 
MariaDB Version 10.1.27-MariaDB
 
 - SSL connections supported
 
Collecting tests...
 
Installing system database...
 
 
 
==============================================================================
 
 
 
TEST                                  WORKER RESULT   TIME (ms) or COMMENT
 
--------------------------------------------------------------------------
 
 
 
worker[2] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
 
worker[3] Using MTR_BUILD_THREAD 302, with reserved ports 16040..16059
 
worker[1] Using MTR_BUILD_THREAD 301, with reserved ports 16020..16039
 
worker[4] Using MTR_BUILD_THREAD 303, with reserved ports 16060..16079
 
worker[6] Using MTR_BUILD_THREAD 307, with reserved ports 16140..16159
 
worker[5] Using MTR_BUILD_THREAD 308, with reserved ports 16160..16179
 
worker[7] Using MTR_BUILD_THREAD 306, with reserved ports 16120..16139
 
worker[8] Using MTR_BUILD_THREAD 305, with reserved ports 16100..16119
 
worker[12] Using MTR_BUILD_THREAD 304, with reserved ports 16080..16099
 
worker[11] Using MTR_BUILD_THREAD 309, with reserved ports 16180..16199
 
worker[10] Using MTR_BUILD_THREAD 311, with reserved ports 16220..16239
 
worker[9] Using MTR_BUILD_THREAD 310, with reserved ports 16200..16219
 
galera.galera_gtid 'innodb_plugin'       w12 [ pass ]   5706
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w2 [ fail ]
 
        Test ended at 2017-09-05 08:28:51
 
 
 
CURRENT_TEST: galera.galera_applier_ftwrl_table_alter
 
mysqltest: At line 22: query 'ALTER TABLE t1 ADD COLUMN f2 INTEGER' failed: 2006: MySQL server has gone away
 
 
 
The result from queries just before the failure was:
 
SET SESSION wsrep_sync_wait = 0;
 
SET SESSION lock_wait_timeout = 60;
 
SET SESSION innodb_lock_wait_timeout=60;
 
SET SESSION wait_timeout=60;
 
CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB;
 
FLUSH TABLE t1 WITH READ LOCK;
 
ALTER TABLE t1 ADD COLUMN f2 INTEGER;
 
 
 
 - saving '/home/buildbot/mariadb-environs/_depot/m-tar/10.1-16005/mysql-test/var/2/log/galera.galera_applier_ftwrl_table_alter-innodb_plugin/' to '/home/buildbot/mariadb-environs/_depot/m-tar/10.1-16005/mysql-test/var/log/galera.galera_applier_ftwrl_table_alter-innodb_plugin/'
 
 
 
Only  1  of 8 completed.
 
--------------------------------------------------------------------------
 
The servers were restarted 0 times
 
Spent 5.706 of 39 seconds executing testcases
 
 
 
Failure: Failed 1/1 tests, 0.00% were successful.
 
 
 
Failing test(s): galera.galera_applier_ftwrl_table_alter
 
 
 
The log files in var/log may give you some hint of what went wrong.
 
 
 
If you want to report this error, please read first the documentation
 
at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html
 
 
 
mysql-test-run: *** ERROR: there were failing test cases


[buildbot@centos73-amd64 mysql-test]$ ./mtr --repeat=20 --suite=galera,mariabackup -par=12 --big-test galera_applier_ftwrl_table_alter galera_applier_ftwrl_table query_cache full_backup incremental_backup
 
Logging: /home/buildbot/mariadb-environs/_depot/m-branch/10.1/mysql-test/mysql-test-run.pl  --repeat=20 --suite=galera,mariabackup -par=12 --big-test galera_applier_ftwrl_table_alter galera_applier_ftwrl_table query_cache full_backup incremental_backup
 
vardir: /home/buildbot/mariadb-environs/m1-10.1/build/mysql-test/var
 
Checking leftover processes...
 
Removing old var directory...
 
Creating var directory '/home/buildbot/mariadb-environs/m1-10.1/build/mysql-test/var'...
 
Checking supported features...
 
MariaDB Version 10.1.27-MariaDB
 
 - SSL connections supported
 
Collecting tests...
 
Installing system database...
 
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
 
worker[3] Using MTR_BUILD_THREAD 301, with reserved ports 16020..16039
 
worker[2] Using MTR_BUILD_THREAD 302, with reserved ports 16040..16059
 
 
 
==============================================================================
 
 
 
TEST                                  WORKER RESULT   TIME (ms) or COMMENT
 
--------------------------------------------------------------------------
 
 
 
worker[7] Using MTR_BUILD_THREAD 303, with reserved ports 16060..16079
 
worker[5] Using MTR_BUILD_THREAD 305, with reserved ports 16100..16119
 
worker[8] Using MTR_BUILD_THREAD 306, with reserved ports 16120..16139
 
worker[6] Using MTR_BUILD_THREAD 304, with reserved ports 16080..16099
 
worker[12] Using MTR_BUILD_THREAD 308, with reserved ports 16160..16179
 
worker[9] Using MTR_BUILD_THREAD 309, with reserved ports 16180..16199
 
worker[10] Using MTR_BUILD_THREAD 307, with reserved ports 16140..16159
 
worker[11] Using MTR_BUILD_THREAD 310, with reserved ports 16200..16219
 
worker[4] Using MTR_BUILD_THREAD 311, with reserved ports 16220..16239
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w2 [ fail ]
 
        Test ended at 2017-09-05 08:02:05
 
 
 
CURRENT_TEST: galera.galera_applier_ftwrl_table_alter
 
mysqltest: At line 22: query 'ALTER TABLE t1 ADD COLUMN f2 INTEGER' failed: 2006: MySQL server has gone away
 
 
 
The result from queries just before the failure was:
 
SET SESSION wsrep_sync_wait = 0;
 
SET SESSION lock_wait_timeout = 60;
 
SET SESSION innodb_lock_wait_timeout=60;
 
SET SESSION wait_timeout=60;
 
CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB;
 
FLUSH TABLE t1 WITH READ LOCK;
 
ALTER TABLE t1 ADD COLUMN f2 INTEGER;
 
 
 
 - saving '/home/buildbot/mariadb-environs/m1-10.1/build/mysql-test/var/2/log/galera.galera_applier_ftwrl_table_alter-innodb_plugin/' to '/home/buildbot/mariadb-environs/m1-10.1/build/mysql-test/var/log/galera.galera_applier_ftwrl_table_alter-innodb_plugin/'
 
 
 
Only  1  of 8 completed.
 
--------------------------------------------------------------------------
 
The servers were restarted 0 times
 
Spent 0.000 of 58 seconds executing testcases
 
 
 
Failure: Failed 1/1 tests, 0.00% were successful.
 
 
 
Failing test(s): galera.galera_applier_ftwrl_table_alter
 
 
 
The log files in var/log may give you some hint of what went wrong.
 
 
 
If you want to report this error, please read first the documentation
 
at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html


[buildbot@centos73-amd64 mysql-test]$ ./mtr --repeat=20 --suite=galera,mariabackup -par=12 --big-test galera_applier_ftwrl_table_alter galera_applier_ftwrl_table query_cache 
Logging: /home/buildbot/mariadb-environs/_depot/m-branch/10.1/mysql-test/mysql-test-run.pl  --repeat=20 --suite=galera,mariabackup -par=12 --big-test galera_applier_ftwrl_table_alter galera_applier_ftwrl_table query_cache
 
vardir: /home/buildbot/mariadb-environs/m1-10.1/build/mysql-test/var
 
Checking leftover processes...
 
Removing old var directory...
 
Creating var directory '/home/buildbot/mariadb-environs/m1-10.1/build/mysql-test/var'...
 
Checking supported features...
 
MariaDB Version 10.1.27-MariaDB
 
 - SSL connections supported
 
Collecting tests...
 
Installing system database...
 
 
 
==============================================================================
 
 
 
TEST                                  WORKER RESULT   TIME (ms) or COMMENT
 
--------------------------------------------------------------------------
 
 
 
worker[2] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
 
worker[3] Using MTR_BUILD_THREAD 301, with reserved ports 16020..16039
 
worker[6] Using MTR_BUILD_THREAD 302, with reserved ports 16040..16059
 
worker[4] Using MTR_BUILD_THREAD 304, with reserved ports 16080..16099
 
worker[5] Using MTR_BUILD_THREAD 305, with reserved ports 16100..16119
 
worker[9] Using MTR_BUILD_THREAD 306, with reserved ports 16120..16139
 
worker[11] Using MTR_BUILD_THREAD 308, with reserved ports 16160..16179
 
worker[10] Using MTR_BUILD_THREAD 309, with reserved ports 16180..16199
 
worker[8] Using MTR_BUILD_THREAD 307, with reserved ports 16140..16159
 
worker[7] Using MTR_BUILD_THREAD 311, with reserved ports 16220..16239
 
worker[12] Using MTR_BUILD_THREAD 310, with reserved ports 16200..16219
 
worker[1] Using MTR_BUILD_THREAD 303, with reserved ports 16060..16079
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   5418
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   5562
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   3676
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   4033
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w11 [ pass ]  13537
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w12 [ pass ]  14115
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   3628
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   4102
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   3342
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   2804
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   2815
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   2797
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w12 [ pass ]  10575
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w11 [ pass ]  11173
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   5928
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   7349
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   2900
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   2912
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w12 [ pass ]  11252
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w11 [ pass ]  11151
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   3632
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   3689
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   4900
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   5735
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   3351
 
galera.query_cache 'innodb_plugin'       w10 [ pass ]  47802
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   4386
 
galera.query_cache 'xtradb'              w8 [ pass ]  48757
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w12 [ pass ]  13349
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w11 [ pass ]  11650
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   3631
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   3625
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   4924
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   5746
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w12 [ pass ]  11077
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w11 [ pass ]  12937
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   4931
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   6602
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   3118
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   3263
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   3036
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   2840
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w12 [ pass ]  12951
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w11 [ pass ]  11414
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   3266
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   3251
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   5766
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   6450
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w12 [ pass ]  11248
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w11 [ pass ]  13903
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   5273
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   5559
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   3652
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   3553
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w12 [ pass ]  10718
 
galera.galera_applier_ftwrl_table 'xtradb' w2 [ pass ]   4003
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w3 [ pass ]   5720
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w11 [ pass ]  12914
 
galera.query_cache 'innodb_plugin'       w10 [ pass ]  60801
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w12 [ pass ]  13972
 
galera.query_cache 'xtradb'              w8 [ pass ]  63959
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w11 [ pass ]  13227
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w12 [ pass ]  11395
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w11 [ pass ]  10164
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w12 [ pass ]   9069
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w11 [ pass ]  13942
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w12 [ pass ]  14791
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w11 [ pass ]  12137
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w12 [ pass ]  14208
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w12 [ fail ]
 
        Test ended at 2017-09-05 08:09:23
 
 
 
CURRENT_TEST: galera.galera_applier_ftwrl_table_alter
 
mysqltest: At line 22: query 'ALTER TABLE t1 ADD COLUMN f2 INTEGER' failed: 2006: MySQL server has gone away
 
 
 
The result from queries just before the failure was:
 
SET SESSION wsrep_sync_wait = 0;
 
SET SESSION lock_wait_timeout = 60;
 
SET SESSION innodb_lock_wait_timeout=60;
 
SET SESSION wait_timeout=60;
 
CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB;
 
FLUSH TABLE t1 WITH READ LOCK;
 
ALTER TABLE t1 ADD COLUMN f2 INTEGER;
 
 
 
 - saving '/home/buildbot/mariadb-environs/m1-10.1/build/mysql-test/var/12/log/galera.galera_applier_ftwrl_table_alter-xtradb/' to '/home/buildbot/mariadb-environs/m1-10.1/build/mysql-test/var/log/galera.galera_applier_ftwrl_table_alter-xtradb/'
 
 
 
Only  3  of 6 completed.
 
--------------------------------------------------------------------------
 
The servers were restarted 0 times
 
Spent 699.356 of 192 seconds executing testcases
 
 
 
Failure: Failed 1/54 tests, 98.15% were successful.
 
 
 
Failing test(s): galera.galera_applier_ftwrl_table_alter
 
 
 
The log files in var/log may give you some hint of what went wrong.
 
 
 
If you want to report this error, please read first the documentation
 
at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html
 
 
 
mysql-test-run: *** ERROR: there were failing test cases


[buildbot@centos73-amd64 mysql-test]$ ./mtr --repeat=20 --suite=galera -par=12 --big-test galera_applier_ftwrl_table_alter galera_gtid  galera_applier_ftwrl_table query_cache
 
Logging: ./mtr  --repeat=20 --suite=galera -par=12 --big-test galera_applier_ftwrl_table_alter galera_gtid galera_applier_ftwrl_table query_cache
 
vardir: /home/buildbot/mariadb-environs/_depot/m-tar/10.1-16005/mysql-test/var
 
Checking leftover processes...
 
Removing old var directory...
 
Creating var directory '/home/buildbot/mariadb-environs/_depot/m-tar/10.1-16005/mysql-test/var'...
 
Checking supported features...
 
MariaDB Version 10.1.27-MariaDB
 
 - SSL connections supported
 
Collecting tests...
 
Installing system database...
 
 
 
==============================================================================
 
 
 
TEST                                  WORKER RESULT   TIME (ms) or COMMENT
 
--------------------------------------------------------------------------
 
 
 
worker[2] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
 
worker[1] Using MTR_BUILD_THREAD 301, with reserved ports 16020..16039
 
worker[3] Using MTR_BUILD_THREAD 302, with reserved ports 16040..16059
 
worker[4] Using MTR_BUILD_THREAD 308, with reserved ports 16160..16179
 
worker[8] Using MTR_BUILD_THREAD 309, with reserved ports 16180..16199
 
worker[5] Using MTR_BUILD_THREAD 307, with reserved ports 16140..16159
 
worker[11] Using MTR_BUILD_THREAD 305, with reserved ports 16100..16119
 
worker[9] Using MTR_BUILD_THREAD 310, with reserved ports 16200..16219
 
worker[10] Using MTR_BUILD_THREAD 304, with reserved ports 16080..16099
 
worker[6] Using MTR_BUILD_THREAD 306, with reserved ports 16120..16139
 
worker[12] Using MTR_BUILD_THREAD 311, with reserved ports 16220..16239
 
worker[7] Using MTR_BUILD_THREAD 303, with reserved ports 16060..16079
 
galera.galera_gtid 'xtradb'              w3 [ pass ]   5738
 
galera.galera_applier_ftwrl_table 'xtradb' w12 [ pass ]   6056
 
galera.galera_gtid 'innodb_plugin'       w11 [ pass ]   6168
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w2 [ pass ]   6302
 
galera.galera_gtid 'xtradb'              w3 [ pass ]   3914
 
galera.galera_gtid 'innodb_plugin'       w11 [ pass ]   4208
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w2 [ pass ]   5179
 
galera.galera_applier_ftwrl_table 'xtradb' w12 [ pass ]   5983
 
galera.galera_gtid 'innodb_plugin'       w11 [ pass ]   5377
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w1 [ pass ]  18062
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w4 [ pass ]  17659
 
galera.galera_gtid 'xtradb'              w3 [ pass ]   6412
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w2 [ pass ]   6922
 
galera.galera_applier_ftwrl_table 'xtradb' w12 [ pass ]   4472
 
galera.galera_gtid 'innodb_plugin'       w11 [ pass ]   4015
 
galera.galera_gtid 'xtradb'              w3 [ pass ]   4363
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w2 [ pass ]   5035
 
galera.galera_applier_ftwrl_table 'xtradb' w12 [ pass ]   4895
 
galera.galera_gtid 'innodb_plugin'       w11 [ pass ]   3001
 
galera.galera_gtid 'xtradb'              w3 [ pass ]   3537
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w2 [ pass ]   2877
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w1 [ pass ]  11911
 
galera.galera_gtid 'innodb_plugin'       w11 [ pass ]   3010
 
galera.galera_gtid 'xtradb'              w3 [ pass ]   5912
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w2 [ pass ]   4550
 
galera.galera_applier_ftwrl_table 'xtradb' w12 [ pass ]   6709
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w4 [ pass ]  15613
 
galera.galera_gtid 'innodb_plugin'       w11 [ pass ]   3558
 
galera.galera_applier_ftwrl_table 'xtradb' w12 [ pass ]   6374
 
galera.galera_gtid 'xtradb'              w3 [ pass ]   5029
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w2 [ pass ]   6475
 
galera.galera_gtid 'innodb_plugin'       w11 [ pass ]   5180
 
galera.galera_gtid 'xtradb'              w3 [ pass ]   3560
 
galera.galera_gtid 'innodb_plugin'       w11 [ pass ]   3044
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w2 [ pass ]   4157
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w1 [ pass ]  15167
 
galera.galera_applier_ftwrl_table 'xtradb' w12 [ pass ]   6984
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w4 [ pass ]  13173
 
galera.galera_gtid 'xtradb'              w3 [ pass ]   4159
 
galera.galera_gtid 'innodb_plugin'       w11 [ pass ]   4027
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w2 [ pass ]   4957
 
galera.galera_applier_ftwrl_table 'xtradb' w12 [ pass ]   6540
 
galera.galera_gtid 'xtradb'              w3 [ pass ]   4595
 
galera.galera_gtid 'innodb_plugin'       w11 [ pass ]   3911
 
galera.galera_gtid 'xtradb'              w3 [ pass ]   4975
 
galera.galera_gtid 'innodb_plugin'       w11 [ pass ]   5515
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w1 [ pass ]  13599
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w2 [ pass ]   6454
 
galera.galera_applier_ftwrl_table 'xtradb' w12 [ pass ]   7080
 
galera.query_cache 'innodb_plugin'       w5 [ pass ]  66110
 
galera.galera_applier_ftwrl_table_alter 'innodb_plugin' w4 [ pass ]  15131
 
galera.galera_gtid 'xtradb'              w3 [ pass ]   4430
 
galera.galera_gtid 'innodb_plugin'       w11 [ pass ]   4256
 
galera.query_cache 'xtradb'              w8 [ pass ]  71096
 
galera.galera_applier_ftwrl_table 'innodb_plugin' w2 [ pass ]   5608
 
galera.galera_applier_ftwrl_table 'xtradb' w12 [ pass ]   6170
 
galera.galera_applier_ftwrl_table_alter 'xtradb' w1 [ fail ]
 
        Test ended at 2017-09-05 08:32:01
 
 
 
CURRENT_TEST: galera.galera_applier_ftwrl_table_alter
 
mysqltest: At line 22: query 'ALTER TABLE t1 ADD COLUMN f2 INTEGER' failed: 2006: MySQL server has gone away
 
 
 
The result from queries just before the failure was:
 
SET SESSION wsrep_sync_wait = 0;
 
SET SESSION lock_wait_timeout = 60;
 
SET SESSION innodb_lock_wait_timeout=60;
 
SET SESSION wait_timeout=60;
 
CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB;
 
FLUSH TABLE t1 WITH READ LOCK;
 
ALTER TABLE t1 ADD COLUMN f2 INTEGER;
 
 
 
 - saving '/home/buildbot/mariadb-environs/_depot/m-tar/10.1-16005/mysql-test/var/1/log/galera.galera_applier_ftwrl_table_alter-xtradb/' to '/home/buildbot/mariadb-environs/_depot/m-tar/10.1-16005/mysql-test/var/log/galera.galera_applier_ftwrl_table_alter-xtradb/'
 
 
 
Only  1  of 8 completed.
 
--------------------------------------------------------------------------
 
The servers were restarted 0 times
 
Spent 489.194 of 100 seconds executing testcases
 
 
 
Failure: Failed 1/5 tests, 80.00% were successful.
 
 
 
Failing test(s): galera.galera_applier_ftwrl_table_alter
 
 
 
The log files in var/log may give you some hint of what went wrong.
 
 
 
If you want to report this error, please read first the documentation
 
at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html
 
 
 
mysql-test-run: *** ERROR: there were failing test cases



 Comments   
Comment by Andrii Nikitin (Inactive) [ 2017-09-05 ]

Addition important note is that I was able to reproduce problem only when var is regular folder (i.e. not mounted to /dev/shm)

I've reproduced the problem in upstream as well mysql-wsrep-bugs

You may need to add galera_log_output_csv and galera_log_bin to disabled.def because they have problems with --repeat (which will be solved as part of other task)

a@UBINTI:~/mysql-wsrep-bugs-bld/mysql-test$ tail -n 4 ~/mysql-wsrep-bugs/mysql-test/suite/galera/t/disabled.def 
galera_toi_ddl_fk_insert : qa#39 galera_toi_ddl_fk_insert fails sporadically
 
galera_sst_xtrabackup-v2-options : SST Encryption does not work with xtrabackup 2.4.2
 
galera_log_output_csv : disabled
 
galera_log_bin : disabled
 
a@UBINTI:~/mysql-wsrep-bugs-bld/mysql-test$ ./mtr --repeat=20 --suite=galera -par=12 --big-test
 
Logging: /home/a/mysql-wsrep-bugs/mysql-test/mysql-test-run.pl  --repeat=20 --suite=galera -par=12 --big-test
 
2017-09-05 11:52:15 0 [Warning] Insecure configuration for --secure-file-priv: Current value does not restrict location of generated files. Consider setting it to a valid, non-empty path.
 
2017-09-05 11:52:15 0 [Note] /home/a/mysql-wsrep-bugs-bld/sql/mysqld (mysqld 5.6.37) starting as process 10413 ...
 
2017-09-05 11:52:15 10413 [Warning] Buffered warning: Changed limits: max_open_files: 1024 (requested 5000)
 
 
 
2017-09-05 11:52:15 10413 [Warning] Buffered warning: Changed limits: table_open_cache: 431 (requested 2000)
 
 
 
2017-09-05 11:52:15 10413 [Note] Plugin 'FEDERATED' is disabled.
 
2017-09-05 11:52:15 10413 [Note] Binlog end
 
2017-09-05 11:52:15 10413 [Note] Shutting down plugin 'MyISAM'
 
2017-09-05 11:52:15 10413 [Note] Shutting down plugin 'CSV'
 
MySQL Version 5.6.37
 
Checking supported features...
 
 - SSL connections supported
 
 - binaries built with wsrep patch
 
Using suites: galera
 
Collecting tests...
 
Checking leftover processes...
 
Removing old var directory...
 
Creating var directory '/home/a/mysql-wsrep-bugs-bld/mysql-test/var'...
 
Installing system database...
 
Using parallel: 12
 
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009
 
 
 
==============================================================================
 
 
 
TEST                                  WORKER RESULT   TIME (ms) or COMMENT
 
--------------------------------------------------------------------------
 
 
 
worker[3] Using MTR_BUILD_THREAD 304, with reserved ports 13040..13049
 
worker[5] Using MTR_BUILD_THREAD 302, with reserved ports 13020..13029
 
worker[6] Using MTR_BUILD_THREAD 305, with reserved ports 13050..13059
 
worker[7] Using MTR_BUILD_THREAD 308, with reserved ports 13080..13089
 
worker[4] Using MTR_BUILD_THREAD 310, with reserved ports 13100..13109
 
worker[2] Using MTR_BUILD_THREAD 309, with reserved ports 13090..13099
 
worker[9] Using MTR_BUILD_THREAD 312, with reserved ports 13120..13129
 
worker[11] Using MTR_BUILD_THREAD 313, with reserved ports 13130..13139
 
worker[10] Using MTR_BUILD_THREAD 311, with reserved ports 13110..13119
 
worker[12] Using MTR_BUILD_THREAD 314, with reserved ports 13140..13149
 
worker[8] Using MTR_BUILD_THREAD 315, with reserved ports 13150..13159
 
galera.galera_toi_ddl_locking            [ skipped ]  Test needs debug binaries
 
galera.galera_bf_abort_for_update        [ disabled ]  mysql-wsrep#26 SELECT FOR UPDATE sometimes allowed to proceed in the face of a concurrent update
 
galera.galera_kill_nochanges             [ disabled ]  mysql-wsrep#24 Galera server does not restart properly if killed
 
galera.galera_log_bin                    [ disabled ]  xx
 
galera.galera_log_output_csv             [ disabled ]  xx
 
galera.galera_sst_xtrabackup-v2-options  [ disabled ]  SST Encryption does not work with xtrabackup 2.4.2
 
galera.galera_toi_ddl_fk_insert          [ disabled ]  qa#39 galera_toi_ddl_fk_insert fails sporadically
 
galera.galera_wsrep_provider_unset_set   [ disabled ]  lp1379204 'Unsupported protocol downgrade: incremental data collection disabled. Expect abort.'
 
galera.MW-86-wait1                       w12 [ skipped ]  Test requires: 'have_debug_sync'
 
galera.galera_binlog_event_max_size_min  w5 [ pass ]   3907
 
galera.MW-86-wait8                       w9 [ skipped ]  Test requires: 'have_debug_sync'
 
galera.galera_v1_row_events              w8 [ pass ]   3900
 
galera.galera_applier_ftwrl_table_alter  w10 [ fail ]
 
        Test ended at 2017-09-05 11:53:07
 
 
 
CURRENT_TEST: galera.galera_applier_ftwrl_table_alter
 
mysqltest: At line 22: query 'ALTER TABLE t1 ADD COLUMN f2 INTEGER' failed: 2013: Lost connection to MySQL server during query
 
 
 
The result from queries just before the failure was:
 
SET SESSION wsrep_sync_wait = 0;
 
SET SESSION lock_wait_timeout = 60;
 
SET SESSION innodb_lock_wait_timeout=60;
 
SET SESSION wait_timeout=60;
 
CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB;
 
FLUSH TABLE t1 WITH READ LOCK;
 
 
 
 - saving '/home/a/mysql-wsrep-bugs-bld/mysql-test/var/10/log/galera.galera_applier_ftwrl_table_alter/' to '/home/a/mysql-wsrep-bugs-bld/mysql-test/var/log/galera.galera_applier_ftwrl_table_alter/'
 
 
 
Only  11  of 240 completed.
 
mysql-test-run: *** ERROR: Not all tests completed

Comment by Jan Lindström (Inactive) [ 2018-09-13 ]

Not repeatable lately based on cross reference search.

Generated at Thu Feb 08 08:07:58 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.