[MDEV-16008] galera mtr test galera_many_tables_nopk crashed mysqld signal 6 Created: 2018-04-24  Updated: 2018-09-11  Resolved: 2018-09-11

Status: Closed
Project: MariaDB Server
Component/s: Galera, Tests
Affects Version/s: 10.3.6
Fix Version/s: 10.3.7

Type: Bug Priority: Major
Reporter: Zdravelina Sokolovska (Inactive) Assignee: Jan Lindström (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Blocks
blocks MDEV-13549 Galera 3 test failures Closed

 Description   

galera test galera_many_tables_nopk from mtr galera test suite
crashed mysqld got signal 6

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x49000
/home/qa-user1/rr/build-mariadb/sql/mysqld(my_print_stacktrace+0x29)[0x5593f3803dd9]
/home/qa-user1/rr/build-mariadb/sql/mysqld(handle_fatal_signal+0x33f)[0x5593f333e53f]
2018-04-24 16:22:50 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection to peer 00000000 with addr tcp://127.0.0.1:16002 timed out, no messages seen in PT3S
/lib64/libpthread.so.0(+0xf5e0)[0x7fccdea625e0]
/lib64/libc.so.6(__poll+0x2d)[0x7fccdd037a3d]
/home/qa-user1/rr/build-mariadb/sql/mysqld(_Z26handle_connections_socketsv+0x20b)[0x5593f30e157b]
2018-04-24 16:23:43 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection to peer 00000000 with addr  timed out, no messages seen in PT3S
/home/qa-user1/rr/build-mariadb/sql/mysqld(_Z11mysqld_mainiPPc+0x2b79)[0x5593f30e4759]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7fccdcf6bc05]
/home/qa-user1/rr/build-mariadb/sql/mysqld(+0x4e2398)[0x5593f30d8398]

./mysql-test-run.pl --suite=galera --force --big-test --max-test-fail=0

2018-04-24 16:02:46 0 [Note] WSREP: Read nil XID from storage engines, skipping position init
2018-04-24 16:02:46 0 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera/libgalera_smm.so'
2018-04-24 16:02:46 0 [Note] WSREP: wsrep_load(): Galera 25.3.22(r3764) by Codership Oy <info@codership.com> loaded successfully.
2018-04-24 16:02:46 0 [Note] WSREP: CRC-32C: using "slicing-by-8" algorithm.
2018-04-24 16:02:46 0 [Note] WSREP: Found saved state: 5707f719-47bf-11e8-a2b8-4683ac5e41e7:-1, safe_to_bootstrap: 0
2018-04-24 16:02:46 0 [Note] WSREP: Passing config to GCS: base_dir = /home/qa-user1/rr/build-mariadb/mysql-test/var/mysqld.2/data/; base_host = 127.0.0.1; base_port = 16005; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT30S; evs.install_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT10S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /home/qa-user1/rr/build-mariadb/mysql-test/var/mysqld.2/data/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /home/qa-user1/rr/build-mariadb/mysql-test/var/mysqld.2/data//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit =
2018-04-24 16:02:46 0 [Note] WSREP: GCache history reset: 5707f719-47bf-11e8-a2b8-4683ac5e41e7:0 -> 5707f719-47bf-11e8-a2b8-4683ac5e41e7:-1
2018-04-24 16:02:46 0 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
2018-04-24 16:02:46 0 [Note] WSREP: wsrep_sst_grab()
2018-04-24 16:02:46 0 [Note] WSREP: Start replication
2018-04-24 16:02:46 0 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
2018-04-24 16:02:46 0 [Note] WSREP: protonet asio version 0
2018-04-24 16:02:46 0 [Note] WSREP: Using CRC-32C for message checksums.
2018-04-24 16:02:46 0 [Note] WSREP: backend: asio
2018-04-24 16:02:46 0 [Note] WSREP: gcomm thread scheduling priority set to other:0
2018-04-24 16:02:46 0 [Note] WSREP: restore pc from disk successfully
2018-04-24 16:02:46 0 [Note] WSREP: GMCast version 0
2018-04-24 16:02:46 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') listening at tcp://0.0.0.0:16005
2018-04-24 16:02:46 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') multicast: , ttl: 1
2018-04-24 16:02:46 0 [Note] WSREP: EVS version 0
2018-04-24 16:02:46 0 [Note] WSREP: gcomm: connecting to group 'my_wsrep_cluster', peer '127.0.0.1:16002'
2018-04-24 16:02:46 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection established to 57076020 tcp://127.0.0.1:16002
2018-04-24 16:02:46 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') turning message relay requesting on, nonlive peers:
2018-04-24 16:02:47 0 [Note] WSREP: gcomm: connected
2018-04-24 16:02:47 0 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
2018-04-24 16:02:47 0 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
2018-04-24 16:02:47 0 [Note] WSREP: Opened channel 'my_wsrep_cluster'
2018-04-24 16:02:47 0 [Note] WSREP: Waiting for SST to complete.
2018-04-24 16:02:47 0 [Note] WSREP: declaring 57076020 at tcp://127.0.0.1:16002 stable
2018-04-24 16:02:47 0 [Note] WSREP: Node 57076020 state prim
2018-04-24 16:02:47 0 [Note] WSREP: view(view_id(PRIM,57076020,4) memb {
        57076020,0
        570d5bae,0
} joined {
} left {
} partitioned {
})
2018-04-24 16:02:47 0 [Note] WSREP: save pc into disk
2018-04-24 16:02:47 0 [Note] WSREP: clear restored view
2018-04-24 16:02:47 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2
2018-04-24 16:02:47 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
2018-04-24 16:02:47 0 [Note] WSREP: STATE EXCHANGE: sent state msg: c933e8b0-47bf-11e8-937f-9a766d2a47b3
2018-04-24 16:02:47 0 [Note] WSREP: STATE EXCHANGE: got state msg: c933e8b0-47bf-11e8-937f-9a766d2a47b3 from 0 (t4w3.xentio.lan)
2018-04-24 16:02:47 0 [Note] WSREP: STATE EXCHANGE: got state msg: c933e8b0-47bf-11e8-937f-9a766d2a47b3 from 1 (t4w3.xentio.lan)
2018-04-24 16:02:47 0 [Note] WSREP: Quorum results:
        version    = 4,
        component  = PRIMARY,
        conf_id    = 3,
        members    = 1/2 (joined/total),
        act_id     = 1190,
        last_appl. = -1,
        protocols  = 0/7/3 (gcs/repl/appl),
        group UUID = 5707f719-47bf-11e8-a2b8-4683ac5e41e7
2018-04-24 16:02:47 0 [Note] WSREP: Flow-control interval: [23, 23]
2018-04-24 16:02:47 0 [Note] WSREP: Trying to continue unpaused monitor
2018-04-24 16:02:47 0 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 1190)
2018-04-24 16:02:47 1 [Note] WSREP: State transfer required:
        Group state: 5707f719-47bf-11e8-a2b8-4683ac5e41e7:1190
        Local state: 5707f719-47bf-11e8-a2b8-4683ac5e41e7:-1
2018-04-24 16:02:47 1 [Note] WSREP: New cluster view: global state: 5707f719-47bf-11e8-a2b8-4683ac5e41e7:1190, view# 4: Primary, number of nodes: 2, my index: 1, protocol version 3
2018-04-24 16:02:47 1 [Warning] WSREP: Gap in state sequence. Need state transfer.
2018-04-24 16:02:47 0 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'joiner' --address '127.0.0.1:16007' --datadir '/home/qa-user1/rr/build-mariadb/mysql-test/var/mysqld.2/data/'  --defaults-file '/home/qa-user1/rr/build-mariadb/mysql-test/var/my.cnf' --defaults-group-suffix '.2'  --parent '13196'  '' '
2018-04-24 16:02:48 1 [Note] WSREP: Prepared SST request: rsync|127.0.0.1:16007/rsync_sst
2018-04-24 16:02:48 1 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2018-04-24 16:02:48 1 [Note] WSREP: REPL Protocols: 7 (3, 2)
2018-04-24 16:02:48 1 [Note] WSREP: Assign initial position for certification: 1190, protocol version: 3
2018-04-24 16:02:48 0 [Note] WSREP: Service thread queue flushed.
2018-04-24 16:02:48 1 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state seqno is undefined: 1 (Operation not permitted)
         at galera/src/replicator_str.cpp:prepare_for_IST():491. IST will be unavailable.
2018-04-24 16:02:48 0 [Note] WSREP: Member 1.0 (t4w3.xentio.lan) requested state transfer from '*any*'. Selected 0.0 (t4w3.xentio.lan)(SYNCED) as donor.
2018-04-24 16:02:48 0 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 1190)
2018-04-24 16:02:48 1 [Note] WSREP: Requesting state transfer: success, donor: 0
2018-04-24 16:02:48 1 [Note] WSREP: GCache history reset: 5707f719-47bf-11e8-a2b8-4683ac5e41e7:0 -> 5707f719-47bf-11e8-a2b8-4683ac5e41e7:1190
2018-04-24 16:02:50 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') turning message relay requesting off
2018-04-24 16:02:56 0 [Note] WSREP: 0.0 (t4w3.xentio.lan): State transfer to 1.0 (t4w3.xentio.lan) complete.
2018-04-24 16:02:56 0 [Note] WSREP: Member 0.0 (t4w3.xentio.lan) synced with group.
WSREP_SST: [INFO] Joiner cleanup. rsync PID: 13239 (20180424 16:02:56.602)
WSREP_SST: [INFO] Joiner cleanup done. (20180424 16:02:57.123)
2018-04-24 16:02:57 0 [Note] WSREP: SST complete, seqno: 1190
2018-04-24 16:02:57 0 [Note] Plugin 'partition' is disabled.
2018-04-24 16:02:57 0 [Note] InnoDB: For Galera, using innodb_lock_schedule_algorithm=fcfs
2018-04-24 16:02:57 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-04-24 16:02:57 0 [Note] InnoDB: Uses event mutexes
2018-04-24 16:02:57 0 [Note] InnoDB: Compressed tables use zlib 1.2.7
2018-04-24 16:02:57 0 [Note] InnoDB: Using Linux native AIO
2018-04-24 16:02:57 0 [Note] InnoDB: Number of pools: 1
2018-04-24 16:02:57 0 [Note] InnoDB: Using generic crc32 instructions
2018-04-24 16:02:57 0 [Note] InnoDB: Initializing buffer pool, total size = 8M, instances = 1, chunk size = 8M
2018-04-24 16:02:57 0 [Note] InnoDB: Completed initialization of buffer pool
2018-04-24 16:02:57 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2018-04-24 16:02:57 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=208888189
2018-04-24 16:02:57 0 [Note] InnoDB: Starting final batch to recover 351 pages from redo log.
2018-04-24 16:02:57 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
2018-04-24 16:02:57 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2018-04-24 16:02:57 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2018-04-24 16:02:57 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2018-04-24 16:02:57 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2018-04-24 16:02:57 0 [Note] InnoDB: Waiting for purge to start
2018-04-24 16:02:57 0 [Note] InnoDB: 5.7.21 started; log sequence number 209540076; transaction id 2622
2018-04-24 16:02:57 0 [Note] Plugin 'SEQUENCE' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_SYS_DATAFILES' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_SYS_TABLESTATS' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_MUTEXES' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_CMP' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_FT_DELETED' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_CMP_RESET' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_TABLESPACES_ENCRYPTION' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_CMPMEM_RESET' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'FEEDBACK' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_FT_INDEX_TABLE' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_CMP_PER_INDEX_RESET' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'user_variables' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_FT_INDEX_CACHE' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_FT_BEING_DELETED' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_FT_DEFAULT_STOPWORD' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_FT_CONFIG' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_SYS_TABLESPACES' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_TABLESPACES_SCRUBBING' is disabled.
2018-04-24 16:02:57 0 [Note] Plugin 'INNODB_SYS_SEMAPHORE_WAITS' is disabled.
2018-04-24 16:02:57 0 [Warning] /home/qa-user1/rr/build-mariadb/sql/mysqld: unknown variable 'loose-feedback-debug-startup-interval=20'
2018-04-24 16:02:57 0 [Warning] /home/qa-user1/rr/build-mariadb/sql/mysqld: unknown variable 'loose-feedback-debug-first-interval=60'
2018-04-24 16:02:57 0 [Warning] /home/qa-user1/rr/build-mariadb/sql/mysqld: unknown variable 'loose-feedback-debug-interval=60'
2018-04-24 16:02:57 0 [Warning] /home/qa-user1/rr/build-mariadb/sql/mysqld: unknown option '--loose-pam-debug'
2018-04-24 16:02:57 0 [Warning] /home/qa-user1/rr/build-mariadb/sql/mysqld: unknown variable 'loose-debug-sync-timeout=300'
2018-04-24 16:02:57 0 [Note] Recovering after a crash using tc.log
2018-04-24 16:02:57 0 [Note] Starting crash recovery...
2018-04-24 16:02:57 0 [Note] Crash recovery finished.
2018-04-24 16:02:57 0 [Note] InnoDB: Loading buffer pool(s) from /home/qa-user1/rr/build-mariadb/mysql-test/var/mysqld.2/data/ib_buffer_pool
2018-04-24 16:02:57 0 [Note] Server socket created on IP: '127.0.0.1'.
2018-04-24 16:02:57 0 [Note] WSREP: Signalling provider to continue.
2018-04-24 16:02:59 0 [Note] InnoDB: Buffer pool(s) load completed at 180424 16:02:59
2018-04-24 16:02:59 0 [Note] WSREP: SST received: 5707f719-47bf-11e8-a2b8-4683ac5e41e7:1190
2018-04-24 16:02:59 0 [Note] WSREP: 1.0 (t4w3.xentio.lan): State transfer from 0.0 (t4w3.xentio.lan) complete.
2018-04-24 16:02:59 0 [Note] WSREP: Shifting JOINER -> JOINED (TO: 1190)
2018-04-24 16:02:59 0 [Note] WSREP: Member 1.0 (t4w3.xentio.lan) synced with group.
2018-04-24 16:02:59 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 1190)
2018-04-24 16:02:59 1 [Note] WSREP: Synchronized with group, ready for connections
2018-04-24 16:02:59 1 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2018-04-24 16:02:59 0 [Note] Reading of all Master_info entries succeded
2018-04-24 16:02:59 0 [Note] Added new Master_info '' to hash table
2018-04-24 16:02:59 0 [Note] /home/qa-user1/rr/build-mariadb/sql/mysqld: ready for connections.
Version: '10.3.7-MariaDB-log'  socket: '/home/qa-user1/rr/build-mariadb/mysql-test/var/tmp/mysqld.2.sock'  port: 16001  Source distribution
CURRENT_TEST: galera.galera_many_columns
CURRENT_TEST: galera.galera_many_rows
CURRENT_TEST: galera.galera_many_tables_nopk
2018-04-24 16:18:05 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.97725S), skipping check
2018-04-24 16:18:05 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection to peer 57076020 with addr tcp://127.0.0.1:16002 timed out, no messages seen in PT3S
2018-04-24 16:18:06 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') turning message relay requesting on, nonlive peers: tcp://127.0.0.1:16002
2018-04-24 16:18:08 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT2.12966S), skipping check
2018-04-24 16:18:09 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') reconnecting to 57076020 (tcp://127.0.0.1:16002), attempt 0
2018-04-24 16:18:11 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT5.06177S), skipping check
2018-04-24 16:18:11 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection established to 57076020 tcp://127.0.0.1:16002
2018-04-24 16:18:11 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection established to 57076020 tcp://127.0.0.1:16002
2018-04-24 16:18:15 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') turning message relay requesting off
2018-04-24 16:18:16 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.92939S), skipping check
2018-04-24 16:18:17 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection to peer 57076020 with addr tcp://127.0.0.1:16002 timed out, no messages seen in PT3S
2018-04-24 16:18:18 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') turning message relay requesting on, nonlive peers: tcp://127.0.0.1:16002
2018-04-24 16:18:18 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT3.47293S), skipping check
2018-04-24 16:18:19 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') reconnecting to 57076020 (tcp://127.0.0.1:16002), attempt 0
2018-04-24 16:18:20 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.77353S), skipping check
2018-04-24 16:18:20 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection established to 57076020 tcp://127.0.0.1:16002
2018-04-24 16:18:20 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection established to 57076020 tcp://127.0.0.1:16002
2018-04-24 16:18:23 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.5754S), skipping check
2018-04-24 16:18:24 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') turning message relay requesting off
2018-04-24 16:18:25 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT2.27279S), skipping check
2018-04-24 16:18:27 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.91676S), skipping check
2018-04-24 16:18:29 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.58083S), skipping check
2018-04-24 16:18:31 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection to peer 57076020 with addr tcp://127.0.0.1:16002 timed out, no messages seen in PT3S
2018-04-24 16:18:32 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') turning message relay requesting on, nonlive peers: tcp://127.0.0.1:16002
2018-04-24 16:18:32 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT2.99903S), skipping check
2018-04-24 16:18:33 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') reconnecting to 57076020 (tcp://127.0.0.1:16002), attempt 0
2018-04-24 16:18:33 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection established to 57076020 tcp://127.0.0.1:16002
2018-04-24 16:18:33 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection established to 57076020 tcp://127.0.0.1:16002
2018-04-24 16:18:37 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT3.26624S), skipping check
2018-04-24 16:18:38 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection to peer 57076020 with addr tcp://127.0.0.1:16002 timed out, no messages seen in PT3S
2018-04-24 16:18:38 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.82026S), skipping check
2018-04-24 16:18:40 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') reconnecting to 57076020 (tcp://127.0.0.1:16002), attempt 0
2018-04-24 16:18:42 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT3.44483S), skipping check
2018-04-24 16:18:42 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection established to 57076020 tcp://127.0.0.1:16002
2018-04-24 16:18:44 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.5172S), skipping check
2018-04-24 16:18:45 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') turning message relay requesting off
2018-04-24 16:19:02 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.68651S), skipping check
2018-04-24 16:19:05 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.79732S), skipping check
2018-04-24 16:19:07 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT3.32921S), skipping check
2018-04-24 16:19:15 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.55216S), skipping check
2018-04-24 16:19:20 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT2.06564S), skipping check
2018-04-24 16:19:22 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.75015S), skipping check
2018-04-24 16:19:29 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.62047S), skipping check
2018-04-24 16:19:31 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.71421S), skipping check
2018-04-24 16:19:34 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT2.22069S), skipping check
2018-04-24 16:19:37 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.59165S), skipping check
2018-04-24 16:19:38 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection to peer 57076020 with addr tcp://127.0.0.1:16002 timed out, no messages seen in PT3S
2018-04-24 16:19:39 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') turning message relay requesting on, nonlive peers: tcp://127.0.0.1:16002
2018-04-24 16:19:40 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT3.3857S), skipping check
2018-04-24 16:19:40 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') reconnecting to 57076020 (tcp://127.0.0.1:16002), attempt 0
2018-04-24 16:19:41 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.80333S), skipping check
2018-04-24 16:19:46 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection established to 57076020 tcp://127.0.0.1:16002
2018-04-24 16:19:46 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.91571S), skipping check
2018-04-24 16:19:46 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection established to 57076020 tcp://127.0.0.1:16002
2018-04-24 16:19:49 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1.7353S), skipping check
2018-04-24 16:19:51 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') turning message relay requesting off
2018-04-24 16:19:52 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT2.89326S), skipping check
2018-04-24 16:19:58 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT2.84684S), skipping check
2018-04-24 16:20:04 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection to peer 57076020 with addr tcp://127.0.0.1:16002 timed out, no messages seen in PT3S
2018-04-24 16:20:12 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') turning message relay requesting on, nonlive peers: tcp://127.0.0.1:16002
2018-04-24 16:20:18 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') reconnecting to 57076020 (tcp://127.0.0.1:16002), attempt 0
2018-04-24 16:21:17 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT1M4.34766S), skipping check
180424 16:22:09 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
 
Server version: 10.3.7-MariaDB-log
key_buffer_size=1048576
read_buffer_size=131072
max_used_connections=2
max_threads=153
thread_count=9
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63163 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x49000
/home/qa-user1/rr/build-mariadb/sql/mysqld(my_print_stacktrace+0x29)[0x5593f3803dd9]
/home/qa-user1/rr/build-mariadb/sql/mysqld(handle_fatal_signal+0x33f)[0x5593f333e53f]
2018-04-24 16:22:50 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection to peer 00000000 with addr tcp://127.0.0.1:16002 timed out, no messages seen in PT3S
/lib64/libpthread.so.0(+0xf5e0)[0x7fccdea625e0]
/lib64/libc.so.6(__poll+0x2d)[0x7fccdd037a3d]
/home/qa-user1/rr/build-mariadb/sql/mysqld(_Z26handle_connections_socketsv+0x20b)[0x5593f30e157b]
2018-04-24 16:23:43 0 [Note] WSREP: (570d5bae, 'tcp://0.0.0.0:16005') connection to peer 00000000 with addr  timed out, no messages seen in PT3S
/home/qa-user1/rr/build-mariadb/sql/mysqld(_Z11mysqld_mainiPPc+0x2b79)[0x5593f30e4759]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7fccdcf6bc05]
/home/qa-user1/rr/build-mariadb/sql/mysqld(+0x4e2398)[0x5593f30d8398]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Writing a core file at /home/qa-user1/rr/build-mariadb/mysql-test/var/mysqld.2/data/



 Comments   
Comment by Sachin Setiya (Inactive) [ 2018-05-02 ]

Hi winstone, Is it specific to 10.3 or reproducible in 10.1 also ?

Comment by Jan Lindström (Inactive) [ 2018-05-03 ]

Hi, Could not repeat on 10.1 and 10.3:

jan@jan-laptop-asus:~/mysql/10.3/mysql-test$ ./mtr galera_many_tables_nopk --ddd
Logging: ./mtr  galera_many_tables_nopk --ddd
vardir: /home/jan/mysql/10.3/mysql-test/var
Checking leftover processes...
Removing old var directory...
Creating var directory '/home/jan/mysql/10.3/mysql-test/var'...
Checking supported features...
MariaDB Version 10.3.7-MariaDB-debug
 - SSL connections supported
 - binaries are debug compiled
Collecting tests...
Installing system database...
 
==============================================================================
 
TEST                                      RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------
 
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
galera.galera_many_tables_nopk 'innodb'  [ pass ]  66078
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 66.078 of 141 seconds executing testcases
 
Completed: All 1 tests were successful.
 
jan@jan-laptop-asus:~/mysql/10.3/mysql-test$ ./mtr galera_many_tables_nopk
Logging: ./mtr  galera_many_tables_nopk
vardir: /home/jan/mysql/10.3/mysql-test/var
Checking leftover processes...
Removing old var directory...
Creating var directory '/home/jan/mysql/10.3/mysql-test/var'...
Checking supported features...
MariaDB Version 10.3.7-MariaDB-debug
 - SSL connections supported
 - binaries are debug compiled
Collecting tests...
Installing system database...
 
==============================================================================
 
TEST                                      RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------
 
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
galera.galera_many_tables_nopk 'innodb'  [ pass ]  75023
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 75.023 of 93 seconds executing testcases
 
Completed: All 1 tests were successful.
 
jan@jan-laptop-asus:~/mysql/10.3/mysql-test$ ./mtr --suite=galera --force --parallel=6
Logging: ./mtr  --suite=galera --force --parallel=6
vardir: /home/jan/mysql/10.3/mysql-test/var
Checking leftover processes...
 - found old pid 3092 in 'mysqld.2.pid', killing it...
   process did not exist!
Removing old var directory...
Creating var directory '/home/jan/mysql/10.3/mysql-test/var'...
Checking supported features...
MariaDB Version 10.3.7-MariaDB-debug
 - SSL connections supported
 - binaries are debug compiled
Using suites: galera
Collecting tests...
Installing system database...
^Cmysql-test-run: *** ERROR: Got ^C signal
jan@jan-laptop-asus:~/mysql/10.3/mysql-test$ ./mtr --suite=galera --force --parallel=6 --vardir=/dev/shm --big
Logging: ./mtr  --suite=galera --force --parallel=6 --vardir=/dev/shm --big
vardir: /dev/shm
Checking leftover processes...
Removing old var directory...
couldn't chmod(0777, /dev/shm/PostgreSQL.1599903900): Operation not permitted at /usr/share/perl/5.26/File/Find.pm line 454.
Couldn't delete file '/dev/shm/PostgreSQL.1599903900': Operation not permitted at /usr/share/perl/5.26/File/Find.pm line 454.
couldn't chmod(0777, /dev/shm): Operation not permitted at /usr/share/perl/5.26/File/Find.pm line 511.
Couldn't remove directory '/dev/shm': Permission denied at /usr/share/perl/5.26/File/Find.pm line 511.
Creating var directory '/dev/shm'...
Checking supported features...
MariaDB Version 10.3.7-MariaDB-debug
 - SSL connections supported
 - binaries are debug compiled
Using suites: galera
Collecting tests...
Installing system database...
 
==============================================================================
 
TEST                                  WORKER RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------
 
worker[3] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
worker[2] Using MTR_BUILD_THREAD 302, with reserved ports 16040..16059
worker[1] Using MTR_BUILD_THREAD 301, with reserved ports 16020..16039
worker[6] Using MTR_BUILD_THREAD 303, with reserved ports 16060..16079
worker[5] Using MTR_BUILD_THREAD 304, with reserved ports 16080..16099
worker[4] Using MTR_BUILD_THREAD 305, with reserved ports 16100..16119
galera.GAL-419                           [ disabled ]  MDEV-13549 Galera test failures
galera.GAL-480                           [ disabled ]  "Lost connection to MySQL"
galera.MW-328A                           [ disabled ]  have_deadlocks test not stable
galera.MW-329                            [ disabled ]  wsrep_local_replays not stable
galera.MW-44                             [ disabled ]  MDEV-15809 Test failure on galera.MW-44
galera.galera_account_management         [ disabled ]  MariaDB 10.0 does not support ALTER USER
galera.galera_as_master_gtid             [ disabled ]  Requires MySQL GTID
galera.galera_as_master_gtid_change_master [ disabled ]  Requires MySQL GTID
galera.galera_as_slave_preordered        [ disabled ]  wsrep-preordered feature not merged to MariaDB
galera.galera_as_slave_replication_bundle [ disabled ]  MDEV-13549 Galera test failures
galera.galera_binlog_rows_query_log_events [ disabled ]  MariaDB does not support binlog_rows_query_log_events
galera.galera_concurrent_ctas            [ disabled ]  MDEV-15845 Test failure on galera.galera_concurrent_ctas
galera.galera_flush                      [ disabled ]  MariaDB does not have global.thread_statistics
galera.galera_gcache_recover             [ disabled ]  MDEV-13549 Galera test failures
galera.galera_gcache_recover_full_gcache [ disabled ]  MDEV-13549 Galera test failures
galera.galera_gcache_recover_manytrx     [ disabled ]  MDEV-13549 Galera test failures
galera.galera_ist_mysqldump              [ disabled ]  MDEV-13549 Galera test failures
galera.galera_ist_progress               [ disabled ]  MDEV-15236 galera_ist_progress fails when trying to read transfer status
galera.galera_kill_applier               [ disabled ]  race condition at the start of the test
galera.galera_migrate                    [ disabled ]  MariaDB does not support START SLAVE USER
galera.galera_pc_ignore_sb               [ disabled ]  MDEV-15811 Test failure on galera_pc_ignore_sb
galera.galera_ssl_upgrade                [ disabled ]  MDEV-13549 Galera test failures
galera.galera_sst_mysqldump              [ disabled ]  MDEV-14069
galera.galera_var_auto_inc_control_on    [ disabled ]  MDEV-15803 Test failure on galera.galera_var_auto_inc_control_on
galera.galera_var_notify_cmd             [ disabled ]  MDEV-13549 Galera test failures
galera.galera_var_retry_autocommit       [ disabled ]  MDEV-15794 Test failure on galera.galera_var_retry_autocommit
galera.query_cache                       [ disabled ]  MDEV-15805 Test failure on galera.query_cache
galera.galera_autoinc_sst_xtrabackup 'innodb' [ skipped ]  Need innobackupex
galera.galera_ist_xtrabackup-v2 'debug,innodb' [ skipped ]  Need innobackupex
galera.galera_sst_xtrabackup-v2-options  [ skipped ]  Need innobackupex
galera.galera_sst_xtrabackup-v2 'innodb' [ skipped ]  Need innobackupex
galera.galera_sst_xtrabackup-v2_encrypt_with_key 'innodb' [ skipped ]  Need innobackupex
galera.galera_binlog_event_max_size_min 'innodb' w5 [ pass ]   2028
galera.galera_binlog_checksum 'innodb'   w2 [ pass ]   2044
galera.MW-313 'innodb'                   w1 [ pass ]   2106
galera.MW-86-wait1 'innodb'              w3 [ pass ]   2136
galera.MW-86-wait8 'innodb'              w6 [ pass ]   4531
galera.enforce_storage_engine2 'innodb'  w5 [ pass ]   2072
galera.galera_log_output_csv 'innodb'    w2 [ pass ]   2064
galera.galera_mdev_13787 'innodb'        w1 [ pass ]   2191
galera.galera_gtid 'innodb'              w6 [ pass ]   2138
galera.galera_applier_ftwrl_table_alter 'innodb' w3 [ pass ]   8146
galera.galera_sbr_binlog 'innodb'        w2 [ pass ]   2069
galera.galera_forced_binlog_format 'innodb' w5 [ pass ]   2135
galera.galera_log_bin 'innodb'           w1 [ pass ]   2490
galera.galera_v1_row_events 'innodb'     w6 [ pass ]   2135
galera.galera_binlog_event_max_size_max 'innodb' w4 [ pass ]  43831
galera.galera_query_cache 'innodb'       w5 [ pass ]   2097
galera.galera_wsrep_log_conficts 'innodb' w4 [ pass ]   2167
galera.galera_wsrep_new_cluster 'innodb' w6 [ pass ]   4012
galera.galera_query_cache_sync_wait 'innodb' w5 [ pass ]   4207
galera.mysql-wsrep#201 'innodb'          w1 [ pass ]   6557
galera.partition 'innodb'                w3 [ pass ]  25171
galera.galera_var_sst_auth 'innodb'      w4 [ pass ]   2052
galera.galera_flush_local 'innodb'       w2 [ pass ]  25599
galera.create 'innodb'                   w5 [ pass ]   2615
galera.galera_fk_cascade_update 'innodb' w1 [ pass ]   2244
galera.enforce_storage_engine 'innodb'   w5 [ pass ]   2124
galera.galera_fk_conflict 'innodb'       w1 [ pass ]   2148
galera.galera_nopk_blob 'innodb'         w3 [ pass ]   2137
galera.ev51914 'innodb'                  w5 [ pass ]   2499
galera.galera_fk_mismatch 'innodb'       w1 [ pass ]   2182
galera.galera_nopk_large_varchar 'innodb' w3 [ pass ]   2148
galera.fk 'innodb'                       w5 [ pass ]   2661
galera.galera_fk_multicolumn 'innodb'    w1 [ pass ]   2227
galera.galera_nopk_unicode 'innodb'      w3 [ pass ]   2109
galera.galera_fk_multitable 'innodb'     w1 [ pass ]   2175
galera.galera_parallel_apply_lock_table 'innodb' w3 [ pass ]   4131
galera.galera_fk_no_pk 'innodb'          w1 [ pass ]   2235
galera.galera_truncate 'innodb'          w2 [ pass ]   2464
galera.galera_parallel_simple 'innodb'   w3 [ pass ]   2289
galera.galera_fk_selfreferential 'innodb' w1 [ pass ]   2110
galera.galera_truncate_temporary 'innodb' w2 [ pass ]   2325
galera.galera_pk_bigint_signed 'innodb'  w3 [ pass ]   2087
galera.galera_admin 'innodb'             w5 [ pass ]  11749
galera.galera_fk_setnull 'innodb'        w1 [ pass ]   2382
galera.galera_unicode_identifiers 'innodb' w2 [ pass ]   2416
galera.galera_pk_bigint_unsigned 'innodb' w3 [ pass ]   2175
galera.galera_alter_engine_innodb 'innodb' w5 [ pass ]   2263
galera.galera_unicode_pk 'innodb'        w2 [ pass ]   2110
galera.galera_prepared_statement 'innodb' w3 [ pass ]   2373
galera.galera_alter_engine_myisam 'innodb' w5 [ pass ]   2138
galera.galera_ftwrl 'innodb'             w1 [ pass ]   4103
galera.galera_update_limit 'innodb'      w2 [ pass ]   2610
galera.galera_read_only 'innodb'         w3 [ pass ]   2182
galera.galera_alter_table_force 'innodb' w5 [ pass ]   2140
galera.galera_ftwrl_drain 'innodb'       w1 [ skipped ]  "Test requires Galera debug library with debug_sync functionality"
galera.galera_var_OSU_method 'innodb'    w2 [ pass ]   2084
galera.galera_repl_key_format_flat16 'innodb' w3 [ pass ]   2154
galera.galera_applier_ftwrl_table 'innodb' w5 [ pass ]   2201
galera.galera_var_OSU_method2 'innodb'   w2 [ pass ]   2285
galera.galera_repl_max_ws_size 'innodb'  w3 [ pass ]   2160
galera.galera_bf_abort 'innodb'          w5 [ pass ]   2138
galera.galera_bf_abort_flush_for_export 'innodb' w5 [ pass ]   2217
galera.galera_var_auto_inc_control_off 'innodb' w2 [ pass ]   4325
galera.galera_bf_abort_for_update 'innodb' w5 [ pass ]   2127
galera.galera_gcs_fc_limit 'innodb'      w1 [ pass ]  13367
galera.galera_var_certify_nonPK_off 'innodb' w2 [ pass ]   2390
galera.galera_split_brain 'innodb'       w4 [ pass ]  34985
galera.galera_bf_abort_ftwrl 'innodb'    w5 [ pass ]   2162
galera.galera_gra_log 'innodb'           w1 [ pass ]   2192
galera.galera_sql_log_bin_zero 'innodb'  w4 [ pass ]   2105
galera.MW-286 'innodb'                   w6 [ fail ]
        Test ended at 2018-05-03 15:09:55
 
CURRENT_TEST: galera.MW-286
mysqltest: At line 26: query 'ALTER TABLE t1 ADD PRIMARY KEY (f1)' succeeded - should have failed with errno 1317...
 
The result from queries just before the failure was:
connection node_1;
CREATE TABLE ten (f1 INTEGER);
INSERT INTO ten VALUES (0),(1),(2),(3),(4),(5),(6),(7),(8),(9);
CREATE TABLE t1 (f1 INTEGER) Engine=InnoDB;
INSERT INTO t1 (f1) SELECT 000000 + (10000 * a1.f1) + (1000 * a2.f1) + (100 * a3.f1) + (10 * a4.f1) + a5.f1 FROM ten AS a1, ten AS a2, ten AS a3, ten AS a4, ten AS a5;
INSERT INTO t1 (f1) SELECT 100000 + (10000 * a1.f1) + (1000 * a2.f1) + (100 * a3.f1) + (10 * a4.f1) + a5.f1 FROM ten AS a1, ten AS a2, ten AS a3, ten AS a4, ten AS a5;;
connection node_2;
SET GLOBAL wsrep_desync = TRUE;
SET wsrep_on = FALSE;
ALTER TABLE t1 ADD PRIMARY KEY (f1);
 
 - saving '/dev/shm/6/log/galera.MW-286-innodb/' to '/dev/shm/log/galera.MW-286-innodb/'
galera.galera_bf_abort_get_lock 'innodb' w5 [ pass ]   3070
galera.galera_insert_ignore 'innodb'     w1 [ pass ]   2140
galera.galera_status_cluster 'innodb'    w4 [ pass ]   2029
galera.galera_bf_abort_lock_table 'innodb' w5 [ pass ]   2064
galera.galera_insert_multi 'innodb'      w1 [ pass ]   2274
galera.galera_status_local_index 'innodb' w4 [ pass ]   2077
galera.galera_restart_nochanges 'innodb' w3 [ pass ]  19191
galera.galera_bf_abort_sleep 'innodb'    w5 [ pass ]   2047
galera.galera_status_local_state 'innodb' w4 [ pass ]   2057
galera.galera_binlog_cache_size 'innodb' w5 [ pass ]   3180
galera.galera_binlog_row_image 'innodb'  w5 [ pass ]   2337
galera.galera_var_cluster_address 'innodb' w2 [ pass ]  19299
galera.galera_create_function 'innodb'   w5 [ pass ]   2055
galera.galera_create_procedure 'innodb'  w5 [ pass ]   2114
galera.galera_var_desync_on 'innodb'     w2 [ pass ]   3380
galera.galera_create_table_like 'innodb' w5 [ pass ]   2347
galera.galera_create_trigger 'innodb'    w5 [ pass ]   2458
galera.galera_ddl_multiline 'innodb'     w5 [ pass ]   2763
galera.galera_kill_ddl 'innodb'          w1 [ pass ]  24966
galera.galera_defaults 'innodb'          w5 [ pass ]   2160
galera.galera_suspend_slave 'innodb'     w4 [ pass ]  25705
galera.galera_delete_limit 'innodb'      w5 [ pass ]   2193
galera.galera_sync_wait_show 'innodb'    w4 [ pass ]   2109
galera.galera_var_dirty_reads 'innodb'   w2 [ pass ]  17301
galera.galera_var_fkchecks 'innodb'      w2 [ pass ]   2234
galera.galera_desync_overlapped 'innodb' w5 [ pass ]   4044
galera.galera_toi_alter_auto_increment 'innodb' w4 [ pass ]   4368
galera.galera_drop_multi 'innodb'        w5 [ pass ]   2103
galera.galera_var_innodb_disallow_writes 'innodb' w2 [ pass ]   3190
galera.galera_enum 'innodb'              w5 [ pass ]   2142
galera.galera_toi_ddl_locking 'innodb'   w4 [ pass ]   5195
galera.galera_kill_nochanges 'innodb'    w1 [ pass ]  18481
galera.galera_toi_ddl_nonconflicting 'innodb' w4 [ pass ]   2182
galera.galera_events 'innodb'            w5 [ pass ]   3042
galera.galera_toi_ddl_sequential 'innodb' w4 [ pass ]   2232
galera.galera_fk_cascade_delete 'innodb' w5 [ pass ]   2270
galera.galera_var_max_ws_size 'innodb'   w5 [ pass ]   2082
galera.galera_var_mysql_replication_bundle 'innodb' w5 [ pass ]   2097
galera.galera_var_reject_queries 'innodb' w5 [ pass ]   4131
galera.galera_var_replicate_myisam_off 'innodb' w5 [ pass ]   2041
galera.galera_var_replicate_myisam_on 'innodb' w5 [ pass ]   2369
galera.galera_kill_smallchanges 'innodb' w1 [ pass ]  26307
galera.galera_var_slave_threads 'innodb' w5 [ pass ]   2841
galera.galera_var_sync_wait 'innodb'     w5 [ pass ]   2217
galera.galera_lock_table 'innodb'        w1 [ pass ]   4203
galera.galera_var_wsrep_on_off 'innodb'  w5 [ pass ]   2070
galera.galera_wsrep_provider_options_syntax 'innodb' w5 [ pass ]   2064
galera.galera_many_indexes 'innodb'      w1 [ pass ]   6327
galera.galera_var_max_ws_rows 'innodb'   w2 [ pass ]  48744
galera.galera_mdev_10812 'innodb'        w1 [ pass ]   6085
galera.lp1276424 'innodb'                w2 [ pass ]   2108
galera.lp1347768 'innodb'                w2 [ pass ]   2140
galera.galera_bf_lock_wait 'innodb'      w6 [ pass ]  102508
galera.galera_restart_on_unknown_option 'innodb' w3 [ pass ]  101360
galera.lp1376747 'innodb'                w2 [ pass ]   4116
galera.galera_mdl_race 'innodb'          w1 [ pass ]  11183
galera.lp1376747-2 'innodb'              w2 [ pass ]   2098
galera.galera_wsrep_provider_unset_set 'innodb' w5 [ pass ]  20517
galera.galera_roles 'innodb'             w3 [ pass ]   5318
galera.lp1376747-3 'innodb'              w2 [ pass ]   2179
galera.galera_multi_database 'innodb'    w1 [ pass ]   2228
galera.galera_zero_length_column 'innodb' w5 [ pass ]   2205
galera.galera_rsu_error 'innodb'         w3 [ pass ]   2164
galera.galera_myisam_autocommit 'innodb' w1 [ pass ]   2069
galera.grant 'innodb'                    w5 [ pass ]   2044
galera.galera_rsu_simple 'innodb'        w3 [ pass ]   2103
galera.galera_myisam_transactions 'innodb' w1 [ pass ]   2120
galera.mysql-wsrep#198 'innodb'          w5 [ pass ]   2345
galera.galera_nopk_bit 'innodb'          w1 [ pass ]   2140
galera.lp1376747-4 'innodb'              w2 [ pass ]   9112
galera.galera_rsu_wsrep_desync 'innodb'  w3 [ pass ]   4291
galera.mysql-wsrep#237 'innodb'          w5 [ pass ]   3114
galera.view 'innodb'                     w1 [ pass ]   2113
galera.lp1438990 'innodb'                w2 [ pass ]   2227
galera.galera_sbr 'innodb'               w3 [ pass ]   2109
galera.mysql-wsrep#247 'innodb'          w5 [ pass ]   3088
galera.lp959512 'innodb'                 w2 [ pass ]   2166
galera.galera_schema_dirty_reads 'innodb' w3 [ pass ]   2057
galera.mdev_9290 'innodb'                w2 [ pass ]   2095
galera.mysql-wsrep#110 'innodb'          w2 [ pass ]   2254
galera.galera_serializable 'innodb'      w3 [ pass ]   6145
galera.galera_server 'innodb'            w3 [ pass ]   2043
galera.mysql-wsrep#31 'innodb'           w5 [ pass ]  22856
galera.MW-284 'innodb'                   w2 [ pass ]   9642
galera.galera_fulltext 'innodb'          w6 [ pass ]  62190
galera.MDEV-15443 'innodb'               w1 [ pass ]  24736
galera.mysql-wsrep#90 'innodb'           w5 [ pass ]   2379
galera.galera#414 'innodb'               w3 [ pass ]  13949
galera.galera_as_master 'innodb'         w2 [ pass ]   2637
galera.galera_as_master_large 'innodb'   w1 [ pass ]   2298
galera.galera_as_slave_autoinc 'innodb'  w2 [ pass ]   2470
galera.galera_as_slave 'innodb'          w3 [ pass ]   2278
galera.pxc-421 'innodb'                  w5 [ fail ]
        Test ended at 2018-05-03 15:13:12
 
CURRENT_TEST: galera.pxc-421
mysqltest: In included file "./include/wait_until_connected_again.inc": 
included from /home/jan/mysql/10.3/mysql-test/suite/galera/t/pxc-421.test at line 40:
At line 15: query 'show status' failed with wrong errno 1205: 'Lock wait timeout exceeded; try restarting transaction', instead of 0...
 
The result from queries just before the failure was:
connection node_1;
connection node_2;
connection node_1;
set GLOBAL wsrep_slave_threads=26;
CREATE TABLE t1 (f1 INTEGER) ENGINE=InnoDB;
INSERT INTO t1 VALUES (1);
INSERT INTO t1 (f1) SELECT * from t1 as x1;
connection node_2;
set GLOBAL wsrep_slave_threads=16;
SET GLOBAL wsrep_provider='none';
INSERT INTO t1 VALUES (2);
connection node_1;
INSERT INTO t1 VALUES (3);
connection node_2;
 
 - saving '/dev/shm/5/log/galera.pxc-421-innodb/' to '/dev/shm/log/galera.pxc-421-innodb/'
galera.galera_as_slave_nonprim 'innodb'  w2 [ skipped ]  Test requires wsrep_on=ON
galera.rename 'innodb'                   w5 [ pass ]   2142
galera.galera_as_slave_gtid 'innodb'     w1 [ pass ]   3170
galera.sql_log_bin 'innodb'              w5 [ pass ]   2240
galera.unique_key 'innodb'               w5 [ pass ]   2150
galera.galera_gcs_fragment 'innodb'      w3 [ skipped ]  "Test requires Galera debug library with debug_sync functionality"
galera.galera_gtid_slave 'innodb'        w1 [ pass ]   2951
galera.galera_gcs_max_packet_size 'innodb' w2 [ pass ]  12304
galera.galera_ist_restart_joiner 'innodb' w2 [ skipped ]  "Test requires Galera debug library with debug_sync functionality"
galera.galera_gtid_slave_sst_rsync 'innodb' w5 [ pass ]  23379
galera.galera_ist_recv_bind 'innodb'     w1 [ pass ]  14684
galera.galera_mdev_15611 'innodb'        w5 [ pass ]   4343
galera.galera_ssl 'innodb'               w1 [ pass ]   2124
worker[3] mysql-test-run: WARNING: Waited 60 seconds for /dev/shm/3/run/mysqld.2.pid to be created, still waiting for 120 seconds...
galera.galera_ssl_compression 'innodb'   w5 [ pass ]   2078
worker[5] mysql-test-run: WARNING: Check-testcase failed, this could also be caused by the previous test run by this worker thread
galera.galera_sst_mysqldump_with_key 'innodb' w5 [ fail ]
        Test ended at 2018-05-03 15:14:57
 
CURRENT_TEST: galera.galera_sst_mysqldump_with_key
 
 
Could not execute 'check-testcase' before testcase 'galera.galera_sst_mysqldump_with_key' (res: 1):
mysqltest: Logging to '/dev/shm/5/tmp/check-mysqld_1.log'.
mysqltest: Results saved in '/dev/shm/5/tmp/check-mysqld_1.result'.
mysqltest: Connecting to server localhost:16080 (socket /dev/shm/tmp/5/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...
mysqltest: Could not open connection 'default': 2026 SSL connection error: tlsv1 alert unknown ca
not ok
mysqltest failed but provided no output
 
 
 - saving '/dev/shm/5/log/galera.galera_sst_mysqldump_with_key-innodb/' to '/dev/shm/log/galera.galera_sst_mysqldump_with_key-innodb/'
galera.galera_ist_rsync 'debug,innodb'   w2 [ pass ]  59322
galera.galera_toi_drop_database 'innodb' w4 [ pass ]  278048
galera.galera_var_gtid_domain_id 'innodb' w2 [ pass ]   2048
galera.galera_toi_ftwrl 'innodb'         w4 [ pass ]   2092
galera.galera_toi_lock_exclusive 'innodb' w4 [ pass ]   3103
worker[3] mysql-test-run: WARNING: Waited 120 seconds for /dev/shm/3/run/mysqld.2.pid to be created, still waiting for 60 seconds...
galera.galera_toi_lock_shared 'innodb'   w4 [ pass ]   2071
galera.galera_var_log_bin 'innodb'       w2 [ pass ]   2077
galera.galera_sst_rsync 'debug,innodb'   w5 [ pass ]  63564
galera.galera_var_node_address 'innodb'  w2 [ pass ]   2140
galera.galera_toi_truncate 'innodb'      w4 [ pass ]  47327
galera.galera_transaction_read_only 'innodb' w4 [ pass ]   2059
galera.galera_wan 'innodb'               w5 [ pass ]   2131
galera.galera_transaction_replay 'innodb' w4 [ skipped ]  "Test requires Galera debug library with debug_sync functionality"
worker[3] mysql-test-run: WARNING: Timeout after mysql-test-run waited 180 seconds for the process [mysqld.2 - pid: 16763, winpid: 16763] to create a pid file.
galera.galera_ist_innodb_flush_logs 'debug,innodb' w3 [ fail ]
        Test ended at 2018-05-03 15:16:38
 
CURRENT_TEST: galera.galera_ist_innodb_flush_logs
 
 
Failed to start mysqld.2
mysqltest failed but provided no output
 
 
 - saving '/dev/shm/3/log/galera.galera_ist_innodb_flush_logs-debug,innodb/' to '/dev/shm/log/galera.galera_ist_innodb_flush_logs-debug,innodb/'
***Warnings generated in error logs during shutdown after running tests: galera.galera_ist_innodb_flush_logs
 
2018-05-03 15:13:38 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '127.0.0.1:16007' --datadir '/dev/shm/3/mysqld.2/data/'  --defaults-file '/dev/shm/3/my.cnf' --defaults-group-suffix '.2'  --parent '16764'  '' 
2018-05-03 15:13:38 0 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '127.0.0.1:16007' --datadir '/dev/shm/3/mysqld.2/data/'  --defaults-file '/dev/shm/3/my.cnf' --defaults-group-suffix '.2'  --parent '16764'  '' : 2 (No such file or directory)
2018-05-03 15:13:38 2 [ERROR] WSREP: Failed to prepare for 'xtrabackup-v2' SST. Unrecoverable.
2018-05-03 15:13:38 2 [ERROR] Aborting
 
galera.galera_sst_mariabackup 'debug,innodb' w1 [ pass ]  111889
galera.mdev_10518 'innodb'               w4 [ pass ]   2127
galera.galera_many_columns 'innodb'      w1 [ pass ]   5270
galera.mysql-wsrep#33 'innodb'           w3 [ pass ]  31044
galera.rpl_row_annotate 'innodb'         w3 [ pass ]   2183
galera.galera_wan_restart_ist 'innodb'   w2 [ pass ]  65989
galera.galera_wan_restart_sst 'innodb'   w5 [ pass ]  77492
galera.galera_many_tables_pk 'innodb'    w3 [ pass ]  23593
galera.galera_parallel_autoinc_manytrx 'innodb' w2 [ pass ]  59824
galera.galera_parallel_autoinc_largetrx 'innodb' w3 [ pass ]  39288
galera.galera_kill_largechanges 'innodb' w6 [ pass ]  388697
galera.galera_many_tables_nopk 'innodb'  w1 [ pass ]  168938
galera.galera_toi_ddl_fk_update 'innodb' w6 [ pass ]  112012
galera.galera_toi_ddl_error 'innodb'     w2 [ pass ]  156122
galera.galera_var_load_data_splitting 'innodb' w1 [ pass ]  99921
galera.GAL-382 'innodb'                  w2 [ pass ]   2361
galera.MW-252 'innodb'                   w2 [ pass ]   4135
galera.MW-258 'innodb'                   w2 [ pass ]  10133
galera.GAL-401 'innodb'                  w1 [ pass ]  20363
galera.MW-259 'innodb'                   w2 [ pass ]   4223
galera.MW-285 'innodb'                   w1 [ pass ]   2423
galera.MW-292 'innodb'                   w2 [ skipped ]  "Test requires Galera debug library with debug_sync functionality"
galera.MW-309 'innodb'                   w1 [ pass ]   2217
galera.MW-328B 'innodb'                  w2 [ fail ]  Found warnings/errors in server log file!
        Test ended at 2018-05-03 15:22:17
line
safe_mutex: Found wrong usage of mutex 'LOCK_wsrep_thd' and 'LOCK_thd_kill'
^ Found warnings in /dev/shm/2/log/mysqld.2.err
ok
 
 - saving '/dev/shm/2/log/galera.MW-328B-innodb/' to '/dev/shm/log/galera.MW-328B-innodb/'
galera.MW-328C 'innodb'                  w1 [ fail ]  Found warnings/errors in server log file!
        Test ended at 2018-05-03 15:22:17
line
safe_mutex: Found wrong usage of mutex 'LOCK_wsrep_thd' and 'LOCK_thd_kill'
^ Found warnings in /dev/shm/1/log/mysqld.2.err
ok
 
 - saving '/dev/shm/1/log/galera.MW-328C-innodb/' to '/dev/shm/log/galera.MW-328C-innodb/'
galera.MW-328D 'innodb'                  w2 [ pass ]   4236
galera.MW-328E 'innodb'                  w1 [ pass ]   4253
galera.MW-357 'innodb'                   w1 [ pass ]   2085
galera.MW-336 'innodb'                   w2 [ pass ]   2657
galera.MW-369 'innodb'                   w1 [ skipped ]  "Test requires Galera debug library with debug_sync functionality"
galera.MW-402 'innodb'                   w1 [ skipped ]  "Test requires Galera debug library with debug_sync functionality"
galera.basic 'innodb'                    w1 [ pass ]   2351
galera.binlog_checksum 'innodb'          w1 [ pass ]   2344
galera.galera_rsu_add_pk 'innodb'        w5 [ pass ]  392934
galera.galera_wsrep_desync_wsrep_on 'innodb' w6 [ pass ]  284279
galera.galera_rsu_drop_pk 'innodb'       w3 [ pass ]  482801
galera.MW-388 'innodb'                   w2 [ fail ]  Found warnings/errors in server log file!
        Test ended at 2018-05-03 15:27:40
line
2018-05-03 15:26:41 0 [Warning] InnoDB: A long semaphore wait:
2018-05-03 15:27:12 0 [Warning] InnoDB: A long semaphore wait:
^ Found warnings in /dev/shm/2/log/mysqld.1.err
ok
 
 - saving '/dev/shm/2/log/galera.MW-388-innodb/' to '/dev/shm/log/galera.MW-388-innodb/'
galera.galera_toi_ddl_fk_insert 'innodb' w3 [ pass ]  27530
galera.galera_many_rows 'innodb'         w4 [ pass ]  657758
--------------------------------------------------------------------------
The servers were restarted 64 times
Spent 4885.011 of 1251 seconds executing testcases
 
Completed: Failed 7/224 tests, 96.88% were successful.
 
Failing test(s): galera.MW-286 galera.pxc-421 galera.galera_sst_mysqldump_with_key galera.galera_ist_innodb_flush_logs galera.MW-328B galera.MW-328C galera.MW-388
 
The log files in var/log may give you some hint of what went wrong.
 
If you want to report this error, please read first the documentation
at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html
 
Errors/warnings were found in logfiles during server shutdown after running the
following sequence(s) of tests:
    galera.galera_ist_innodb_flush_logs
13 tests were skipped, 8 by the test itself.
 
mysql-test-run: *** ERROR: there were failing test cases

Generated at Thu Feb 08 08:25:40 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.