170328 14:44:22 mysqld_safe Starting mysqld daemon with databases from /var/persistent 170328 14:44:22 mysqld_safe WSREP: Running position recovery with --log_error='/var/persistent/wsrep_recovery.9L58Ui' --pid-file='/var/persistent/vmnds23node2-recover.pid' 2017-03-28 14:44:22 139626429830976 [Note] /usr/local/mariadb/mariadb_base/bin/mysqld (mysqld 10.1.21-MariaDB) starting as process 17301 ... 170328 14:44:25 mysqld_safe WSREP: Recovered position 2e8c7a29-13e5-11e7-8cdf-eb6e2d8900a8:150 2017-03-28 14:44:25 139781765760832 [Note] /usr/local/mariadb/mariadb_base/bin/mysqld (mysqld 10.1.21-MariaDB) starting as process 17411 ... 2017-03-28 14:44:25 139781765760832 [Note] WSREP: Read nil XID from storage engines, skipping position init 2017-03-28 14:44:25 139781765760832 [Note] WSREP: wsrep_load(): loading provider library '/lib/libgalera_smm.so' 2017-03-28 14:44:25 139781765760832 [Note] WSREP: wsrep_load(): Galera 25.3.19(r3667) by Codership Oy loaded successfully. 2017-03-28 14:44:25 139781765760832 [Note] WSREP: CRC-32C: using "slicing-by-8" algorithm. 2017-03-28 14:44:25 139781765760832 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1, safe_to_bootsrap: 0 2017-03-28 14:44:25 139781765760832 [Note] WSREP: Passing config to GCS: base_dir = /var/persistent/; base_host = 10.9.29.32; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT2.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 512; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 256; evs.view_forget_timeout = PT24H; gcache.dir = /var/persistent/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/persistent//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 2410151936; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.listen_addr = tcp://0.0.0.0:4567; gmcast.segment = 0; gmc 2017-03-28 14:44:25 139781765760832 [Note] WSREP: GCache history reset: old(00000000-0000-0000-0000-000000000000:0) -> new(00000000-0000-0000-0000-000000000000:-1) 2017-03-28 14:44:25 139781765760832 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1 2017-03-28 14:44:25 139781765760832 [Note] WSREP: wsrep_sst_grab() 2017-03-28 14:44:25 139781765760832 [Note] WSREP: Start replication 2017-03-28 14:44:25 139781765760832 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1 2017-03-28 14:44:25 139781765760832 [Note] WSREP: protonet asio version 0 2017-03-28 14:44:25 139781765760832 [Note] WSREP: Using CRC-32C for message checksums. 2017-03-28 14:44:25 139781765760832 [Note] WSREP: backend: asio 2017-03-28 14:44:25 139781765760832 [Note] WSREP: gcomm thread scheduling priority set to other:0 2017-03-28 14:44:25 139781765760832 [Warning] WSREP: access file(/var/persistent//gvwstate.dat) failed(No such file or directory) 2017-03-28 14:44:25 139781765760832 [Note] WSREP: restore pc from disk failed 2017-03-28 14:44:25 139781765760832 [Note] WSREP: GMCast version 0 2017-03-28 14:44:25 139781765760832 [Note] WSREP: (91433ac6, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567 2017-03-28 14:44:25 139781765760832 [Note] WSREP: (91433ac6, 'tcp://0.0.0.0:4567') multicast: , ttl: 1 2017-03-28 14:44:25 139781765760832 [Note] WSREP: EVS version 0 2017-03-28 14:44:25 139781765760832 [Note] WSREP: gcomm: connecting to group 'mdbcluster', peer '10.9.29.32:4567,10.9.29.31:4567' 2017-03-28 14:44:25 139781765760832 [Note] WSREP: (91433ac6, 'tcp://0.0.0.0:4567') connection established to 91433ac6 tcp://10.9.29.32:4567 2017-03-28 14:44:25 139781765760832 [Warning] WSREP: (91433ac6, 'tcp://0.0.0.0:4567') address 'tcp://10.9.29.32:4567' points to own listening address, blacklisting 2017-03-28 14:44:26 139781765760832 [Note] WSREP: (91433ac6, 'tcp://0.0.0.0:4567') connection established to 79b129b2 tcp://10.9.29.31:4567 2017-03-28 14:44:26 139781765760832 [Note] WSREP: (91433ac6, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 2017-03-28 14:44:26 139781765760832 [Note] WSREP: declaring 79b129b2 at tcp://10.9.29.31:4567 stable 2017-03-28 14:44:27 139781765760832 [Note] WSREP: Node 79b129b2 state prim 2017-03-28 14:44:27 139781765760832 [Note] WSREP: view(view_id(PRIM,79b129b2,2) memb { 79b129b2,0 91433ac6,0 } joined { } left { } partitioned { }) 2017-03-28 14:44:27 139781765760832 [Note] WSREP: save pc into disk 2017-03-28 14:44:27 139781765760832 [Note] WSREP: gcomm: connected 2017-03-28 14:44:27 139781765760832 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636 2017-03-28 14:44:27 139781765760832 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0) 2017-03-28 14:44:27 139781765760832 [Note] WSREP: Opened channel 'mdbcluster' 2017-03-28 14:44:27 139781438686976 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2 2017-03-28 14:44:27 139781438686976 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2017-03-28 14:44:27 139781765760832 [Note] WSREP: Waiting for SST to complete. 2017-03-28 14:44:27 139781438686976 [Note] WSREP: STATE EXCHANGE: sent state msg: 91f79195-13e6-11e7-bb61-8313c46bb176 2017-03-28 14:44:27 139781438686976 [Note] WSREP: STATE EXCHANGE: got state msg: 91f79195-13e6-11e7-bb61-8313c46bb176 from 0 (10.9.29.31) 2017-03-28 14:44:27 139781438686976 [Note] WSREP: STATE EXCHANGE: got state msg: 91f79195-13e6-11e7-bb61-8313c46bb176 from 1 (10.9.29.32) 2017-03-28 14:44:27 139781438686976 [Note] WSREP: Quorum results: version = 4, component = PRIMARY, conf_id = 1, members = 1/2 (joined/total), act_id = 0, last_appl. = -1, protocols = 0/7/3 (gcs/repl/appl), group UUID = 79b1e0e3-13e6-11e7-9c02-2bee35880b7e 2017-03-28 14:44:27 139781438686976 [Note] WSREP: Flow-control interval: [23, 23] 2017-03-28 14:44:27 139781438686976 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 0) 2017-03-28 14:44:27 139781764279040 [Note] WSREP: State transfer required: Group state: 79b1e0e3-13e6-11e7-9c02-2bee35880b7e:0 Local state: 00000000-0000-0000-0000-000000000000:-1 2017-03-28 14:44:27 139781764279040 [Note] WSREP: New cluster view: global state: 79b1e0e3-13e6-11e7-9c02-2bee35880b7e:0, view# 2: Primary, number of nodes: 2, my index: 1, protocol version 3 2017-03-28 14:44:27 139781764279040 [Warning] WSREP: Gap in state sequence. Need state transfer. 2017-03-28 14:44:27 139781430294272 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'joiner' --address '10.9.29.32:4444' --datadir '/var/persistent/' --defaults-file '/usr/local/mariadb/mariadb_base/etc/my.cnf' --parent '17411' '' ' 2017-03-28 14:44:27 139781764279040 [Note] WSREP: Prepared SST request: rsync|10.9.29.32:4444/rsync_sst 2017-03-28 14:44:27 139781764279040 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2017-03-28 14:44:27 139781764279040 [Note] WSREP: REPL Protocols: 7 (3, 2) 2017-03-28 14:44:27 139781764279040 [Note] WSREP: Assign initial position for certification: 0, protocol version: 3 2017-03-28 14:44:27 139781682370304 [Note] WSREP: Service thread queue flushed. 2017-03-28 14:44:27 139781764279040 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (79b1e0e3-13e6-11e7-9c02-2bee35880b7e): 1 (Operation not permitted) at galera/src/replicator_str.cpp:prepare_for_IST():482. IST will be unavailable. 2017-03-28 14:44:27 139781438686976 [Note] WSREP: Member 1.0 (10.9.29.32) requested state transfer from '*any*'. Selected 0.0 (10.9.29.31)(SYNCED) as donor. 2017-03-28 14:44:27 139781438686976 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 0) 2017-03-28 14:44:27 139781764279040 [Note] WSREP: Requesting state transfer: success, donor: 0 2017-03-28 14:44:27 139781764279040 [Note] WSREP: GCache history reset: old(00000000-0000-0000-0000-000000000000:0) -> new(79b1e0e3-13e6-11e7-9c02-2bee35880b7e:0) 2017-03-28 14:44:28 139781447079680 [Note] WSREP: (91433ac6, 'tcp://0.0.0.0:4567') connection to peer 91433ac6 with addr tcp://10.9.29.32:4567 timed out, no messages seen in PT3S 2017-03-28 14:44:29 139781447079680 [Note] WSREP: (91433ac6, 'tcp://0.0.0.0:4567') turning message relay requesting off 2017-03-28 14:45:02 139781438686976 [Note] WSREP: 0.0 (10.9.29.31): State transfer to 1.0 (10.9.29.32) complete. WSREP_SST: [INFO] Joiner cleanup. rsync PID: 17455 (20170328 14:45:02.551) 2017-03-28 14:45:02 139781438686976 [Note] WSREP: Member 0.0 (10.9.29.31) synced with group. WSREP_SST: [INFO] Joiner cleanup done. (20170328 14:45:03.070) 2017-03-28 14:45:03 139781765760832 [Note] WSREP: SST complete, seqno: 0 2017-3-28 14:45:3 BW Encrypt plug [Note] Initializing mdb plugin... 2017-3-28 14:45:3 BW Encrypt plug [Note] Plugin properly initialized 2017-3-28 14:45:3 BW Encrypt plug [Note] Load keystore request 2017-3-28 14:45:3 BW Encrypt plug [Note] Latest key version = 2 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: The InnoDB memory heap is disabled 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: GCC builtin __sync_synchronize() is used for memory barrier 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: Compressed tables use zlib 1.2.3 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: Using Linux native AIO 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: Using generic crc32 instructions 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: Initializing buffer pool, size = 919.0M 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: Completed initialization of buffer pool 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: Highest supported file format is Barracuda. 2017-3-28 14:45:3 BW Encrypt plug [Note] Key version 2 found 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: The log sequence numbers 9347065 and 9347065 in ibdata files do not match the log sequence number 9353279 in the ib_logfiles! 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: Database was not shutdown normally! 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: Starting crash recovery. 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: Reading tablespace information from the .ibd files... 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: Restoring possible half-written data pages 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: from the doublewrite buffer... 2017-3-28 14:45:3 BW Encrypt plug [Note] Key version 2 found 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: 128 rollback segment(s) are active. 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: Waiting for purge to start 2017-3-28 14:45:3 BW Encrypt plug [Note] Key version 2 found 2017-3-28 14:45:3 BW Encrypt plug [Note] Key version 2 found 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.34-79.1 started; log sequence number 9353279 2017-03-28 14:45:03 139781765760832 [Note] InnoDB: Creating #1 thread id 139779366708992 total threads 1. 2017-03-28 14:45:03 139779467355904 [Note] InnoDB: Dumping buffer pool(s) not yet started 2017-03-28 14:45:03 7f20f17fa700 InnoDB: Loading buffer pool(s) from .//ib_buffer_pool 2017-03-28 14:45:03 139781765760832 [Note] Plugin 'FEEDBACK' is disabled. 170328 14:45:03 server_audit: MariaDB Audit Plugin version 1.4.0 STARTED. 170328 14:45:03 server_audit: logging started to the file /var/logs/auditLog. 2017-3-28 14:45:3 BW Encrypt plug [Note] Key version 2 found 2017-03-28 14:45:03 7f20f17fa700 InnoDB: Buffer pool(s) load completed at 170328 14:45:03 2017-03-28 14:45:03 139781765760832 [Note] Server socket created on IP: '::'. 2017-03-28 14:45:03 139781765760832 [Note] WSREP: Signalling provider to continue. 2017-03-28 14:45:03 139781765760832 [Note] WSREP: SST received: 79b1e0e3-13e6-11e7-9c02-2bee35880b7e:0 2017-3-28 14:45:3 BW Encrypt plug [Note] Key version 2 found 2017-03-28 14:45:04 139781765760832 [Note] /usr/local/mariadb/mariadb_base/bin/mysqld: ready for connections. Version: '10.1.21-MariaDB' socket: '/var/run/mysql.sock' port: 3306 Source distribution 2017-03-28 14:45:04 139781438686976 [Note] WSREP: 1.0 (10.9.29.32): State transfer from 0.0 (10.9.29.31) complete. 2017-03-28 14:45:04 139781438686976 [Note] WSREP: Shifting JOINER -> JOINED (TO: 0) 2017-03-28 14:45:04 139781438686976 [Note] WSREP: Member 1.0 (10.9.29.32) synced with group. 2017-03-28 14:45:04 139781438686976 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 0) 2017-03-28 14:45:04 139781764279040 [Note] WSREP: Synchronized with group, ready for connections 2017-03-28 14:45:04 139781764279040 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2017-3-28 14:45:4 BW Encrypt plug [Note] Key version 2 found 2017-3-28 14:46:50 BW Encrypt plug [Note] Key version 2 found 2017-3-28 14:46:50 BW Encrypt plug [Note] Key version 2 found 2017-3-28 14:47:11 BW Encrypt plug [Note] Key version 2 found 2017-3-28 14:47:14 BW Encrypt plug [Note] Key version 2 found 2017-03-28 14:47:46 139781764279040 [ERROR] Slave SQL: Could not execute Delete_rows_v1 event on table jfd.pbs; Cannot delete or update a parent row: a foreign key constraint fails (`jfd`.`pbt`, CONSTRAINT `pbt_ibfk_1` FOREIGN KEY (`pbs_uid`) REFERENCES `pbs` (`pbs_uid`)), Error_code: 1451; handler error HA_ERR_ROW_IS_REFERENCED; the event's master log FIRST, end_log_pos 143, Internal MariaDB error code: 1451 2017-03-28 14:47:46 139781764279040 [Warning] WSREP: RBR event 2 Delete_rows_v1 apply warning: 152, 43 2017-03-28 14:47:46 139781764279040 [Warning] WSREP: Failed to apply app buffer: seqno: 43, status: 1 at galera/src/trx_handle.cpp:apply():351 Retrying 2th time 2017-03-28 14:47:46 139781764279040 [ERROR] Slave SQL: Could not execute Delete_rows_v1 event on table jfd.pbs; Cannot delete or update a parent row: a foreign key constraint fails (`jfd`.`pbt`, CONSTRAINT `pbt_ibfk_1` FOREIGN KEY (`pbs_uid`) REFERENCES `pbs` (`pbs_uid`)), Error_code: 1451; handler error HA_ERR_ROW_IS_REFERENCED; the event's master log FIRST, end_log_pos 143, Internal MariaDB error code: 1451 2017-03-28 14:47:46 139781764279040 [Warning] WSREP: RBR event 2 Delete_rows_v1 apply warning: 152, 43 2017-03-28 14:47:46 139781764279040 [Warning] WSREP: Failed to apply app buffer: seqno: 43, status: 1 at galera/src/trx_handle.cpp:apply():351 Retrying 3th time 2017-03-28 14:47:46 139781764279040 [ERROR] Slave SQL: Could not execute Delete_rows_v1 event on table jfd.pbs; Cannot delete or update a parent row: a foreign key constraint fails (`jfd`.`pbt`, CONSTRAINT `pbt_ibfk_1` FOREIGN KEY (`pbs_uid`) REFERENCES `pbs` (`pbs_uid`)), Error_code: 1451; handler error HA_ERR_ROW_IS_REFERENCED; the event's master log FIRST, end_log_pos 143, Internal MariaDB error code: 1451 2017-03-28 14:47:46 139781764279040 [Warning] WSREP: RBR event 2 Delete_rows_v1 apply warning: 152, 43 2017-03-28 14:47:46 139781764279040 [Warning] WSREP: Failed to apply app buffer: seqno: 43, status: 1 at galera/src/trx_handle.cpp:apply():351 Retrying 4th time 2017-03-28 14:47:46 139781764279040 [ERROR] Slave SQL: Could not execute Delete_rows_v1 event on table jfd.pbs; Cannot delete or update a parent row: a foreign key constraint fails (`jfd`.`pbt`, CONSTRAINT `pbt_ibfk_1` FOREIGN KEY (`pbs_uid`) REFERENCES `pbs` (`pbs_uid`)), Error_code: 1451; handler error HA_ERR_ROW_IS_REFERENCED; the event's master log FIRST, end_log_pos 143, Internal MariaDB error code: 1451 2017-03-28 14:47:46 139781764279040 [Warning] WSREP: RBR event 2 Delete_rows_v1 apply warning: 152, 43 2017-03-28 14:47:46 139781764279040 [ERROR] WSREP: Failed to apply trx: source: 79b129b2-13e6-11e7-be78-1ef9b108083a version: 3 local: 0 state: APPLYING flags: 1 conn_id: 81 trx_id: 48291 seqnos (l: 47, g: 43, s: 41, d: 42, ts: 21192340744708) 2017-03-28 14:47:46 139781764279040 [ERROR] WSREP: Failed to apply trx 43 4 times 2017-03-28 14:47:46 139781764279040 [ERROR] WSREP: Node consistency compromized, aborting... 2017-03-28 14:47:46 139781764279040 [Note] WSREP: Closing send monitor... 2017-03-28 14:47:46 139781764279040 [Note] WSREP: Closed send monitor. 2017-03-28 14:47:46 139781764279040 [Note] WSREP: gcomm: terminating thread 2017-03-28 14:47:46 139781764279040 [Note] WSREP: gcomm: joining thread 2017-03-28 14:47:46 139781764279040 [Note] WSREP: gcomm: closing backend 2017-03-28 14:47:50 139781764279040 [Note] WSREP: (91433ac6, 'tcp://0.0.0.0:4567') connection to peer 79b129b2 with addr tcp://10.9.29.31:4567 timed out, no messages seen in PT3S 2017-03-28 14:47:50 139781764279040 [Note] WSREP: (91433ac6, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://10.9.29.31:4567 2017-03-28 14:47:51 139781764279040 [Note] WSREP: (91433ac6, 'tcp://0.0.0.0:4567') reconnecting to 79b129b2 (tcp://10.9.29.31:4567), attempt 0 2017-03-28 14:47:54 139781764279040 [Note] WSREP: evs::proto(91433ac6, LEAVING, view_id(REG,79b129b2,2)) suspecting node: 79b129b2 2017-03-28 14:47:54 139781764279040 [Note] WSREP: evs::proto(91433ac6, LEAVING, view_id(REG,79b129b2,2)) suspected node without join message, declaring inactive 2017-03-28 14:47:54 139781764279040 [Note] WSREP: view(view_id(NON_PRIM,79b129b2,2) memb { 91433ac6,0 } joined { } left { } partitioned { 79b129b2,0 }) 2017-03-28 14:47:54 139781764279040 [Note] WSREP: view((empty)) 2017-03-28 14:47:54 139781764279040 [Note] WSREP: gcomm: closed 2017-03-28 14:47:54 139781438686976 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 2017-03-28 14:47:54 139781438686976 [Note] WSREP: Flow-control interval: [16, 16] 2017-03-28 14:47:54 139781438686976 [Note] WSREP: Received NON-PRIMARY. 2017-03-28 14:47:54 139781438686976 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 43) 2017-03-28 14:47:54 139781438686976 [Note] WSREP: Received self-leave message. 2017-03-28 14:47:54 139781438686976 [Note] WSREP: Flow-control interval: [0, 0] 2017-03-28 14:47:54 139781438686976 [Note] WSREP: Received SELF-LEAVE. Closing connection. 2017-03-28 14:47:54 139781438686976 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 43) 2017-03-28 14:47:54 139781438686976 [Note] WSREP: RECV thread exiting 0: Success 2017-03-28 14:47:54 139781764279040 [Note] WSREP: recv_thread() joined. 2017-03-28 14:47:54 139781764279040 [Note] WSREP: Closing replication queue. 2017-03-28 14:47:54 139781764279040 [Note] WSREP: Closing slave action queue. 2017-03-28 14:47:54 139781764279040 [Note] WSREP: /usr/local/mariadb/mariadb_base/bin/mysqld: Terminated. 170328 14:47:54 mysqld_safe Number of processes running now: 0 170328 14:47:54 mysqld_safe WSREP: not restarting wsrep node automatically 170328 14:47:54 mysqld_safe mysqld from pid file /var/run/mysql.pid ended 170328 14:47:56 mysqld_safe Starting mysqld daemon with databases from /var/persistent