2019-11-07 23:48:01 0 [Note] WSREP: Loading provider /usr/lib64/galera-4/libgalera_smm.so initial position: 00000000-0000-0000-0000-000000000000:-1 wsrep loader: [INFO] wsrep_load(): loading provider library '/usr/lib64/galera-4/libgalera_smm.so' wsrep loader: [INFO] wsrep_load(): Galera 26.4.2(r4498) by Codership Oy loaded successfully. 2019-11-07 23:48:01 0 [Note] WSREP: CRC-32C: using hardware acceleration. 2019-11-07 23:48:01 0 [Warning] WSREP: Could not open state file for reading: '/var/lib/mysql//grastate.dat' 2019-11-07 23:48:01 0 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1, safe_to_bootstrap: 1 2019-11-07 23:48:01 0 [Note] WSREP: GCache DEBUG: opened preamble: Version: 0 UUID: 00000000-0000-0000-0000-000000000000 Seqno: -1 - -1 Offset: -1 Synced: 0 2019-11-07 23:48:01 0 [Note] WSREP: Skipped GCache ring buffer recovery: could not determine history UUID. 2019-11-07 23:48:01 0 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 10.1.30.60; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT2.5S; evs.inactive_timeout = PT1M; evs.install_timeout = PT1M; evs.join_retrans_period = PT1S; evs.keepalive_period = PT3S; evs.max_install_timeouts = 3; evs.send_window = 1024; evs.stats_report_period = PT1M; evs.suspect_timeout = PT30S; evs.user_send_window = 1024; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = galera.cache; gcache.page_size = 128M; gcache.recover = yes; gcache.size = 1G; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 0.8; gcs.fc_limit = 160; gcs.fc_master_slave = no; gcs.max_packet_size = 2097152; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; 2019-11-07 23:48:01 0 [Note] WSREP: GCache history reset: old(00000000-0000-0000-0000-000000000000:0 -> 00000000-0000-0000-0000-000000000000:-1 2019-11-07 23:48:01 0 [Note] WSREP: Start replication 2019-11-07 23:48:01 0 [Note] WSREP: Connecting with bootstrap option: 0 2019-11-07 23:48:01 0 [Note] WSREP: Setting GCS initial position to 00000000-0000-0000-0000-000000000000:-1 2019-11-07 23:48:01 0 [Note] WSREP: protonet asio version 0 2019-11-07 23:48:01 0 [Note] WSREP: Using CRC-32C for message checksums. 2019-11-07 23:48:01 0 [Note] WSREP: initializing ssl context 2019-11-07 23:48:01 0 [Note] WSREP: backend: asio 2019-11-07 23:48:01 0 [Note] WSREP: gcomm thread scheduling priority set to other:0 2019-11-07 23:48:01 0 [Warning] WSREP: access file(/var/lib/mysql//gvwstate.dat) failed(No such file or directory) 2019-11-07 23:48:01 0 [Note] WSREP: restore pc from disk failed 2019-11-07 23:48:01 0 [Note] WSREP: GMCast version 0 2019-11-07 23:48:01 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') listening at ssl://0.0.0.0:4567 2019-11-07 23:48:01 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') multicast: , ttl: 1 2019-11-07 23:48:01 0 [Note] WSREP: EVS version 1 2019-11-07 23:48:01 0 [Note] WSREP: gcomm: connecting to group 'galera_cluster', peer '10.1.30.60:,10.1.30.61:,10.20.11.45:,10.20.11.46:,172.25.1.10:' 2019-11-07 23:48:01 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.1.30.61:4567 local endpoint ssl://10.1.30.60:42622 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-07 23:48:01 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.1.30.60:38332 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-07 23:48:01 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.1.30.60:4567 local endpoint ssl://10.1.30.60:38332 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-07 23:48:01 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to f19a4d7f ssl://10.1.30.60:4567 2019-11-07 23:48:01 0 [Warning] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') address 'ssl://10.1.30.60:4567' points to own listening address, blacklisting 2019-11-07 23:48:01 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to d929f93e ssl://10.1.30.61:4567 2019-11-07 23:48:01 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 2019-11-07 23:48:01 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.20.11.45:4567 local endpoint ssl://10.1.30.60:43250 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-07 23:48:01 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to fd37624f ssl://10.20.11.45:4567 2019-11-07 23:48:01 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.20.11.46:56000 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-07 23:48:01 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.20.11.46:4567 local endpoint ssl://10.1.30.60:40860 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-07 23:48:01 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to c108bb1a ssl://10.20.11.46:4567 2019-11-07 23:48:01 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49324 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-07 23:48:01 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-07 23:48:03 0 [Note] WSREP: EVS version upgrade 0 -> 1 2019-11-07 23:48:03 0 [Note] WSREP: declaring 45421c9c at ssl://172.25.1.10:4567 stable 2019-11-07 23:48:03 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-07 23:48:03 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-07 23:48:03 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-07 23:48:03 0 [Note] WSREP: PC protocol upgrade 0 -> 1 2019-11-07 23:48:04 0 [Note] WSREP: Node 45421c9c state prim 2019-11-07 23:48:04 0 [Note] WSREP: view(view_id(PRIM,45421c9c,93) memb { 45421c9c,2 c108bb1a,1 d929f93e,0 f19a4d7f,0 fd37624f,1 } joined { } left { } partitioned { }) 2019-11-07 23:48:04 0 [Note] WSREP: save pc into disk 2019-11-07 23:48:04 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 00000000 with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-07 23:48:04 0 [Note] WSREP: gcomm: connected 2019-11-07 23:48:04 0 [Note] WSREP: Changing maximum packet size to 2097152, resulting msg size: 32636 2019-11-07 23:48:04 0 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0) 2019-11-07 23:48:04 0 [Note] WSREP: Opened channel 'galera_cluster' 2019-11-07 23:48:04 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 3, memb_num = 5 2019-11-07 23:48:04 1 [Note] WSREP: Starting rollbacker thread 1 2019-11-07 23:48:04 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2019-11-07 23:48:04 0 [Note] WSREP: STATE EXCHANGE: sent state msg: f3351b1a-01e2-11ea-856a-2691a814bc36 2019-11-07 23:48:04 0 [Note] WSREP: STATE EXCHANGE: got state msg: f3351b1a-01e2-11ea-856a-2691a814bc36 from 0 (garb) 2019-11-07 23:48:04 2 [Note] WSREP: Starting applier thread 2 2019-11-07 23:48:04 0 [Note] WSREP: STATE EXCHANGE: got state msg: f3351b1a-01e2-11ea-856a-2691a814bc36 from 2 (Galeradbprod02) 2019-11-07 23:48:04 0 [Note] WSREP: STATE EXCHANGE: got state msg: f3351b1a-01e2-11ea-856a-2691a814bc36 from 4 (GaleraDBColo1) 2019-11-07 23:48:04 0 [Note] WSREP: STATE EXCHANGE: got state msg: f3351b1a-01e2-11ea-856a-2691a814bc36 from 1 (FLLGaleraDBColo2) 2019-11-07 23:48:04 0 [Note] WSREP: STATE EXCHANGE: got state msg: f3351b1a-01e2-11ea-856a-2691a814bc36 from 3 (HQGaleraDBProd1) 2019-11-07 23:48:04 0 [Note] WSREP: Quorum results: version = 5, component = PRIMARY, conf_id = 23, members = 4/5 (joined/total), act_id = 107827529, last_appl. = 107827513, protocols = 1/10/4 (gcs/repl/appl), vote policy= 0, group UUID = 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 2019-11-07 23:48:04 0 [Note] WSREP: Writing down CC checksum: 3861ff3a 00df0622 06c5be37 e37f3d9e at offset 384 2019-11-07 23:48:04 0 [Note] WSREP: Flow-control interval: [286, 358] 2019-11-07 23:48:04 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-07 23:48:04 0 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 107827530) 2019-11-07 23:48:04 2 [Note] WSREP: ####### processing CC 107827530, local, ordered 2019-11-07 23:48:04 2 [Note] WSREP: ####### drain monitors upto -1 2019-11-07 23:48:04 2 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-07 23:48:04 2 [Note] WSREP: ####### My UUID: f19a4d7f-01e2-11ea-95f3-5b60e72438da 2019-11-07 23:48:04 2 [Note] WSREP: Server HQGaleraDBProd1 connected to cluster at position 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107827530 with ID f19a4d7f-01e2-11ea-95f3-5b60e72438da 2019-11-07 23:48:04 2 [Note] WSREP: Server status change disconnected -> connected 2019-11-07 23:48:04 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-07 23:48:04 2 [Note] WSREP: State transfer required: Group state: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107827530 Local state: 00000000-0000-0000-0000-000000000000:-1 2019-11-07 23:48:04 2 [Note] WSREP: Server status change connected -> joiner 2019-11-07 23:48:04 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-07 23:48:04 0 [Note] WSREP: Running: 'wsrep_sst_mariabackup --role 'joiner' --address '10.1.30.60' --datadir '/var/lib/mysql/' --parent '30309' --mysqld-args --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --wsrep_on=ON --wsrep_provider=/usr/lib64/galera-4/libgalera_smm.so --log-error=/var/lib/mysql/error.log --pid-file=/var/lib/mysql/HQGaleraDBProd1.pid --wsrep_start_position=00000000-0000-0000-0000-000000000000:-1' 2019-11-07 23:48:05 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer f19a4d7f with addr ssl://10.1.30.60:4567 timed out, no messages seen in PT3S 2019-11-07 23:48:05 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off WSREP_SST: [INFO] Streaming with xbstream (20191107 23:48:05.303) WSREP_SST: [INFO] Using socat as streamer (20191107 23:48:05.311) WSREP_SST: [INFO] Evaluating timeout -k 110 100 socat -u TCP-LISTEN:4444,reuseaddr stdio | mbstream -x; RC=( ${PIPESTATUS[@]} ) (20191107 23:48:05.417) 2019-11-07 23:48:05 2 [Note] WSREP: Prepared SST request: mariabackup|10.1.30.60:4444/xtrabackup_sst//1 2019-11-07 23:48:05 2 [Note] WSREP: Cert index reset to 00000000-0000-0000-0000-000000000000:0 (proto: 10), state transfer needed: yes 2019-11-07 23:48:05 0 [Note] WSREP: Service thread queue flushed. 2019-11-07 23:48:05 2 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:0, protocol version: 5 2019-11-07 23:48:05 2 [Note] WSREP: ####### IST uuid:00000000-0000-0000-0000-000000000000 f: 0, l: 107827530, STRv: 3 2019-11-07 23:48:05 2 [Note] WSREP: IST receiver addr using ssl://10.1.30.60:4568 2019-11-07 23:48:05 2 [Note] WSREP: IST receiver using ssl 2019-11-07 23:48:05 2 [Note] WSREP: Prepared IST receiver for 0-107827530, listening at: ssl://10.1.30.60:4568 2019-11-07 23:48:05 0 [Note] WSREP: Member 3.0 (HQGaleraDBProd1) requested state transfer from '*any*'. Selected 2.0 (Galeradbprod02)(SYNCED) as donor. 2019-11-07 23:48:05 0 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 107827530) 2019-11-07 23:48:05 2 [Note] WSREP: Requesting state transfer: success, donor: 2 2019-11-07 23:48:05 2 [Note] WSREP: Resetting GCache seqno map due to different histories. 2019-11-07 23:48:05 2 [Note] WSREP: GCache history reset: old(00000000-0000-0000-0000-000000000000:0 -> 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107827530 WSREP_SST: [INFO] Proceeding with SST (20191107 23:48:06.605) WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:4444,reuseaddr stdio | mbstream -x; RC=( ${PIPESTATUS[@]} ) (20191107 23:48:06.607) WSREP_SST: [INFO] Cleaning the existing datadir and innodb-data/log directories (20191107 23:48:06.612) removed ‘/var/lib/mysql/aria_log_control’ removed ‘/var/lib/mysql/aria_log.00000001’ removed ‘/var/lib/mysql/ibdata1’ removed ‘/var/lib/mysql/ib_logfile1’ removed ‘/var/lib/mysql/mysql.sock’ removed ‘/var/lib/mysql/wsrep_recovery.ok’ WSREP_SST: [INFO] Waiting for SST streaming to complete! (20191107 23:48:06.659) 2019-11-08 0:25:45 0 [Note] WSREP: 2.0 (Galeradbprod02): State transfer to 3.0 (HQGaleraDBProd1) complete. 2019-11-08 0:25:45 0 [Note] WSREP: Member 2.0 (Galeradbprod02) synced with group. WSREP_SST: [INFO] Preparing the backup at /var/lib/mysql//.sst (20191108 00:25:45.865) WSREP_SST: [INFO] Evaluating /usr/bin/mariabackup --innobackupex $INNOEXTRA --apply-log $rebuildcmd ${DATA} &> /var/lib/mysql//mariabackup.prepare.log (20191108 00:25:45.873) WSREP_SST: [INFO] Moving the backup to /var/lib/mysql/ (20191108 00:28:28.667) WSREP_SST: [INFO] Evaluating /usr/bin/mariabackup --innobackupex --move-back --force-non-empty-directories ${DATA} &> /var/lib/mysql//mariabackup.move.log (20191108 00:28:28.674) WSREP_SST: [INFO] Move successful, removing /var/lib/mysql//.sst (20191108 00:28:29.333) WSREP_SST: [INFO] Galera co-ords from recovery: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829218 (20191108 00:28:29.352) WSREP_SST: [INFO] Total time on joiner: 0 seconds (20191108 00:28:29.361) WSREP_SST: [INFO] Removing the sst_in_progress file (20191108 00:28:29.368) 2019-11-08 0:28:29 3 [Note] WSREP: SST received 2019-11-08 0:28:29 3 [Note] WSREP: Server status change joiner -> initializing 2019-11-08 0:28:29 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:28:29 0 [Warning] InnoDB: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead; Please refer to https://mariadb.com/kb/en/library/set-transaction/ 2019-11-08 0:28:29 0 [Note] InnoDB: Using Linux native AIO 2019-11-08 0:28:29 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2019-11-08 0:28:29 0 [Note] InnoDB: Uses event mutexes 2019-11-08 0:28:29 0 [Note] InnoDB: Compressed tables use zlib 1.2.7 2019-11-08 0:28:29 0 [Note] InnoDB: Number of pools: 1 2019-11-08 0:28:29 0 [Note] InnoDB: Using SSE2 crc32 instructions 2019-11-08 0:28:29 0 [Note] InnoDB: Initializing buffer pool, total size = 20G, instances = 20, chunk size = 128M 2019-11-08 0:28:31 0 [Note] InnoDB: Completed initialization of buffer pool 2019-11-08 0:28:31 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2019-11-08 0:28:31 0 [Note] InnoDB: Setting log file ./ib_logfile101 size to 2147483648 bytes 2019-11-08 0:28:31 0 [Note] InnoDB: Setting log file ./ib_logfile1 size to 2147483648 bytes 2019-11-08 0:28:31 0 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0 2019-11-08 0:28:31 0 [Note] InnoDB: New log files created, LSN=5412040778724 2019-11-08 0:28:32 0 [Note] InnoDB: 4 transaction(s) which must be rolled back or cleaned up in total 4 row operations to undo 2019-11-08 0:28:32 0 [Note] InnoDB: Trx id counter is 1161492006 2019-11-08 0:28:33 0 [Note] InnoDB: 128 out of 128 rollback segments are active. 2019-11-08 0:28:33 0 [Note] InnoDB: Starting in background the rollback of recovered transactions 2019-11-08 0:28:33 0 [Note] InnoDB: Creating shared tablespace for temporary tables 2019-11-08 0:28:33 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2019-11-08 0:28:33 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2019-11-08 0:28:33 0 [Note] InnoDB: Waiting for purge to start 2019-11-08 0:28:33 0 [Note] InnoDB: 10.4.8 started; log sequence number 5412040778764; transaction id 1161492008 2019-11-08 0:28:33 0 [Note] Plugin 'FEEDBACK' is disabled. 191108 0:28:33 server_audit: MariaDB Audit Plugin version 1.4.8 STARTED. 2019-11-08 0:28:33 0 [Note] Server socket created on IP: '0.0.0.0'. 2019-11-08 0:28:33 0 [Warning] 'user' entry 'root@galeradb1' ignored in --skip-name-resolve mode. 2019-11-08 0:28:33 0 [Warning] 'user' entry 'ghwdbadmin@galeradb1' ignored in --skip-name-resolve mode. 2019-11-08 0:28:33 0 [Warning] 'db' entry 'TESTIMONIAL ghwdbadmin@galeradb1' ignored in --skip-name-resolve mode. 2019-11-08 0:28:33 0 [Warning] 'proxies_priv' entry '@% root@galeradb1' ignored in --skip-name-resolve mode. 2019-11-08 0:28:33 0 [Note] WSREP: wsrep_init_schema_and_SR (nil) 2019-11-08 0:28:33 0 [Note] WSREP: Server initialized 2019-11-08 0:28:33 0 [Note] WSREP: Server status change initializing -> initialized 2019-11-08 0:28:33 0 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:28:33 3 [Note] WSREP: Server status change initialized -> joined 2019-11-08 0:28:33 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:28:33 3 [Note] WSREP: Recovered position from storage: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829218 2019-11-08 0:28:33 11 [Note] WSREP: Starting applier thread 11 2019-11-08 0:28:33 12 [Note] WSREP: Starting applier thread 12 2019-11-08 0:28:33 13 [Note] WSREP: Starting applier thread 13 2019-11-08 0:28:33 15 [Note] WSREP: Starting applier thread 15 2019-11-08 0:28:33 10 [Note] WSREP: Starting applier thread 10 2019-11-08 0:28:33 16 [Note] WSREP: Starting applier thread 16 2019-11-08 0:28:33 17 [Note] WSREP: Starting applier thread 17 2019-11-08 0:28:33 14 [Note] WSREP: Starting applier thread 14 2019-11-08 0:28:33 18 [Note] WSREP: Starting applier thread 18 2019-11-08 0:28:33 19 [Note] WSREP: Starting applier thread 19 2019-11-08 0:28:33 21 [Note] WSREP: Starting applier thread 21 2019-11-08 0:28:33 20 [Note] WSREP: Starting applier thread 20 2019-11-08 0:28:33 22 [Note] WSREP: Starting applier thread 22 2019-11-08 0:28:33 24 [Note] WSREP: Starting applier thread 24 2019-11-08 0:28:33 27 [Note] WSREP: Starting applier thread 27 2019-11-08 0:28:33 25 [Note] WSREP: Starting applier thread 25 2019-11-08 0:28:33 23 [Note] WSREP: Starting applier thread 23 2019-11-08 0:28:33 26 [Note] WSREP: Starting applier thread 26 2019-11-08 0:28:33 29 [Note] WSREP: Starting applier thread 29 2019-11-08 0:28:33 28 [Note] WSREP: Starting applier thread 28 2019-11-08 0:28:33 30 [Note] WSREP: Starting applier thread 30 2019-11-08 0:28:33 32 [Note] WSREP: Starting applier thread 32 2019-11-08 0:28:33 34 [Note] WSREP: Starting applier thread 34 2019-11-08 0:28:33 31 [Note] WSREP: Starting applier thread 31 2019-11-08 0:28:33 33 [Note] WSREP: Starting applier thread 33 2019-11-08 0:28:33 37 [Note] WSREP: Starting applier thread 37 2019-11-08 0:28:33 35 [Note] WSREP: Starting applier thread 35 2019-11-08 0:28:33 36 [Note] WSREP: Starting applier thread 36 2019-11-08 0:28:33 38 [Note] WSREP: Starting applier thread 38 2019-11-08 0:28:33 40 [Note] WSREP: Starting applier thread 40 2019-11-08 0:28:33 39 [Note] WSREP: Starting applier thread 39 2019-11-08 0:28:33 0 [Note] Reading of all Master_info entries succeeded 2019-11-08 0:28:33 3 [Note] WSREP: Recovered view from SST: id: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107827530 status: primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 3 members(5): 0: 45421c9c-01d0-11ea-870b-5a3a6780e8b6, garb 1: c108bb1a-01a9-11ea-9105-3730ecd6c800, FLLGaleraDBColo2 2: d929f93e-01cd-11ea-87ce-0f169e037fd4, Galeradbprod02 3: f19a4d7f-01e2-11ea-95f3-5b60e72438da, HQGaleraDBProd1 4: fd37624f-01ca-11ea-9c93-77cc6d9c9f87, GaleraDBColo1 2019-11-08 0:28:33 0 [Note] Added new Master_info '' to hash table 2019-11-08 0:28:33 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:28:33 0 [Note] /usr/sbin/mysqld: ready for connections. Version: '10.4.8-MariaDB-log' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server 2019-11-08 0:28:33 43 [Note] WSREP: Recovered cluster id 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 2019-11-08 0:28:33 3 [Note] WSREP: SST received: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829218 2019-11-08 0:28:33 2 [Note] WSREP: Installed new state from SST: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829218 2019-11-08 0:28:33 2 [Note] WSREP: Cert. index preload up to 107829218 2019-11-08 0:28:33 0 [Note] WSREP: ####### IST applying starts with 107829219 2019-11-08 0:28:33 0 [Note] WSREP: ####### IST current seqno initialized to 107827514 2019-11-08 0:28:33 0 [Note] WSREP: Receiving IST... 0.0% ( 0/17 events) complete. 2019-11-08 0:28:33 0 [Note] WSREP: IST preload starting at 107827514 2019-11-08 0:28:33 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 0:28:33 0 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:107827513, protocol version: 5 2019-11-08 0:28:33 0 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-08 0:28:33 0 [Note] WSREP: ####### Adjusting cert position: 107827529 -> 107827530 2019-11-08 0:28:33 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 0:28:33 0 [Note] WSREP: Lowest cert indnex boundary for CC from preload: 107827514 2019-11-08 0:28:33 0 [Note] WSREP: Min available from gcache for CC from preload: 107827514 2019-11-08 0:28:33 0 [Note] WSREP: Receiving IST...100.0% (17/17 events) complete. 2019-11-08 0:28:33 2 [Note] WSREP: Cert. index preloaded up to 107827530 2019-11-08 0:28:33 0 [Note] InnoDB: Rolled back recovered transaction 1161465564 2019-11-08 0:28:33 0 [Note] InnoDB: Rolled back recovered transaction 1161465559 2019-11-08 0:28:33 0 [Note] InnoDB: Rolled back recovered transaction 1161465543 2019-11-08 0:28:33 0 [Note] InnoDB: Rolled back recovered transaction 1161465575 2019-11-08 0:28:33 0 [Note] InnoDB: Rollback of non-prepared transactions completed 2019-11-08 0:32:23 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 45421c9c with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:32:23 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: ssl://172.25.1.10:4567 2019-11-08 0:32:24 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') reconnecting to 45421c9c (ssl://172.25.1.10:4567), attempt 0 2019-11-08 0:32:25 0 [Note] WSREP: 3.0 (HQGaleraDBProd1): State transfer from 2.0 (Galeradbprod02) complete. 2019-11-08 0:32:25 0 [Note] WSREP: Shifting JOINER -> JOINED (TO: 107829219) 2019-11-08 0:32:26 0 [Note] WSREP: Member 2.0 (Galeradbprod02) desyncs itself from group 2019-11-08 0:32:27 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 00000000 with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:32:31 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 00000000 with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:32:31 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.20.11.46:56274 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:32:31 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to c108bb1a ssl://10.20.11.46:4567 2019-11-08 0:32:35 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 00000000 with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:32:35 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') reconnecting to c108bb1a (ssl://10.20.11.46:4567), attempt 0 2019-11-08 0:32:35 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.20.11.46:4567 local endpoint ssl://10.1.30.60:41052 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:32:35 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to c108bb1a ssl://10.20.11.46:4567 2019-11-08 0:32:39 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 00000000 with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:32:43 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 00000000 with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:32:43 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49334 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:32:43 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:32:46 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off 2019-11-08 0:32:53 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:32:53 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:32:53 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:32:53 0 [Note] WSREP: Member 3.0 (HQGaleraDBProd1) synced with group. 2019-11-08 0:32:53 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 107829233) 2019-11-08 0:32:53 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:33:01 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:33:01 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:33:01 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:33:01 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:33:01 0 [Note] WSREP: view(view_id(PRIM,c108bb1a,95) memb { c108bb1a,1 d929f93e,0 f19a4d7f,0 fd37624f,1 } joined { } left { } partitioned { 45421c9c,2 }) 2019-11-08 0:33:01 0 [Note] WSREP: save pc into disk 2019-11-08 0:33:01 0 [Note] WSREP: forgetting 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:33:01 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 4 2019-11-08 0:33:01 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2019-11-08 0:33:01 0 [Warning] WSREP: Last Applied Action message in non-primary configuration from member 0 2019-11-08 0:33:01 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 3ad3459d-01e9-11ea-870b-16ef58d9f1cc 2019-11-08 0:33:01 0 [Note] WSREP: STATE EXCHANGE: got state msg: 3ad3459d-01e9-11ea-870b-16ef58d9f1cc from 0 (FLLGaleraDBColo2) 2019-11-08 0:33:01 0 [Note] WSREP: STATE EXCHANGE: got state msg: 3ad3459d-01e9-11ea-870b-16ef58d9f1cc from 1 (Galeradbprod02) 2019-11-08 0:33:01 0 [Note] WSREP: STATE EXCHANGE: got state msg: 3ad3459d-01e9-11ea-870b-16ef58d9f1cc from 2 (HQGaleraDBProd1) 2019-11-08 0:33:01 0 [Note] WSREP: STATE EXCHANGE: got state msg: 3ad3459d-01e9-11ea-870b-16ef58d9f1cc from 3 (GaleraDBColo1) 2019-11-08 0:33:01 0 [Note] WSREP: Quorum results: version = 5, component = PRIMARY, conf_id = 24, members = 4/4 (joined/total), act_id = 107829233, last_appl. = 107829173, protocols = 1/10/4 (gcs/repl/appl), vote policy= 0, group UUID = 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 2019-11-08 0:33:01 0 [Note] WSREP: Writing down CC checksum: 21269cd0 9e30a6f1 0f1012a4 b4880b58 at offset 336 2019-11-08 0:33:01 0 [Note] WSREP: Flow-control interval: [256, 320] 2019-11-08 0:33:01 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 0:33:01 17 [Note] WSREP: ####### processing CC 107829234, local, ordered 2019-11-08 0:33:01 0 [Note] WSREP: Member 1.0 (Galeradbprod02) resyncs itself to group 2019-11-08 0:33:01 0 [Note] WSREP: Member 1.0 (Galeradbprod02) synced with group. 2019-11-08 0:33:01 0 [Warning] WSREP: Protocol violation. JOIN message sender 1.0 (Galeradbprod02) is not in state transfer (SYNCED). Message ignored. 2019-11-08 0:33:02 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49336 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:33:02 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:33:02 0 [Warning] WSREP: discarding established (time wait) 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:33:04 0 [Note] WSREP: cleaning up 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:33:07 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49338 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:33:07 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:33:07 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 2019-11-08 0:33:10 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off 2019-11-08 0:33:20 37 [Note] WSREP: Server HQGaleraDBProd1 synced with group 2019-11-08 0:33:20 37 [Note] WSREP: Server status change joined -> synced 2019-11-08 0:33:20 37 [Note] WSREP: Synchronized with group, ready for connections 2019-11-08 0:33:20 37 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:33:20 17 [Note] WSREP: ####### drain monitors upto 107829233 2019-11-08 0:33:20 17 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-08 0:33:20 17 [Note] WSREP: ####### My UUID: f19a4d7f-01e2-11ea-95f3-5b60e72438da 2019-11-08 0:33:20 17 [Note] WSREP: ####### ST not required 2019-11-08 0:33:20 17 [Note] WSREP: Skipping cert index reset 2019-11-08 0:33:20 17 [Note] WSREP: ####### Adjusting cert position: 107829233 -> 107829234 2019-11-08 0:33:20 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 0:33:20 17 [Note] WSREP: ####### Setting monitor position to 107829234 2019-11-08 0:33:20 17 [Note] WSREP: Lowest cert indnex boundary for CC from group: 107829174 2019-11-08 0:33:20 17 [Note] WSREP: Min available from gcache for CC from group: 107827514 2019-11-08 0:33:20 17 [Note] WSREP: ================================================ View: id: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829234 status: primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 2 members(4): 0: c108bb1a-01a9-11ea-9105-3730ecd6c800, FLLGaleraDBColo2 1: d929f93e-01cd-11ea-87ce-0f169e037fd4, Galeradbprod02 2: f19a4d7f-01e2-11ea-95f3-5b60e72438da, HQGaleraDBProd1 3: fd37624f-01ca-11ea-9c93-77cc6d9c9f87, GaleraDBColo1 ================================================= 2019-11-08 0:33:20 17 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:35:17 0 [Note] WSREP: Member 1.0 (Galeradbprod02) desyncs itself from group 2019-11-08 0:35:18 0 [Note] WSREP: Member 1.0 (Galeradbprod02) resyncs itself to group 2019-11-08 0:35:18 0 [Note] WSREP: Member 1.0 (Galeradbprod02) synced with group. 2019-11-08 0:36:04 0 [Note] WSREP: declaring 45421c9c at ssl://172.25.1.10:4567 stable 2019-11-08 0:36:04 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:36:04 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:36:04 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:36:05 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:36:09 0 [Note] WSREP: view(view_id(PRIM,45421c9c,126) memb { 45421c9c,2 c108bb1a,1 d929f93e,0 f19a4d7f,0 fd37624f,1 } joined { } left { } partitioned { }) 2019-11-08 0:36:09 0 [Note] WSREP: save pc into disk 2019-11-08 0:36:09 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 3, memb_num = 5 2019-11-08 0:36:09 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2019-11-08 0:36:12 0 [Note] WSREP: STATE EXCHANGE: sent state msg: ab076c33-01e9-11ea-a213-7bd8e596440e 2019-11-08 0:36:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: ab076c33-01e9-11ea-a213-7bd8e596440e from 1 (FLLGaleraDBColo2) 2019-11-08 0:36:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: ab076c33-01e9-11ea-a213-7bd8e596440e from 2 (Galeradbprod02) 2019-11-08 0:36:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: ab076c33-01e9-11ea-a213-7bd8e596440e from 4 (GaleraDBColo1) 2019-11-08 0:36:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: ab076c33-01e9-11ea-a213-7bd8e596440e from 3 (HQGaleraDBProd1) 2019-11-08 0:36:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: ab076c33-01e9-11ea-a213-7bd8e596440e from 0 (garb) 2019-11-08 0:36:14 0 [Note] WSREP: Quorum results: version = 5, component = PRIMARY, conf_id = 25, members = 4/5 (joined/total), act_id = 107829361, last_appl. = 107829301, protocols = 1/10/4 (gcs/repl/appl), vote policy= 0, group UUID = 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 2019-11-08 0:36:14 0 [Note] WSREP: Writing down CC checksum: 319b4d24 4b71bbb3 ce4226f4 0289b4a3 at offset 384 2019-11-08 0:36:14 0 [Note] WSREP: Flow-control interval: [286, 358] 2019-11-08 0:36:14 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 0:36:14 21 [Note] WSREP: ####### processing CC 107829362, local, ordered 2019-11-08 0:36:14 21 [Note] WSREP: ####### drain monitors upto 107829361 2019-11-08 0:36:14 21 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-08 0:36:14 21 [Note] WSREP: ####### My UUID: f19a4d7f-01e2-11ea-95f3-5b60e72438da 2019-11-08 0:36:14 21 [Note] WSREP: ####### ST not required 2019-11-08 0:36:14 21 [Note] WSREP: Skipping cert index reset 2019-11-08 0:36:14 21 [Note] WSREP: ####### Adjusting cert position: 107829361 -> 107829362 2019-11-08 0:36:14 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 0:36:14 21 [Note] WSREP: ####### Setting monitor position to 107829362 2019-11-08 0:36:14 21 [Note] WSREP: Lowest cert indnex boundary for CC from group: 107829302 2019-11-08 0:36:14 21 [Note] WSREP: Min available from gcache for CC from group: 107827514 2019-11-08 0:36:14 21 [Note] WSREP: ================================================ View: id: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829362 status: primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 3 members(5): 0: 45421c9c-01d0-11ea-870b-5a3a6780e8b6, garb 1: c108bb1a-01a9-11ea-9105-3730ecd6c800, FLLGaleraDBColo2 2: d929f93e-01cd-11ea-87ce-0f169e037fd4, Galeradbprod02 3: f19a4d7f-01e2-11ea-95f3-5b60e72438da, HQGaleraDBProd1 4: fd37624f-01ca-11ea-9c93-77cc6d9c9f87, GaleraDBColo1 ================================================= 2019-11-08 0:36:14 21 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:36:17 0 [Note] WSREP: Member 0.2 (garb) requested state transfer from '*any*'. Selected 4.1 (GaleraDBColo1)(SYNCED) as donor. 2019-11-08 0:36:19 0 [Note] WSREP: 4.1 (GaleraDBColo1): State transfer to 0.2 (garb) complete. 2019-11-08 0:36:19 0 [Note] WSREP: 0.2 (garb): State transfer from 4.1 (GaleraDBColo1) complete. 2019-11-08 0:36:20 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 45421c9c with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:36:20 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: ssl://172.25.1.10:4567 2019-11-08 0:36:21 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') reconnecting to 45421c9c (ssl://172.25.1.10:4567), attempt 0 2019-11-08 0:36:24 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 00000000 with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:36:28 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 00000000 with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:36:32 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 00000000 with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:36:35 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49344 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:36:35 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:36:36 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 00000000 with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:36:38 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 45421c9c with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:36:39 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') reconnecting to 45421c9c (ssl://172.25.1.10:4567), attempt 0 2019-11-08 0:36:42 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 00000000 with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:36:46 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 00000000 with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:36:50 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 00000000 with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:36:54 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 00000000 with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:36:58 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 00000000 with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:36:59 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49346 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:36:59 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:37:01 0 [Note] WSREP: view(view_id(NON_PRIM,45421c9c,126) memb { f19a4d7f,0 } joined { } left { } partitioned { 45421c9c,2 c108bb1a,1 d929f93e,0 fd37624f,1 }) 2019-11-08 0:37:01 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 2019-11-08 0:37:01 0 [Note] WSREP: Writing down CC checksum: e9ba6051 04734b8b a8221c05 95993950 at offset 128 2019-11-08 0:37:01 0 [Note] WSREP: Flow-control interval: [128, 160] 2019-11-08 0:37:01 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 0:37:01 0 [Note] WSREP: Received NON-PRIMARY. 2019-11-08 0:37:01 0 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 107829366) 2019-11-08 0:37:01 24 [Note] WSREP: ####### processing CC -1, local, ordered 2019-11-08 0:37:01 24 [Note] WSREP: ####### drain monitors upto 107829366 2019-11-08 0:37:01 24 [Note] WSREP: ####### My UUID: f19a4d7f-01e2-11ea-95f3-5b60e72438da 2019-11-08 0:37:01 24 [Note] WSREP: ####### ST not required 2019-11-08 0:37:01 24 [Note] WSREP: ================================================ View: id: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:-1 status: non-primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 0 members(1): 0: f19a4d7f-01e2-11ea-95f3-5b60e72438da, HQGaleraDBProd1 ================================================= 2019-11-08 0:37:01 24 [Note] WSREP: Non-primary view 2019-11-08 0:37:01 24 [Note] WSREP: Server status change synced -> connected 2019-11-08 0:37:01 24 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:37:01 24 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:37:01 0 [Note] WSREP: view(view_id(NON_PRIM,f19a4d7f,127) memb { f19a4d7f,0 } joined { } left { } partitioned { 45421c9c,2 c108bb1a,1 d929f93e,0 fd37624f,1 }) 2019-11-08 0:37:01 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 2019-11-08 0:37:01 0 [Note] WSREP: Writing down CC checksum: e9ba6051 04734b8b a8221c05 95993950 at offset 128 2019-11-08 0:37:01 0 [Note] WSREP: Flow-control interval: [128, 160] 2019-11-08 0:37:01 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 0:37:01 0 [Note] WSREP: Received NON-PRIMARY. 2019-11-08 0:37:01 24 [Note] WSREP: ####### processing CC -1, local, ordered 2019-11-08 0:37:01 24 [Note] WSREP: ####### drain monitors upto 107829366 2019-11-08 0:37:01 24 [Note] WSREP: ####### My UUID: f19a4d7f-01e2-11ea-95f3-5b60e72438da 2019-11-08 0:37:01 24 [Note] WSREP: ####### ST not required 2019-11-08 0:37:01 24 [Note] WSREP: ================================================ View: id: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:-1 status: non-primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 0 members(1): 0: f19a4d7f-01e2-11ea-95f3-5b60e72438da, HQGaleraDBProd1 ================================================= 2019-11-08 0:37:01 24 [Note] WSREP: Non-primary view 2019-11-08 0:37:01 24 [Note] WSREP: Server status change connected -> connected 2019-11-08 0:37:01 24 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:37:01 24 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:37:02 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off 2019-11-08 0:37:04 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer d929f93e with addr ssl://10.1.30.61:4567 timed out, no messages seen in PT3S 2019-11-08 0:37:04 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer fd37624f with addr ssl://10.20.11.45:4567 timed out, no messages seen in PT3S 2019-11-08 0:37:04 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer c108bb1a with addr ssl://10.20.11.46:4567 timed out, no messages seen in PT3S 2019-11-08 0:37:04 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: ssl://10.1.30.61:4567 ssl://10.20.11.45:4567 ssl://10.20.11.46:4567 2019-11-08 0:37:05 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') reconnecting to d929f93e (ssl://10.1.30.61:4567), attempt 0 2019-11-08 0:37:05 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.1.30.61:4567 local endpoint ssl://10.1.30.60:42860 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:37:05 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to d929f93e ssl://10.1.30.61:4567 2019-11-08 0:37:05 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') reconnecting to fd37624f (ssl://10.20.11.45:4567), attempt 0 2019-11-08 0:37:05 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') reconnecting to c108bb1a (ssl://10.20.11.46:4567), attempt 0 2019-11-08 0:37:05 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.20.11.45:4567 local endpoint ssl://10.1.30.60:43488 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:37:05 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to fd37624f ssl://10.20.11.45:4567 2019-11-08 0:37:05 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.20.11.46:4567 local endpoint ssl://10.1.30.60:41098 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:37:05 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to c108bb1a ssl://10.20.11.46:4567 2019-11-08 0:37:07 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:37:07 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:37:07 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:37:08 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off 2019-11-08 0:37:10 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:37:10 0 [Note] WSREP: view(view_id(PRIM,c108bb1a,128) memb { c108bb1a,1 d929f93e,0 f19a4d7f,0 fd37624f,1 } joined { } left { } partitioned { 45421c9c,2 }) 2019-11-08 0:37:10 0 [Note] WSREP: save pc into disk 2019-11-08 0:37:10 0 [Note] WSREP: forgetting 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:37:10 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 4 2019-11-08 0:37:10 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2019-11-08 0:37:10 0 [Note] WSREP: STATE EXCHANGE: sent state msg: cf2ee1f2-01e9-11ea-bf82-963f997fb7da 2019-11-08 0:37:10 0 [Note] WSREP: STATE EXCHANGE: got state msg: cf2ee1f2-01e9-11ea-bf82-963f997fb7da from 0 (FLLGaleraDBColo2) 2019-11-08 0:37:10 0 [Note] WSREP: STATE EXCHANGE: got state msg: cf2ee1f2-01e9-11ea-bf82-963f997fb7da from 2 (HQGaleraDBProd1) 2019-11-08 0:37:10 0 [Note] WSREP: STATE EXCHANGE: got state msg: cf2ee1f2-01e9-11ea-bf82-963f997fb7da from 3 (GaleraDBColo1) 2019-11-08 0:37:10 0 [Note] WSREP: STATE EXCHANGE: got state msg: cf2ee1f2-01e9-11ea-bf82-963f997fb7da from 1 (Galeradbprod02) 2019-11-08 0:37:10 0 [Note] WSREP: Quorum results: version = 5, component = PRIMARY, conf_id = 27, members = 3/4 (joined/total), act_id = 107829375, last_appl. = 107829328, protocols = 1/10/4 (gcs/repl/appl), vote policy= 0, group UUID = 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 2019-11-08 0:37:10 0 [Note] WSREP: Writing down CC checksum: 2c5ab3be 439d0ab7 3db19d77 a109524d at offset 336 2019-11-08 0:37:10 0 [Note] WSREP: Flow-control interval: [256, 320] 2019-11-08 0:37:10 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 0:37:10 0 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 107829376) 2019-11-08 0:37:10 24 [Note] WSREP: ####### processing CC 107829376, local, ordered 2019-11-08 0:37:10 24 [Note] WSREP: ####### drain monitors upto 107829366 2019-11-08 0:37:10 24 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-08 0:37:10 24 [Note] WSREP: ####### My UUID: f19a4d7f-01e2-11ea-95f3-5b60e72438da 2019-11-08 0:37:10 24 [Note] WSREP: State transfer required: Group state: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829376 Local state: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829366 2019-11-08 0:37:10 24 [Note] WSREP: Server status change connected -> joiner 2019-11-08 0:37:10 24 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:37:10 24 [Note] WSREP: You have configured 'mariabackup' state snapshot transfer method which cannot be performed on a running server. Wsrep provider won't be able to fall back to it if other means of state transfer are unavailable. In that case you will need to restart the server. 2019-11-08 0:37:10 24 [Note] WSREP: Cert index reset to 00000000-0000-0000-0000-000000000000:0 (proto: 10), state transfer needed: yes 2019-11-08 0:37:10 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 0:37:10 24 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:0, protocol version: 5 2019-11-08 0:37:10 24 [Note] WSREP: ####### IST uuid:1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 f: 107829367, l: 107829376, STRv: 3 2019-11-08 0:37:10 24 [Note] WSREP: IST receiver addr using ssl://10.1.30.60:4568 2019-11-08 0:37:10 24 [Note] WSREP: IST receiver using ssl 2019-11-08 0:37:10 24 [Note] WSREP: Prepared IST receiver for 107829367-107829376, listening at: ssl://10.1.30.60:4568 2019-11-08 0:37:10 0 [Note] WSREP: Member 2.0 (HQGaleraDBProd1) requested state transfer from '*any*'. Selected 1.0 (Galeradbprod02)(SYNCED) as donor. 2019-11-08 0:37:10 0 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 107829378) 2019-11-08 0:37:10 24 [Note] WSREP: Requesting state transfer: success, donor: 1 2019-11-08 0:37:10 24 [Note] WSREP: Receiving IST: 10 writesets, seqnos 107829367-107829376 2019-11-08 0:37:11 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49348 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:37:11 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:37:11 0 [Warning] WSREP: discarding established (time wait) 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:37:13 0 [Note] WSREP: cleaning up 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:37:16 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49350 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:37:16 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:37:16 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 2019-11-08 0:37:19 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off 2019-11-08 0:37:58 0 [Note] WSREP: ####### IST applying starts with 107829367 2019-11-08 0:37:58 0 [Note] WSREP: ####### IST current seqno initialized to 107829302 2019-11-08 0:37:58 0 [Note] WSREP: Receiving IST... 0.0% ( 0/75 events) complete. 2019-11-08 0:37:58 0 [Note] WSREP: IST preload starting at 107829302 2019-11-08 0:37:58 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 0:37:58 0 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:107829301, protocol version: 5 2019-11-08 0:37:58 0 [Note] WSREP: 1.0 (Galeradbprod02): State transfer to 2.0 (HQGaleraDBProd1) complete. 2019-11-08 0:37:58 0 [Note] WSREP: Member 1.0 (Galeradbprod02) synced with group. 2019-11-08 0:37:58 0 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-08 0:37:58 0 [Note] WSREP: ####### Adjusting cert position: 107829361 -> 107829362 2019-11-08 0:37:58 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 0:37:58 0 [Note] WSREP: Lowest cert indnex boundary for CC from preload: 107829302 2019-11-08 0:37:58 0 [Note] WSREP: Min available from gcache for CC from preload: 107827514 2019-11-08 0:37:58 0 [Note] WSREP: ####### processing CC 107829368, from IST, ordered 2019-11-08 0:37:58 0 [Note] WSREP: ####### drain monitors upto 107829367 2019-11-08 0:40:08 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:40:08 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:40:08 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:40:08 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:40:39 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:40:39 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:40:39 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:40:39 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:40:39 0 [Note] WSREP: view(view_id(PRIM,c108bb1a,130) memb { c108bb1a,1 d929f93e,0 f19a4d7f,0 fd37624f,1 } joined { } left { } partitioned { }) 2019-11-08 0:40:39 0 [Note] WSREP: save pc into disk 2019-11-08 0:40:39 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 4 2019-11-08 0:40:39 0 [Note] WSREP: forgetting 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:40:39 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2019-11-08 0:40:39 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 4be890c7-01ea-11ea-8498-ba0a19045981 2019-11-08 0:40:39 0 [Note] WSREP: STATE EXCHANGE: got state msg: 4be890c7-01ea-11ea-8498-ba0a19045981 from 0 (FLLGaleraDBColo2) 2019-11-08 0:40:39 0 [Note] WSREP: STATE EXCHANGE: got state msg: 4be890c7-01ea-11ea-8498-ba0a19045981 from 3 (GaleraDBColo1) 2019-11-08 0:40:39 0 [Note] WSREP: STATE EXCHANGE: got state msg: 4be890c7-01ea-11ea-8498-ba0a19045981 from 1 (Galeradbprod02) 2019-11-08 0:40:39 0 [Note] WSREP: STATE EXCHANGE: got state msg: 4be890c7-01ea-11ea-8498-ba0a19045981 from 2 (HQGaleraDBProd1) 2019-11-08 0:40:39 0 [Note] WSREP: Quorum results: version = 5, component = PRIMARY, conf_id = 28, members = 3/4 (joined/total), act_id = 107829398, last_appl. = 107829378, protocols = 1/10/4 (gcs/repl/appl), vote policy= 0, group UUID = 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 2019-11-08 0:40:39 0 [Note] WSREP: Writing down CC checksum: 644c7a4c 987a10cb 893c2fab be2c1327 at offset 336 2019-11-08 0:40:39 0 [Note] WSREP: Flow-control interval: [256, 320] 2019-11-08 0:40:39 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 0:40:41 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49354 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:40:41 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:40:41 0 [Warning] WSREP: discarding established (time wait) 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:40:42 0 [Note] WSREP: cleaning up 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:40:45 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49358 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:40:45 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: ssl://172.25.1.10:4567 2019-11-08 0:40:45 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:40:48 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off 2019-11-08 0:41:16 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:41:16 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:41:16 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:41:16 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:41:46 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:41:46 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:41:46 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:41:46 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:42:16 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:42:16 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:42:16 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:42:16 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:42:47 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:42:47 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:42:47 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:42:47 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:43:18 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:43:18 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:43:18 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:43:18 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:43:48 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:43:48 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:43:48 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:43:48 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:43:48 0 [Note] WSREP: view(view_id(PRIM,c108bb1a,136) memb { c108bb1a,1 d929f93e,0 f19a4d7f,0 fd37624f,1 } joined { } left { } partitioned { }) 2019-11-08 0:43:48 0 [Note] WSREP: save pc into disk 2019-11-08 0:43:48 0 [Note] WSREP: forgetting 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:43:48 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 4 2019-11-08 0:43:48 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2019-11-08 0:43:48 0 [Note] WSREP: STATE EXCHANGE: sent state msg: bcb03405-01ea-11ea-9f85-5b9d81a1aba6 2019-11-08 0:43:48 0 [Note] WSREP: STATE EXCHANGE: got state msg: bcb03405-01ea-11ea-9f85-5b9d81a1aba6 from 0 (FLLGaleraDBColo2) 2019-11-08 0:43:48 0 [Note] WSREP: STATE EXCHANGE: got state msg: bcb03405-01ea-11ea-9f85-5b9d81a1aba6 from 2 (HQGaleraDBProd1) 2019-11-08 0:43:48 0 [Note] WSREP: STATE EXCHANGE: got state msg: bcb03405-01ea-11ea-9f85-5b9d81a1aba6 from 3 (GaleraDBColo1) 2019-11-08 0:43:48 0 [Note] WSREP: STATE EXCHANGE: got state msg: bcb03405-01ea-11ea-9f85-5b9d81a1aba6 from 1 (Galeradbprod02) 2019-11-08 0:43:48 0 [Note] WSREP: Quorum results: version = 5, component = PRIMARY, conf_id = 29, members = 3/4 (joined/total), act_id = 107829408, last_appl. = 107829378, protocols = 1/10/4 (gcs/repl/appl), vote policy= 0, group UUID = 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 2019-11-08 0:43:48 0 [Note] WSREP: Writing down CC checksum: 2db9e6c6 9d76db3b 7634d34f 03fe2cfc at offset 336 2019-11-08 0:43:48 0 [Note] WSREP: Flow-control interval: [256, 320] 2019-11-08 0:43:48 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 0:43:51 0 [Note] WSREP: cleaning up 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:43:53 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49364 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:43:53 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:43:53 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 2019-11-08 0:43:56 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off 2019-11-08 0:44:24 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:44:24 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:44:24 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:44:24 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:44:54 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:44:54 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:44:54 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:44:54 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:44:54 0 [Note] WSREP: view(view_id(PRIM,c108bb1a,138) memb { c108bb1a,1 d929f93e,0 f19a4d7f,0 fd37624f,1 } joined { } left { } partitioned { }) 2019-11-08 0:44:54 0 [Note] WSREP: save pc into disk 2019-11-08 0:44:54 0 [Note] WSREP: forgetting 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:44:54 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 4 2019-11-08 0:44:54 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2019-11-08 0:44:54 0 [Note] WSREP: STATE EXCHANGE: sent state msg: e418b791-01ea-11ea-b98c-0a556444fa7c 2019-11-08 0:44:54 0 [Note] WSREP: STATE EXCHANGE: got state msg: e418b791-01ea-11ea-b98c-0a556444fa7c from 0 (FLLGaleraDBColo2) 2019-11-08 0:44:54 0 [Note] WSREP: STATE EXCHANGE: got state msg: e418b791-01ea-11ea-b98c-0a556444fa7c from 1 (Galeradbprod02) 2019-11-08 0:44:54 0 [Note] WSREP: STATE EXCHANGE: got state msg: e418b791-01ea-11ea-b98c-0a556444fa7c from 2 (HQGaleraDBProd1) 2019-11-08 0:44:54 0 [Note] WSREP: STATE EXCHANGE: got state msg: e418b791-01ea-11ea-b98c-0a556444fa7c from 3 (GaleraDBColo1) 2019-11-08 0:44:54 0 [Note] WSREP: Quorum results: version = 5, component = PRIMARY, conf_id = 30, members = 3/4 (joined/total), act_id = 107829418, last_appl. = 107829378, protocols = 1/10/4 (gcs/repl/appl), vote policy= 0, group UUID = 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 2019-11-08 0:44:54 0 [Note] WSREP: Writing down CC checksum: 2d9219e4 244d2c5f b633e680 a47a002f at offset 336 2019-11-08 0:44:54 0 [Note] WSREP: Flow-control interval: [256, 320] 2019-11-08 0:44:54 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 0:44:57 0 [Note] WSREP: cleaning up 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:44:59 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49370 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:44:59 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:44:59 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 2019-11-08 0:45:02 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off 2019-11-08 0:45:04 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: ssl://10.20.11.45:4567 2019-11-08 0:45:05 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.20.11.45:57650 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:45:05 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to fd37624f ssl://10.20.11.45:4567 2019-11-08 0:45:08 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off 2019-11-08 0:45:29 0 [Note] WSREP: view(view_id(NON_PRIM,c108bb1a,138) memb { d929f93e,0 f19a4d7f,0 } joined { } left { } partitioned { c108bb1a,1 fd37624f,1 }) 2019-11-08 0:45:29 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:45:29 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 1, memb_num = 2 2019-11-08 0:45:29 0 [Note] WSREP: Writing down CC checksum: 682909fa 553ca194 26487987 99f3fa45 at offset 192 2019-11-08 0:45:29 0 [Note] WSREP: Flow-control interval: [181, 226] 2019-11-08 0:45:29 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 0:45:29 0 [Note] WSREP: Received NON-PRIMARY. 2019-11-08 0:45:29 0 [Note] WSREP: Shifting JOINER -> OPEN (TO: 107829436) 2019-11-08 0:45:29 0 [Note] WSREP: view(view_id(NON_PRIM,d929f93e,139) memb { d929f93e,0 f19a4d7f,0 } joined { } left { } partitioned { c108bb1a,1 fd37624f,1 }) 2019-11-08 0:45:29 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 1, memb_num = 2 2019-11-08 0:45:29 0 [Note] WSREP: Writing down CC checksum: 682909fa 553ca194 26487987 99f3fa45 at offset 192 2019-11-08 0:45:29 0 [Note] WSREP: Flow-control interval: [181, 226] 2019-11-08 0:45:29 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 0:45:29 0 [Note] WSREP: Received NON-PRIMARY. 2019-11-08 0:46:00 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:46:00 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:46:00 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:46:00 0 [Note] WSREP: re-bootstrapping prim from partitioned components 2019-11-08 0:46:31 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:46:31 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:46:31 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:46:31 0 [Note] WSREP: re-bootstrapping prim from partitioned components 2019-11-08 0:46:31 0 [Note] WSREP: view(view_id(PRIM,c108bb1a,141) memb { c108bb1a,1 d929f93e,0 f19a4d7f,0 fd37624f,1 } joined { } left { } partitioned { }) 2019-11-08 0:46:31 0 [Note] WSREP: save pc into disk 2019-11-08 0:46:31 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 4 2019-11-08 0:46:31 0 [Note] WSREP: forgetting 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:46:31 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2019-11-08 0:46:32 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49374 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:46:32 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:46:32 0 [Warning] WSREP: discarding established (time wait) 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:46:34 0 [Note] WSREP: cleaning up 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:46:35 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: ssl://172.25.1.10:4567 2019-11-08 0:46:37 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49378 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:46:37 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:46:39 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer 00000000 with addr ssl://172.25.1.10:4567 timed out, no messages seen in PT3S 2019-11-08 0:46:39 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:46:39 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:46:39 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:46:39 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:46:39 0 [Note] WSREP: view(view_id(PRIM,c108bb1a,142) memb { c108bb1a,1 d929f93e,0 f19a4d7f,0 fd37624f,1 } joined { } left { } partitioned { }) 2019-11-08 0:46:39 0 [Note] WSREP: save pc into disk 2019-11-08 0:46:39 0 [Note] WSREP: forgetting 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:46:39 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 1da1ed35-01eb-11ea-8126-63be8eabddac 2019-11-08 0:46:39 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 4 2019-11-08 0:46:39 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2019-11-08 0:46:39 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 22709db9-01eb-11ea-8f91-b66b44cfe0bf 2019-11-08 0:46:39 0 [Note] WSREP: STATE EXCHANGE: got state msg: 22709db9-01eb-11ea-8f91-b66b44cfe0bf from 0 (FLLGaleraDBColo2) 2019-11-08 0:46:39 0 [Note] WSREP: STATE EXCHANGE: got state msg: 22709db9-01eb-11ea-8f91-b66b44cfe0bf from 3 (GaleraDBColo1) 2019-11-08 0:46:39 0 [Note] WSREP: STATE EXCHANGE: got state msg: 22709db9-01eb-11ea-8f91-b66b44cfe0bf from 1 (Galeradbprod02) 2019-11-08 0:46:39 0 [Note] WSREP: STATE EXCHANGE: got state msg: 22709db9-01eb-11ea-8f91-b66b44cfe0bf from 2 (HQGaleraDBProd1) 2019-11-08 0:46:39 0 [Warning] WSREP: Quorum: No node with complete state: Version : 5 Flags : 0x3 Protocols : 1 / 10 / 4 State : NON-PRIMARY Desync count : 0 Prim state : SYNCED Prim UUID : e418b791-01ea-11ea-b98c-0a556444fa7c Prim seqno : 31 First seqno : 107829221 Last seqno : 107829436 Commit cut : 107829378 Last vote : -1.0 Vote policy : 0 Prim JOINED : 3 State UUID : 22709db9-01eb-11ea-8f91-b66b44cfe0bf Group UUID : 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 Name : 'FLLGaleraDBColo2' Incoming addr: 'AUTO' Version : 5 Flags : 0x2 Protocols : 1 / 10 / 4 State : NON-PRIMARY Desync count : 0 Prim state : SYNCED Prim UUID : e418b791-01ea-11ea-b98c-0a556444fa7c Prim seqno : 31 First seqno : 107818363 Last seqno : 107829436 Commit cut : 107829378 Last vote : -1.0 Vote policy : 0 Prim JOINED : 3 State UUID : 22709db9-01eb-11ea-8f91-b66b44cfe0bf Group UUID : 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 Name : 'Galeradbprod02' Incoming addr: 'AUTO' Version : 5 Flags : 00 Protocols : 1 / 10 / 4 State : NON-PRIMARY Desync count : 0 Prim state : JOINER Prim UUID : e418b791-01ea-11ea-b98c-0a556444fa7c Prim seqno : 31 First seqno : 107827514 Last seqno : 107829436 Commit cut : 0 Last vote : -1.0 Vote policy : 0 Prim JOINED : 3 State UUID : 22709db9-01eb-11ea-8f91-b66b44cfe0bf Group UUID : 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 Name : 'HQGaleraDBProd1' Incoming addr: 'AUTO' Version : 5 Flags : 0x2 Protocols : 1 / 10 / 4 State : NON-PRIMARY Desync count : 0 Prim state : SYNCED Prim UUID : e418b791-01ea-11ea-b98c-0a556444fa7c Prim seqno : 31 First seqno : 107820311 Last seqno : 107829436 Commit cut : 107829415 Last vote : -1.0 Vote policy : 0 Prim JOINED : 3 State UUID : 22709db9-01eb-11ea-8f91-b66b44cfe0bf Group UUID : 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 Name : 'GaleraDBColo1' Incoming addr: 'AUTO' 2019-11-08 0:46:39 0 [Note] WSREP: Full re-merge of primary e418b791-01ea-11ea-b98c-0a556444fa7c found: 3 of 3. 2019-11-08 0:46:39 0 [Note] WSREP: Quorum results: version = 5, component = PRIMARY, conf_id = 31, members = 3/4 (joined/total), act_id = 107829436, last_appl. = 107829378, protocols = 1/10/4 (gcs/repl/appl), vote policy= 0, group UUID = 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 2019-11-08 0:46:39 0 [Note] WSREP: Writing down CC checksum: ea338fb1 668777e8 ff0a76b7 0686529d at offset 336 2019-11-08 0:46:39 0 [Note] WSREP: Flow-control interval: [256, 320] 2019-11-08 0:46:39 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 0:46:39 0 [Note] WSREP: Restored state OPEN -> JOINER (107829437) 2019-11-08 0:46:40 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off 2019-11-08 0:46:40 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49380 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:46:40 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:46:40 0 [Warning] WSREP: discarding established (time wait) 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:46:42 0 [Note] WSREP: cleaning up 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:46:42 0 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-08 0:46:42 0 [Note] WSREP: ####### ST not required 2019-11-08 0:46:42 0 [Note] WSREP: Skipping cert index reset 2019-11-08 0:46:42 0 [Note] WSREP: ####### Adjusting cert position: 107829367 -> 107829368 2019-11-08 0:46:42 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 0:46:42 0 [Note] WSREP: ####### Setting monitor position to 107829368 2019-11-08 0:46:42 0 [Note] WSREP: Lowest cert indnex boundary for CC from group: 107829302 2019-11-08 0:46:42 0 [Note] WSREP: Min available from gcache for CC from group: 107827514 2019-11-08 0:46:42 10 [Note] WSREP: ================================================ View: id: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829368 status: primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: -1 members(3): 0: c108bb1a-01a9-11ea-9105-3730ecd6c800, FLLGaleraDBColo2 1: d929f93e-01cd-11ea-87ce-0f169e037fd4, Galeradbprod02 2: fd37624f-01ca-11ea-9c93-77cc6d9c9f87, GaleraDBColo1 ================================================= 2019-11-08 0:46:42 10 [Note] WSREP: Server status change joiner -> joined 2019-11-08 0:46:42 10 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:46:42 10 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:46:42 0 [Note] WSREP: ####### processing CC 107829376, from IST, ordered 2019-11-08 0:46:42 0 [Note] WSREP: ####### drain monitors upto 107829375 2019-11-08 0:46:45 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49384 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:46:45 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:46:45 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 2019-11-08 0:46:48 0 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-08 0:46:48 0 [Note] WSREP: ####### ST not required 2019-11-08 0:46:48 0 [Note] WSREP: Skipping cert index reset 2019-11-08 0:46:48 0 [Note] WSREP: ####### Adjusting cert position: 107829375 -> 107829376 2019-11-08 0:46:48 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 0:46:48 0 [Note] WSREP: ####### Setting monitor position to 107829376 2019-11-08 0:46:48 0 [Note] WSREP: Lowest cert indnex boundary for CC from group: 107829302 2019-11-08 0:46:48 0 [Note] WSREP: Min available from gcache for CC from group: 107827514 2019-11-08 0:46:48 0 [Note] WSREP: Receiving IST...100.0% (75/75 events) complete. 2019-11-08 0:46:48 38 [Note] WSREP: ================================================ View: id: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829376 status: primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 2 members(4): 0: c108bb1a-01a9-11ea-9105-3730ecd6c800, FLLGaleraDBColo2 1: d929f93e-01cd-11ea-87ce-0f169e037fd4, Galeradbprod02 2: f19a4d7f-01e2-11ea-95f3-5b60e72438da, HQGaleraDBProd1 3: fd37624f-01ca-11ea-9c93-77cc6d9c9f87, GaleraDBColo1 ================================================= 2019-11-08 0:46:48 38 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:46:48 24 [Note] WSREP: IST received: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829376 2019-11-08 0:46:48 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off 2019-11-08 0:47:16 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:47:16 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:47:16 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:47:16 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:47:16 0 [Note] WSREP: view(view_id(PRIM,c108bb1a,143) memb { c108bb1a,1 d929f93e,0 f19a4d7f,0 fd37624f,1 } joined { } left { } partitioned { }) 2019-11-08 0:47:16 0 [Note] WSREP: save pc into disk 2019-11-08 0:47:16 0 [Note] WSREP: forgetting 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:47:16 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 4 2019-11-08 0:47:16 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2019-11-08 0:47:16 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 38871ea3-01eb-11ea-8213-7a258b154ed4 2019-11-08 0:47:16 0 [Warning] WSREP: Action message in non-primary configuration from member 3 2019-11-08 0:47:16 0 [Note] WSREP: STATE EXCHANGE: got state msg: 38871ea3-01eb-11ea-8213-7a258b154ed4 from 1 (Galeradbprod02) 2019-11-08 0:47:16 0 [Note] WSREP: STATE EXCHANGE: got state msg: 38871ea3-01eb-11ea-8213-7a258b154ed4 from 2 (HQGaleraDBProd1) 2019-11-08 0:47:16 0 [Note] WSREP: STATE EXCHANGE: got state msg: 38871ea3-01eb-11ea-8213-7a258b154ed4 from 3 (GaleraDBColo1) 2019-11-08 0:47:16 0 [Note] WSREP: STATE EXCHANGE: got state msg: 38871ea3-01eb-11ea-8213-7a258b154ed4 from 0 (FLLGaleraDBColo2) 2019-11-08 0:47:16 0 [Note] WSREP: Quorum results: version = 5, component = PRIMARY, conf_id = 32, members = 3/4 (joined/total), act_id = 107829438, last_appl. = 107829378, protocols = 1/10/4 (gcs/repl/appl), vote policy= 0, group UUID = 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 2019-11-08 0:47:16 0 [Note] WSREP: Writing down CC checksum: 636b6704 51e9d152 876a8a99 98d1d07f at offset 336 2019-11-08 0:47:16 0 [Note] WSREP: Flow-control interval: [256, 320] 2019-11-08 0:47:16 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 0:47:16 0 [Note] WSREP: 2.0 (HQGaleraDBProd1): State transfer from 1.0 (Galeradbprod02) complete. 2019-11-08 0:47:16 0 [Note] WSREP: Shifting JOINER -> JOINED (TO: 107829439) 2019-11-08 0:47:16 0 [Warning] WSREP: Protocol violation. JOIN message sender 2.0 (HQGaleraDBProd1) is not in state transfer (JOINED). Message ignored. 2019-11-08 0:47:16 27 [Note] WSREP: ####### processing CC 107829399, local, ordered 2019-11-08 0:47:16 0 [Note] WSREP: Member 2.0 (HQGaleraDBProd1) synced with group. 2019-11-08 0:47:16 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 107829439) 2019-11-08 0:47:16 27 [Note] WSREP: ####### drain monitors upto 107829398 2019-11-08 0:47:17 27 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-08 0:47:17 27 [Note] WSREP: ####### My UUID: f19a4d7f-01e2-11ea-95f3-5b60e72438da 2019-11-08 0:47:17 27 [Note] WSREP: ####### ST not required 2019-11-08 0:47:17 27 [Note] WSREP: Skipping cert index reset 2019-11-08 0:47:17 27 [Note] WSREP: ####### Adjusting cert position: 107829398 -> 107829399 2019-11-08 0:47:17 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 0:47:17 27 [Note] WSREP: ####### Setting monitor position to 107829399 2019-11-08 0:47:17 27 [Note] WSREP: Lowest cert indnex boundary for CC from group: 107829375 2019-11-08 0:47:17 27 [Note] WSREP: Min available from gcache for CC from group: 107827514 2019-11-08 0:47:17 27 [Note] WSREP: ================================================ View: id: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829399 status: primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 2 members(4): 0: c108bb1a-01a9-11ea-9105-3730ecd6c800, FLLGaleraDBColo2 1: d929f93e-01cd-11ea-87ce-0f169e037fd4, Galeradbprod02 2: f19a4d7f-01e2-11ea-95f3-5b60e72438da, HQGaleraDBProd1 3: fd37624f-01ca-11ea-9c93-77cc6d9c9f87, GaleraDBColo1 ================================================= 2019-11-08 0:47:17 27 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:47:17 12 [Note] WSREP: ####### processing CC 107829409, local, ordered 2019-11-08 0:47:17 12 [Note] WSREP: ####### drain monitors upto 107829408 2019-11-08 0:47:17 12 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-08 0:47:17 12 [Note] WSREP: ####### My UUID: f19a4d7f-01e2-11ea-95f3-5b60e72438da 2019-11-08 0:47:17 12 [Note] WSREP: ####### ST not required 2019-11-08 0:47:17 12 [Note] WSREP: Skipping cert index reset 2019-11-08 0:47:17 12 [Note] WSREP: ####### Adjusting cert position: 107829408 -> 107829409 2019-11-08 0:47:17 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 0:47:17 12 [Note] WSREP: ####### Setting monitor position to 107829409 2019-11-08 0:47:17 12 [Note] WSREP: Lowest cert indnex boundary for CC from group: 107829375 2019-11-08 0:47:17 12 [Note] WSREP: Min available from gcache for CC from group: 107827514 2019-11-08 0:47:17 12 [Note] WSREP: ================================================ View: id: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829409 status: primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 2 members(4): 0: c108bb1a-01a9-11ea-9105-3730ecd6c800, FLLGaleraDBColo2 1: d929f93e-01cd-11ea-87ce-0f169e037fd4, Galeradbprod02 2: f19a4d7f-01e2-11ea-95f3-5b60e72438da, HQGaleraDBProd1 3: fd37624f-01ca-11ea-9c93-77cc6d9c9f87, GaleraDBColo1 ================================================= 2019-11-08 0:47:17 12 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 0:47:17 31 [Note] WSREP: ####### processing CC 107829419, local, ordered 2019-11-08 0:47:17 31 [Note] WSREP: ####### drain monitors upto 107829418 2019-11-08 0:47:17 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49388 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:47:17 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:47:17 0 [Warning] WSREP: discarding established (time wait) 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:47:19 0 [Note] WSREP: cleaning up 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:47:22 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49392 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:47:22 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:47:22 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 2019-11-08 0:47:25 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off 2019-11-08 0:47:52 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:47:52 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:47:52 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:47:52 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:48:23 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:48:23 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:48:23 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:48:23 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:48:54 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:48:54 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:48:54 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:48:54 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:49:24 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:49:24 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:49:24 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:49:24 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:49:54 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:49:54 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:49:54 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:49:54 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:50:24 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:50:24 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:50:24 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:50:24 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 0:50:24 0 [Note] WSREP: view(view_id(PRIM,c108bb1a,149) memb { c108bb1a,1 d929f93e,0 f19a4d7f,0 fd37624f,1 } joined { } left { } partitioned { }) 2019-11-08 0:50:24 0 [Note] WSREP: save pc into disk 2019-11-08 0:50:24 0 [Note] WSREP: forgetting 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:50:24 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 4 2019-11-08 0:50:24 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2019-11-08 0:50:24 0 [Note] WSREP: STATE EXCHANGE: sent state msg: a8ca9243-01eb-11ea-8628-768a011272c3 2019-11-08 0:50:24 0 [Warning] WSREP: Action message in non-primary configuration from member 1 2019-11-08 0:50:24 0 [Note] WSREP: STATE EXCHANGE: got state msg: a8ca9243-01eb-11ea-8628-768a011272c3 from 0 (FLLGaleraDBColo2) 2019-11-08 0:50:24 0 [Note] WSREP: STATE EXCHANGE: got state msg: a8ca9243-01eb-11ea-8628-768a011272c3 from 1 (Galeradbprod02) 2019-11-08 0:50:24 0 [Note] WSREP: STATE EXCHANGE: got state msg: a8ca9243-01eb-11ea-8628-768a011272c3 from 2 (HQGaleraDBProd1) 2019-11-08 0:50:24 0 [Note] WSREP: STATE EXCHANGE: got state msg: a8ca9243-01eb-11ea-8628-768a011272c3 from 3 (GaleraDBColo1) 2019-11-08 0:50:24 0 [Note] WSREP: Quorum results: version = 5, component = PRIMARY, conf_id = 33, members = 4/4 (joined/total), act_id = 107829440, last_appl. = 107829378, protocols = 1/10/4 (gcs/repl/appl), vote policy= 0, group UUID = 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 2019-11-08 0:50:24 0 [Note] WSREP: Writing down CC checksum: 476cca30 ab299508 009af85b a63d1174 at offset 336 2019-11-08 0:50:24 0 [Note] WSREP: Flow-control interval: [256, 320] 2019-11-08 0:50:24 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 0:50:26 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49396 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:50:26 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:50:26 0 [Warning] WSREP: discarding established (time wait) 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:50:27 0 [Note] WSREP: cleaning up 45421c9c (ssl://172.25.1.10:4567) 2019-11-08 0:50:30 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:49400 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 0:50:30 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 45421c9c ssl://172.25.1.10:4567 2019-11-08 0:50:30 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 2019-11-08 0:50:34 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off 2019-11-08 0:51:01 0 [Note] WSREP: declaring c108bb1a at ssl://10.20.11.46:4567 stable 2019-11-08 0:51:01 0 [Note] WSREP: declaring d929f93e at ssl://10.1.30.61:4567 stable 2019-11-08 0:51:01 0 [Note] WSREP: declaring fd37624f at ssl://10.20.11.45:4567 stable 2019-11-08 0:51:01 0 [Note] WSREP: Node c108bb1a state prim 2019-11-08 1:10:35 0 [Warning] InnoDB: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead; Please refer to https://mariadb.com/kb/en/library/set-transaction/ 2019-11-08 1:10:35 0 [Note] InnoDB: Using Linux native AIO 2019-11-08 1:10:35 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2019-11-08 1:10:35 0 [Note] InnoDB: Uses event mutexes 2019-11-08 1:10:35 0 [Note] InnoDB: Compressed tables use zlib 1.2.7 2019-11-08 1:10:35 0 [Note] InnoDB: Number of pools: 1 2019-11-08 1:10:35 0 [Note] InnoDB: Using SSE2 crc32 instructions 2019-11-08 1:10:35 0 [Note] InnoDB: Initializing buffer pool, total size = 20G, instances = 20, chunk size = 128M 2019-11-08 1:10:36 0 [Note] InnoDB: Completed initialization of buffer pool 2019-11-08 1:10:36 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2019-11-08 1:10:36 0 [Note] InnoDB: 2 transaction(s) which must be rolled back or cleaned up in total 14140 row operations to undo 2019-11-08 1:10:36 0 [Note] InnoDB: Trx id counter is 1161492850 2019-11-08 1:10:36 0 [Note] InnoDB: 128 out of 128 rollback segments are active. 2019-11-08 1:10:36 0 [Note] InnoDB: Starting in background the rollback of recovered transactions 2019-11-08 1:10:36 0 [Note] InnoDB: Creating shared tablespace for temporary tables 2019-11-08 1:10:36 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2019-11-08 1:10:36 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2019-11-08 1:10:36 0 [Note] InnoDB: Waiting for purge to start 2019-11-08 1:10:36 0 [Note] InnoDB: 10.4.8 started; log sequence number 5412197172594; transaction id 1161492852 2019-11-08 1:10:36 0 [Warning] InnoDB: Skipping buffer pool dump/restore during wsrep recovery. 2019-11-08 1:10:36 0 [Note] Plugin 'FEEDBACK' is disabled. 191108 1:10:36 server_audit: MariaDB Audit Plugin version 1.4.8 STARTED. 2019-11-08 1:10:36 0 [Note] Server socket created on IP: '0.0.0.0'. 2019-11-08 1:10:36 0 [Note] WSREP: Recovered position: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829415 191108 1:10:36 server_audit: STOPPED 2019-11-08 1:26:33 0 [Note] WSREP: Loading provider /usr/lib64/galera-4/libgalera_smm.so initial position: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829415 wsrep loader: [INFO] wsrep_load(): loading provider library '/usr/lib64/galera-4/libgalera_smm.so' wsrep loader: [INFO] wsrep_load(): Galera 26.4.2(r4498) by Codership Oy loaded successfully. 2019-11-08 1:26:33 0 [Note] WSREP: CRC-32C: using hardware acceleration. 2019-11-08 1:26:33 0 [Note] WSREP: Found saved state: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:-1, safe_to_bootstrap: 0 2019-11-08 1:26:33 0 [Note] WSREP: GCache DEBUG: opened preamble: Version: 2 UUID: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 Seqno: -1 - -1 Offset: -1 Synced: 0 2019-11-08 1:26:33 0 [Note] WSREP: Recovering GCache ring buffer: version: 2, UUID: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22, offset: -1 2019-11-08 1:26:33 0 [Note] WSREP: GCache::RingBuffer initial scan... 0.0% ( 0/1073741848 bytes) complete. 2019-11-08 1:26:35 0 [Note] WSREP: GCache::RingBuffer initial scan...100.0% (1073741848/1073741848 bytes) complete. 2019-11-08 1:26:35 0 [Note] WSREP: Recovering GCache ring buffer: found gapless sequence 107827514-107829418 2019-11-08 1:26:35 0 [Note] WSREP: GCache::RingBuffer unused buffers scan... 0.0% ( 0/245169648 bytes) complete. 2019-11-08 1:26:35 0 [Note] WSREP: GCache::RingBuffer unused buffers scan...100.0% (245169648/245169648 bytes) complete. 2019-11-08 1:26:35 0 [Note] WSREP: GCache DEBUG: RingBuffer::recover(): found 8/1934 locked buffers 2019-11-08 1:26:35 0 [Note] WSREP: GCache DEBUG: RingBuffer::recover(): used space: 245169648/1073741824 2019-11-08 1:26:35 0 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 10.1.30.60; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.keepalive_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = galera.cache; gcache.page_size = 128M; gcache.recover = yes; gcache.size = 1G; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 0.8; gcs.fc_limit = 160; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_time 2019-11-08 1:26:35 0 [Note] WSREP: Start replication 2019-11-08 1:26:35 0 [Note] WSREP: Connecting with bootstrap option: 0 2019-11-08 1:26:35 0 [Note] WSREP: Setting GCS initial position to 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829415 2019-11-08 1:26:35 0 [Note] WSREP: protonet asio version 0 2019-11-08 1:26:35 0 [Note] WSREP: Using CRC-32C for message checksums. 2019-11-08 1:26:35 0 [Note] WSREP: initializing ssl context 2019-11-08 1:26:35 0 [Note] WSREP: backend: asio 2019-11-08 1:26:35 0 [Note] WSREP: gcomm thread scheduling priority set to other:0 2019-11-08 1:26:35 0 [Note] WSREP: restore pc from disk successfully 2019-11-08 1:26:35 0 [Note] WSREP: GMCast version 0 2019-11-08 1:26:35 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') listening at ssl://0.0.0.0:4567 2019-11-08 1:26:35 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') multicast: , ttl: 1 2019-11-08 1:26:35 0 [Note] WSREP: EVS version 1 2019-11-08 1:26:35 0 [Note] WSREP: gcomm: connecting to group 'galera_cluster', peer '10.1.30.60:,10.1.30.61:,10.20.11.45:,10.20.11.46:,172.25.1.10:' 2019-11-08 1:26:35 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.1.30.60:38778 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 1:26:35 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.1.30.60:4567 local endpoint ssl://10.1.30.60:38778 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 1:26:35 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to f19a4d7f ssl://10.1.30.60:4567 2019-11-08 1:26:35 0 [Warning] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') address 'ssl://10.1.30.60:4567' points to own listening address, blacklisting 2019-11-08 1:26:35 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.1.30.61:4567 local endpoint ssl://10.1.30.60:43068 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 1:26:35 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 678b492c ssl://10.1.30.61:4567 2019-11-08 1:26:35 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 2019-11-08 1:26:35 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.20.11.46:4567 local endpoint ssl://10.1.30.60:41306 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 1:26:35 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.20.11.45:4567 local endpoint ssl://10.1.30.60:43696 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 1:26:35 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 0602cb8a ssl://10.20.11.46:4567 2019-11-08 1:26:35 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to c4669934 ssl://10.20.11.45:4567 2019-11-08 1:26:35 0 [Note] WSREP: gcomm: connected 2019-11-08 1:26:35 0 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636 2019-11-08 1:26:35 0 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0) 2019-11-08 1:26:35 0 [Note] WSREP: Opened channel 'galera_cluster' 2019-11-08 1:26:35 2 [Note] WSREP: Starting rollbacker thread 2 2019-11-08 1:26:35 1 [Note] WSREP: Starting applier thread 1 2019-11-08 1:26:36 0 [Note] WSREP: EVS version upgrade 0 -> 1 2019-11-08 1:26:36 0 [Note] WSREP: declaring 0602cb8a at ssl://10.20.11.46:4567 stable 2019-11-08 1:26:36 0 [Note] WSREP: declaring 678b492c at ssl://10.1.30.61:4567 stable 2019-11-08 1:26:36 0 [Note] WSREP: declaring c4669934 at ssl://10.20.11.45:4567 stable 2019-11-08 1:26:36 0 [Note] WSREP: PC protocol upgrade 0 -> 1 2019-11-08 1:26:36 0 [Note] WSREP: Node 0602cb8a state prim 2019-11-08 1:26:36 0 [Note] WSREP: view(view_id(PRIM,0602cb8a,151) memb { 0602cb8a,1 678b492c,0 c4669934,1 f19a4d7f,0 } joined { } left { } partitioned { }) 2019-11-08 1:26:36 0 [Note] WSREP: save pc into disk 2019-11-08 1:26:36 0 [Note] WSREP: discarding pending addr without UUID: ssl://172.25.1.10:4567 2019-11-08 1:26:36 0 [Note] WSREP: discarding pending addr proto entry 0x564029bd6620 2019-11-08 1:26:36 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 3, memb_num = 4 2019-11-08 1:26:36 0 [Note] WSREP: clear restored view 2019-11-08 1:26:36 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2019-11-08 1:26:36 0 [Note] WSREP: STATE EXCHANGE: sent state msg: b72b56d4-01f0-11ea-b40e-be42b9494ae2 2019-11-08 1:26:36 0 [Note] WSREP: STATE EXCHANGE: got state msg: b72b56d4-01f0-11ea-b40e-be42b9494ae2 from 0 (FLLGaleraDBColo2) 2019-11-08 1:26:36 0 [Note] WSREP: STATE EXCHANGE: got state msg: b72b56d4-01f0-11ea-b40e-be42b9494ae2 from 1 (Galeradbprod02) 2019-11-08 1:26:36 0 [Note] WSREP: STATE EXCHANGE: got state msg: b72b56d4-01f0-11ea-b40e-be42b9494ae2 from 2 (GaleraDBColo1) 2019-11-08 1:26:36 0 [Note] WSREP: STATE EXCHANGE: got state msg: b72b56d4-01f0-11ea-b40e-be42b9494ae2 from 3 (HQGaleraDBProd1) 2019-11-08 1:26:36 0 [Note] WSREP: Quorum results: version = 5, component = PRIMARY, conf_id = 3, members = 3/4 (joined/total), act_id = 107831076, last_appl. = 0, protocols = 1/10/4 (gcs/repl/appl), vote policy= 0, group UUID = 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 2019-11-08 1:26:36 0 [Note] WSREP: Writing down CC checksum: 94007f13 cc2b96b5 bbb79091 f81a802e at offset 336 2019-11-08 1:26:36 0 [Note] WSREP: Flow-control interval: [256, 320] 2019-11-08 1:26:36 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 1:26:36 0 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 107831077) 2019-11-08 1:26:36 1 [Note] WSREP: ####### processing CC 107831077, local, ordered 2019-11-08 1:26:36 1 [Note] WSREP: ####### drain monitors upto -1 2019-11-08 1:26:38 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection to peer f19a4d7f with addr ssl://10.1.30.60:4567 timed out, no messages seen in PT3S 2019-11-08 1:26:38 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off 2019-11-08 1:27:28 1 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-08 1:27:28 1 [Note] WSREP: ####### My UUID: f19a4d7f-01e2-11ea-95f4-5b60e72438da 2019-11-08 1:27:28 1 [Note] WSREP: Server HQGaleraDBProd1 connected to cluster at position 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107831077 with ID f19a4d7f-01e2-11ea-95f4-5b60e72438da 2019-11-08 1:27:28 1 [Note] WSREP: Server status change disconnected -> connected 2019-11-08 1:27:28 1 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 1:27:28 1 [Note] WSREP: State transfer required: Group state: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107831077 Local state: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829415 2019-11-08 1:27:28 1 [Note] WSREP: Server status change connected -> joiner 2019-11-08 1:27:28 1 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 1:27:28 0 [Note] WSREP: Running: 'wsrep_sst_mariabackup --role 'joiner' --address '10.1.30.60' --datadir '/var/lib/mysql/' --parent '26118' --mysqld-args --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --wsrep_on=ON --wsrep_provider=/usr/lib64/galera-4/libgalera_smm.so --log-error=/var/lib/mysql/error.log --pid-file=/var/lib/mysql/HQGaleraDBProd1.pid --wsrep_start_position=1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829415' WSREP_SST: [INFO] Moving /var/lib/mysql//mariabackup.prepare.log to /tmp/sst_log_archive/mariabackup.prepare.log.2019.11.08-01.27.28.799790124 (20191108 01:27:28.807) WSREP_SST: [INFO] Moving /var/lib/mysql//mariabackup.move.log to /tmp/sst_log_archive/mariabackup.move.log.2019.11.08-01.27.28.799790124 (20191108 01:27:28.825) WSREP_SST: [INFO] Streaming with xbstream (20191108 01:27:28.862) WSREP_SST: [INFO] Using socat as streamer (20191108 01:27:28.870) WSREP_SST: [INFO] Evaluating timeout -k 110 100 socat -u TCP-LISTEN:4444,reuseaddr stdio | mbstream -x; RC=( ${PIPESTATUS[@]} ) (20191108 01:27:29.030) 2019-11-08 1:27:29 1 [Note] WSREP: Prepared SST request: mariabackup|10.1.30.60:4444/xtrabackup_sst//1 2019-11-08 1:27:29 1 [Note] WSREP: Cert index reset to 00000000-0000-0000-0000-000000000000:0 (proto: 10), state transfer needed: yes 2019-11-08 1:27:29 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 1:27:29 1 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:0, protocol version: 5 2019-11-08 1:27:29 1 [Note] WSREP: ####### IST uuid:1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 f: 107829416, l: 107831077, STRv: 3 2019-11-08 1:27:29 1 [Note] WSREP: IST receiver addr using ssl://10.1.30.60:4568 2019-11-08 1:27:29 1 [Note] WSREP: IST receiver using ssl 2019-11-08 1:27:29 1 [Note] WSREP: Prepared IST receiver for 107829416-107831077, listening at: ssl://10.1.30.60:4568 2019-11-08 1:27:29 0 [Note] WSREP: Member 3.0 (HQGaleraDBProd1) requested state transfer from '*any*'. Selected 1.0 (Galeradbprod02)(SYNCED) as donor. 2019-11-08 1:27:29 0 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 107831077) 2019-11-08 1:27:29 1 [Note] WSREP: Requesting state transfer: success, donor: 1 2019-11-08 1:27:29 0 [Note] WSREP: 1.0 (Galeradbprod02): State transfer to 3.0 (HQGaleraDBProd1) complete. 2019-11-08 1:27:29 0 [Note] WSREP: Member 1.0 (Galeradbprod02) synced with group. WSREP_SST: [INFO] xtrabackup_ist received from donor: Running IST (20191108 01:27:29.863) WSREP_SST: [INFO] Galera co-ords from recovery: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829415 0 (20191108 01:27:29.872) WSREP_SST: [INFO] Total time on joiner: 0 seconds (20191108 01:27:29.881) WSREP_SST: [INFO] Removing the sst_in_progress file (20191108 01:27:29.888) 2019-11-08 1:27:29 3 [Note] WSREP: SST received 2019-11-08 1:27:29 3 [Note] WSREP: Server status change joiner -> initializing 2019-11-08 1:27:29 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 1:27:29 0 [Warning] InnoDB: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead; Please refer to https://mariadb.com/kb/en/library/set-transaction/ 2019-11-08 1:27:29 0 [Note] InnoDB: Using Linux native AIO 2019-11-08 1:27:29 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2019-11-08 1:27:29 0 [Note] InnoDB: Uses event mutexes 2019-11-08 1:27:29 0 [Note] InnoDB: Compressed tables use zlib 1.2.7 2019-11-08 1:27:29 0 [Note] InnoDB: Number of pools: 1 2019-11-08 1:27:29 0 [Note] InnoDB: Using SSE2 crc32 instructions 2019-11-08 1:27:29 0 [Note] InnoDB: Initializing buffer pool, total size = 20G, instances = 20, chunk size = 128M 2019-11-08 1:27:30 0 [Note] InnoDB: Completed initialization of buffer pool 2019-11-08 1:27:30 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2019-11-08 1:27:30 0 [Note] InnoDB: Setting log file ./ib_logfile101 size to 2147483648 bytes 2019-11-08 1:27:30 0 [Note] InnoDB: Setting log file ./ib_logfile1 size to 2147483648 bytes 2019-11-08 1:27:30 0 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0 2019-11-08 1:27:30 0 [Note] InnoDB: New log files created, LSN=5412197205443 2019-11-08 1:27:30 0 [Note] InnoDB: 2 transaction(s) which must be rolled back or cleaned up in total 14140 row operations to undo 2019-11-08 1:27:30 0 [Note] InnoDB: Trx id counter is 1161492850 2019-11-08 1:27:31 0 [Note] InnoDB: 128 out of 128 rollback segments are active. 2019-11-08 1:27:31 0 [Note] InnoDB: Starting in background the rollback of recovered transactions 2019-11-08 1:27:31 0 [Note] InnoDB: Creating shared tablespace for temporary tables 2019-11-08 1:27:31 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2019-11-08 1:27:31 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2019-11-08 1:27:31 0 [Note] InnoDB: Waiting for purge to start 2019-11-08 1:27:31 0 [Note] InnoDB: 10.4.8 started; log sequence number 5412197205516; transaction id 1161492852 2019-11-08 1:27:31 0 [Note] Plugin 'FEEDBACK' is disabled. 191108 1:27:31 server_audit: MariaDB Audit Plugin version 1.4.8 STARTED. 2019-11-08 1:27:31 0 [Note] Server socket created on IP: '0.0.0.0'. 2019-11-08 1:27:31 0 [Warning] 'user' entry 'root@galeradb1' ignored in --skip-name-resolve mode. 2019-11-08 1:27:31 0 [Warning] 'user' entry 'ghwdbadmin@galeradb1' ignored in --skip-name-resolve mode. 2019-11-08 1:27:31 0 [Warning] 'db' entry 'TESTIMONIAL ghwdbadmin@galeradb1' ignored in --skip-name-resolve mode. 2019-11-08 1:27:31 0 [Warning] 'proxies_priv' entry '@% root@galeradb1' ignored in --skip-name-resolve mode. 2019-11-08 1:27:31 0 [Note] WSREP: wsrep_init_schema_and_SR (nil) 2019-11-08 1:27:31 0 [Note] WSREP: Server initialized 2019-11-08 1:27:31 0 [Note] WSREP: Server status change initializing -> initialized 2019-11-08 1:27:31 0 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 1:27:31 3 [Note] WSREP: Server status change initialized -> joined 2019-11-08 1:27:31 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 1:27:31 3 [Note] WSREP: Recovered position from storage: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829415 2019-11-08 1:27:31 11 [Note] WSREP: Starting applier thread 11 2019-11-08 1:27:31 10 [Note] WSREP: Starting applier thread 10 2019-11-08 1:27:31 12 [Note] WSREP: Starting applier thread 12 2019-11-08 1:27:31 13 [Note] WSREP: Starting applier thread 13 2019-11-08 1:27:31 21 [Note] WSREP: Starting applier thread 21 2019-11-08 1:27:31 15 [Note] WSREP: Starting applier thread 15 2019-11-08 1:27:31 3 [Note] WSREP: Recovered view from SST: id: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829409 status: primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: -1 members(4): 0: c108bb1a-01a9-11ea-9105-3730ecd6c800, FLLGaleraDBColo2 1: d929f93e-01cd-11ea-87ce-0f169e037fd4, Galeradbprod02 2: f19a4d7f-01e2-11ea-95f3-5b60e72438da, HQGaleraDBProd1 3: fd37624f-01ca-11ea-9c93-77cc6d9c9f87, GaleraDBColo1 2019-11-08 1:27:31 14 [Note] WSREP: Starting applier thread 14 2019-11-08 1:27:31 20 [Note] WSREP: Starting applier thread 20 2019-11-08 1:27:31 17 [Note] WSREP: Starting applier thread 17 2019-11-08 1:27:31 23 [Note] WSREP: Starting applier thread 23 2019-11-08 1:27:31 18 [Note] WSREP: Starting applier thread 18 2019-11-08 1:27:31 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 1:27:31 22 [Note] WSREP: Starting applier thread 22 2019-11-08 1:27:31 16 [Note] WSREP: Starting applier thread 16 2019-11-08 1:27:31 25 [Note] WSREP: Starting applier thread 25 2019-11-08 1:27:31 27 [Note] WSREP: Recovered cluster id 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 2019-11-08 1:27:31 28 [Note] WSREP: Starting applier thread 28 2019-11-08 1:27:31 19 [Note] WSREP: Starting applier thread 19 2019-11-08 1:27:31 26 [Note] WSREP: Starting applier thread 26 2019-11-08 1:27:31 24 [Note] WSREP: Starting applier thread 24 2019-11-08 1:27:31 29 [Note] WSREP: Starting applier thread 29 2019-11-08 1:27:31 31 [Note] WSREP: Starting applier thread 31 2019-11-08 1:27:31 3 [Note] WSREP: SST received: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829415 2019-11-08 1:27:31 1 [Note] WSREP: Installed new state from SST: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829415 2019-11-08 1:27:31 30 [Note] WSREP: Starting applier thread 30 2019-11-08 1:27:31 1 [Note] WSREP: Receiving IST: 1662 writesets, seqnos 107829416-107831077 2019-11-08 1:27:31 0 [Note] WSREP: ####### IST applying starts with 107829416 2019-11-08 1:27:31 33 [Note] WSREP: Starting applier thread 33 2019-11-08 1:27:31 35 [Note] WSREP: Starting applier thread 35 2019-11-08 1:27:31 32 [Note] WSREP: Starting applier thread 32 2019-11-08 1:27:31 36 [Note] WSREP: Starting applier thread 36 2019-11-08 1:27:31 39 [Note] WSREP: Starting applier thread 39 2019-11-08 1:27:31 34 [Note] WSREP: Starting applier thread 34 2019-11-08 1:27:31 38 [Note] WSREP: Starting applier thread 38 2019-11-08 1:27:31 40 [Note] WSREP: Starting applier thread 40 2019-11-08 1:27:31 41 [Note] WSREP: Starting applier thread 41 2019-11-08 1:27:31 37 [Note] WSREP: Starting applier thread 37 2019-11-08 1:27:31 0 [Note] Reading of all Master_info entries succeeded 2019-11-08 1:27:31 0 [Note] Added new Master_info '' to hash table 2019-11-08 1:27:31 0 [Note] /usr/sbin/mysqld: ready for connections. Version: '10.4.8-MariaDB-log' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server 2019-11-08 1:27:32 0 [Note] WSREP: ####### IST current seqno initialized to 107829416 2019-11-08 1:27:32 0 [Note] WSREP: Receiving IST... 0.0% ( 0/1662 events) complete. 2019-11-08 1:27:32 0 [Note] WSREP: ####### processing CC 107829419, from IST, ordered 2019-11-08 1:27:32 0 [Note] WSREP: ####### drain monitors upto 0 2019-11-08 1:27:32 10 [Note] InnoDB: *** Priority TRANSACTION: TRANSACTION 1161492858, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 1 lock struct(s), heap size 1136, 0 row lock(s) MySQL thread id 10, OS thread handle 140433039890176, query id 17 Write_rows_log_event::write_row(107829417) insert into table (id, field, field) values (value,value,value) 2019-11-08 1:27:32 10 [Note] InnoDB: *** Victim TRANSACTION: TRANSACTION 1161492832, ACTIVE 2 sec recovered trx 2 lock struct(s), heap size 1136, 1 row lock(s), undo log entries 1 2019-11-08 1:27:32 10 [Note] InnoDB: *** WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 333101 page no 3 n bits 96 index PRIMARY of table `dc-domo-skuvault`.`job_results` trx id 1161492832 lock_mode X locks rec but not gap Record lock, heap no 30 PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 30; hex 35636232396434312d383863342d346638642d613930392d656461383834; asc 5cb29d41-88c4-4f8d-a909-eda884; (total 36 bytes); 1: len 6; hex 0000453af960; asc E: `;; 2: len 7; hex bd0001800303c0; asc ;; 3: len 4; hex 8000000f; asc ;; 4: len 30; hex 2047657474696e6720646174612066726f6d20323031392d31312d303754; asc Getting data from 2019-11-07T; (total 264 bytes); 2019-11-08 1:27:32 10 [Note] InnoDB: SQL1: 2019-11-08 1:27:32 10 [Note] InnoDB: SQL2: 2019-11-08 1:27:32 10 [Warning] WSREP: no THD for trx: 1161492832 2019-11-08 1:27:32 47 [Warning] Access denied for user 'root'@'localhost' (using password: NO) 2019-11-08 1:27:45 0 [Note] InnoDB: To roll back: 2 transactions, 12743 rows 2019-11-08 1:27:54 11 [Note] InnoDB: *** Priority TRANSACTION: TRANSACTION 1161492862, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 1 lock struct(s), heap size 1136, 0 row lock(s) MySQL thread id 11, OS thread handle 140433016878848, query id 278 Write_rows_log_event::write_row(107829416) insert into sales (channel, channel_id, client, company, created_date, email, external_id, first_name, last_name, last_printed_date, marketplace, marketplace_id, notes, phone, printed_status, sale_date, shipping_address1, shipping_address2, shipping_carrier, shipping_charge, shipping_city, shipping_class, shipping_cost, shipping_country, shipping_postal_code, shipping_region, status, updated_date, id) values ('ShipStation', '662295-112-6988791-9249063', 'c1frlr7q2njrxtz@marketplace.amazon.com', '', '2019-11-08 05:37:36.365', 'c1frlr7q2njrxtz@marketplace.amazon.com', '1-341902-5-7829-662295-112-6988791-9249063', 'Fatima Luna', '', '0001-01-01 00:00:00', 'AmazonUS', '112-6988791-9249063', '', '+1 763-225-9463 ext. 12219', 0, '2019-11-08 00:01:07', '1728 PANORAMA RD', ',', 'Second US D2D Dom', 0, 'VISTA', 'amazon_fedex_second_day_one_rate', 0, 'US', '92083-1919', 'CA', 'ReadyToShip', '2019-11-08 05:37:3 2019-11-08 1:27:54 11 [Note] InnoDB: *** Victim TRANSACTION: TRANSACTION 1161492834, ACTIVE 24 sec recovered trx ROLLING BACK 6 lock struct(s), heap size 1136, 1 row lock(s), undo log entries 11750 2019-11-08 1:27:54 11 [Note] InnoDB: *** WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 332284 page no 59671 n bits 120 index PRIMARY of table `dc-domo-skuvault`.`sales` trx id 1161492834 lock_mode X locks rec but not gap Record lock, heap no 47 PHYSICAL RECORD: n_fields 31; compact format; info bits 0 0: len 30; hex 30653938353762362d653462322d343664322d383030642d356661313237; asc 0e9857b6-e4b2-46d2-800d-5fa127; (total 36 bytes); 1: len 6; hex 0000453af962; asc E: b;; 2: len 7; hex bf000180090201; asc ;; 3: len 8; hex 99a49059640591c8; asc Yd ;; 4: len 8; hex 99a49059640591c8; asc Yd ;; 5: len 30; hex 312d3334313930322d352d373832392d3636323239352d3131322d363938; asc 1-341902-5-7829-662295-112-698; (total 42 bytes); 6: len 19; hex 3131322d363938383739312d39323439303633; asc 112-6988791-9249063;; 7: len 26; hex 3636323239352d3131322d363938383739312d39323439303633; asc 662295-112-6988791-9249063;; 8: len 11; hex 5265616479546f53686970; asc ReadyToShip;; 9: len 8; hex 99a4900047000000; asc G ;; 10: len 8; hex 416d617a6f6e5553; asc AmazonUS;; 11: len 30; hex 633166726c723771326e6a7278747a406d61726b6574706c6163652e616d; asc c1frlr7q2njrxtz@marketplace.am; (total 38 bytes); 12: len 11; hex 5368697053746174696f6e; asc ShipStation;; 13: len 5; hex 8000000000; asc ;; 14: len 5; hex 8000000000; asc ;; 15: len 17; hex 5365636f6e642055532044324420446f6d; asc Second US D2D Dom;; 16: len 30; hex 616d617a6f6e5f66656465785f7365636f6e645f6461795f6f6e655f7261; asc amazon_fedex_second_day_one_ra; (total 32 bytes); 17: len 5; hex 5649535441; asc VISTA;; 18: len 2; hex 4341; asc CA;; 19: len 2; hex 5553; asc US;; 20: len 10; hex 39323038332d31393139; asc 92083-1919;; 21: len 16; hex 313732382050414e4f52414d41205244; asc 1728 PANORAMA RD;; 22: len 1; hex 2c; asc ,;; 23: len 11; hex 466174696d61204c756e61; asc Fatima Luna;; 24: len 0; hex ; asc ;; 25: len 0; hex ; asc ;; 26: len 26; hex 2b31203736332d3232352d39343633206578742e203132323139; asc +1 763-225-9463 ext. 12219;; 27: len 30; hex 633166726c723771326e6a7278747a406d61726b6574706c6163652e616d; asc c1frlr7q2njrxtz@marketplace.am; (total 38 bytes); 28: len 0; hex ; asc ;; 29: len 1; hex 80; asc ;; 30: len 8; hex 8003820000000000; asc ;; 2019-11-08 1:27:54 11 [Note] InnoDB: SQL1: insert into sales (channel, channel_id, client, company, created_date, email, external_id, first_name, last_name, last_printed_date, marketplace, marketplace_id, notes, phone, printed_status, sale_date, shipping_address1, shipping_address2, shipping_carrier, shipping_charge, shipping_city, shipping_class, shipping_cost, shipping_country, shipping_postal_code, shipping_region, status, updated_date, id) values ('ShipStation', '662295-112-6988791-9249063', 'c1frlr7q2njrxtz@marketplace.amazon.com', '', '2019-11-08 05:37:36.365', 'c1frlr7q2njrxtz@marketplace.amazon.com', '1-341902-5-7829-662295-112-6988791-9249063', 'Fatima Luna', '', '0001-01-01 00:00:00', 'AmazonUS', '112-6988791-9249063', '', '+1 763-225-9463 ext. 12219', 0, '2019-11-08 00:01:07', '1728 PANORAMA RD', ',', 'Second US D2D Dom', 0, 'VISTA', 'amazon_fedex_second_day_one_rate', 0, 'US', '92083-1919', 'CA', 'ReadyToShip', '2019-11-08 05:37:36.365', '0e9857b6-e4b2-46d2-800d-5fa127ee397e')k��] 2019-11-08 1:27:54 11 [Note] InnoDB: SQL2: 2019-11-08 1:27:54 11 [Warning] WSREP: no THD for trx: 1161492834 2019-11-08 1:28:00 0 [Note] InnoDB: To roll back: 2 transactions, 11088 rows 2019-11-08 1:28:14 0 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-08 1:28:14 0 [Note] WSREP: ####### ST not required 2019-11-08 1:28:14 0 [Note] WSREP: Skipping cert index reset 2019-11-08 1:28:14 0 [Note] WSREP: ####### Adjusting cert position: 0 -> 107829419 2019-11-08 1:28:14 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 1:28:14 0 [Note] WSREP: ####### Setting monitor position to 107829419 2019-11-08 1:28:14 49 [Warning] Aborted connection 49 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 51 [Warning] Aborted connection 51 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 59 [Warning] Aborted connection 59 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 67 [Warning] Aborted connection 67 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 45 [Warning] Aborted connection 45 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 57 [Warning] Aborted connection 57 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 87 [Warning] Aborted connection 87 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 94 [Warning] Aborted connection 94 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 104 [Warning] Aborted connection 104 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 116 [Warning] Aborted connection 116 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 118 [Warning] Aborted connection 118 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 126 [Warning] Aborted connection 126 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 72 [Warning] Aborted connection 72 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 78 [Warning] Aborted connection 78 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 88 [Warning] Aborted connection 88 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 66 [Warning] Aborted connection 66 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 80 [Warning] Aborted connection 80 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 74 [Warning] Aborted connection 74 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 54 [Warning] Aborted connection 54 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 84 [Warning] Aborted connection 84 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 76 [Warning] Aborted connection 76 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 91 [Warning] Aborted connection 91 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 60 [Warning] Aborted connection 60 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 81 [Warning] Aborted connection 81 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 82 [Warning] Aborted connection 82 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 50 [Warning] Aborted connection 50 to db: 'information_schema' user: 'cmon' host: '10.1.30.30' (Got an error reading communication packets) 2019-11-08 1:28:14 44 [Warning] Aborted connection 44 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 98 [Warning] Aborted connection 98 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 70 [Warning] Aborted connection 70 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 61 [Warning] Aborted connection 61 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 90 [Warning] Aborted connection 90 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 64 [Warning] Aborted connection 64 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 96 [Warning] Aborted connection 96 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 100 [Warning] Aborted connection 100 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 102 [Warning] Aborted connection 102 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 73 [Warning] Aborted connection 73 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 58 [Warning] Aborted connection 58 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 63 [Warning] Aborted connection 63 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 107 [Warning] Aborted connection 107 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 114 [Warning] Aborted connection 114 to db: 'information_schema' user: 'cmon' host: '10.1.30.30' (Got an error reading communication packets) 2019-11-08 1:28:14 111 [Warning] Aborted connection 111 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 113 [Warning] Aborted connection 113 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 85 [Warning] Aborted connection 85 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 122 [Warning] Aborted connection 122 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 52 [Warning] Aborted connection 52 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 71 [Warning] Aborted connection 71 to db: 'information_schema' user: 'cmon' host: '10.1.30.30' (Got an error reading communication packets) 2019-11-08 1:28:14 124 [Warning] Aborted connection 124 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 109 [Warning] Aborted connection 109 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 115 [Warning] Aborted connection 115 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 120 [Warning] Aborted connection 120 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 65 [Warning] Aborted connection 65 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 79 [Warning] Aborted connection 79 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 129 [Warning] Aborted connection 129 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 69 [Warning] Aborted connection 69 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 130 [Warning] Aborted connection 130 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 56 [Warning] Aborted connection 56 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 133 [Warning] Aborted connection 133 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:28:14 128 [Warning] Aborted connection 128 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 68 [Warning] Aborted connection 68 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 95 [Warning] Aborted connection 95 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 62 [Warning] Aborted connection 62 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 123 [Warning] Aborted connection 123 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 75 [Warning] Aborted connection 75 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 117 [Warning] Aborted connection 117 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 93 [Warning] Aborted connection 93 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 89 [Warning] Aborted connection 89 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 55 [Warning] Aborted connection 55 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 97 [Warning] Aborted connection 97 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 83 [Warning] Aborted connection 83 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 132 [Warning] Aborted connection 132 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 101 [Warning] Aborted connection 101 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 112 [Warning] Aborted connection 112 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 77 [Warning] Aborted connection 77 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 127 [Warning] Aborted connection 127 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 125 [Warning] Aborted connection 125 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 131 [Warning] Aborted connection 131 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 99 [Warning] Aborted connection 99 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 110 [Warning] Aborted connection 110 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 105 [Warning] Aborted connection 105 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 119 [Warning] Aborted connection 119 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 108 [Warning] Aborted connection 108 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 103 [Warning] Aborted connection 103 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 86 [Warning] Aborted connection 86 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 121 [Warning] Aborted connection 121 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets) 2019-11-08 1:28:14 92 [Warning] Aborted connection 92 to db: 'information_schema' user: 'cmon' host: '10.1.30.30' (Got an error reading communication packets) 2019-11-08 1:28:14 0 [Note] WSREP: Lowest cert indnex boundary for CC from group: 107829419 2019-11-08 1:28:14 0 [Note] WSREP: Min available from gcache for CC from group: 107827514 2019-11-08 1:28:14 21 [Note] WSREP: ================================================ View: id: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829419 status: primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: -1 members(4): 0: c108bb1a-01a9-11ea-9105-3730ecd6c800, FLLGaleraDBColo2 1: d929f93e-01cd-11ea-87ce-0f169e037fd4, Galeradbprod02 2: f19a4d7f-01e2-11ea-95f3-5b60e72438da, HQGaleraDBProd1 3: fd37624f-01ca-11ea-9c93-77cc6d9c9f87, GaleraDBColo1 ================================================= 2019-11-08 1:28:14 21 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 1:28:14 0 [Note] WSREP: Receiving IST... 1.0% ( 16/1662 events) complete. 2019-11-08 1:28:14 0 [Note] WSREP: ####### processing CC 107829437, from IST, ordered 2019-11-08 1:28:14 0 [Note] WSREP: ####### drain monitors upto 107829419 2019-11-08 1:28:14 15 [Note] InnoDB: BF-BF X lock conflict,mode: 1027 supremum: 0conflicts states: my executing locked committing RECORD LOCKS space id 332262 page no 3 n bits 88 index PRIMARY of table `dc-domo-skuvault`.`jobs` trx id 1161492860 lock_mode X locks rec but not gap Record lock, heap no 17 2019-11-08 1:28:14 15 [Note] InnoDB: SQL1: update jobs set description='Sync POs', email_bcc=NULL, email_cc=NULL, email_to='IFernandez@greathealthworks.com', executing=0, executing_date='2019-11-08 04:59:59.000999', last_executing_date='2019-11-08 05:42:38.647', restrict_execution=1, state=1, time_alert_executing=0, type='YESTERDAY', write_log_when_empty=0 where id=16� SQL2: 2019-11-08 1:28:14 0 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-08 1:28:14 0 [Note] WSREP: ####### ST not required 2019-11-08 1:28:14 0 [Note] WSREP: Skipping cert index reset 2019-11-08 1:28:14 0 [Note] WSREP: ####### Adjusting cert position: 107829419 -> 107829437 2019-11-08 1:28:14 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 1:28:14 0 [Note] WSREP: ####### Setting monitor position to 107829437 2019-11-08 1:28:14 0 [Note] WSREP: Lowest cert indnex boundary for CC from group: 107829437 2019-11-08 1:28:14 0 [Note] WSREP: Min available from gcache for CC from group: 107827514 2019-11-08 1:28:14 1 [Note] WSREP: ================================================ View: id: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829437 status: primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: -1 members(4): 0: c108bb1a-01a9-11ea-9105-3730ecd6c800, FLLGaleraDBColo2 1: d929f93e-01cd-11ea-87ce-0f169e037fd4, Galeradbprod02 2: f19a4d7f-01e2-11ea-95f3-5b60e72438da, HQGaleraDBProd1 3: fd37624f-01ca-11ea-9c93-77cc6d9c9f87, GaleraDBColo1 ================================================= 2019-11-08 1:28:14 1 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 1:28:14 0 [Note] WSREP: ####### processing CC 107829439, from IST, ordered 2019-11-08 1:28:14 0 [Note] WSREP: ####### drain monitors upto 107829437 2019-11-08 1:28:14 0 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-08 1:28:14 0 [Note] WSREP: ####### ST not required 2019-11-08 1:28:14 0 [Note] WSREP: Skipping cert index reset 2019-11-08 1:28:14 0 [Note] WSREP: ####### Adjusting cert position: 107829437 -> 107829439 2019-11-08 1:28:14 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 1:28:14 0 [Note] WSREP: ####### Setting monitor position to 107829439 2019-11-08 1:28:14 0 [Note] WSREP: Lowest cert indnex boundary for CC from group: 107829439 2019-11-08 1:28:14 0 [Note] WSREP: Min available from gcache for CC from group: 107827514 2019-11-08 1:28:14 0 [Note] WSREP: ####### processing CC 107829441, from IST, ordered 2019-11-08 1:28:14 0 [Note] WSREP: ####### drain monitors upto 107829439 2019-11-08 1:28:14 32 [Note] WSREP: ================================================ View: id: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829439 status: primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: -1 members(4): 0: c108bb1a-01a9-11ea-9105-3730ecd6c800, FLLGaleraDBColo2 1: d929f93e-01cd-11ea-87ce-0f169e037fd4, Galeradbprod02 2: f19a4d7f-01e2-11ea-95f3-5b60e72438da, HQGaleraDBProd1 3: fd37624f-01ca-11ea-9c93-77cc6d9c9f87, GaleraDBColo1 ================================================= 2019-11-08 1:28:14 32 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 1:28:14 0 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-08 1:28:14 0 [Note] WSREP: ####### ST not required 2019-11-08 1:28:14 0 [Note] WSREP: Skipping cert index reset 2019-11-08 1:28:14 0 [Note] WSREP: ####### Adjusting cert position: 107829439 -> 107829441 2019-11-08 1:28:14 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 1:28:14 0 [Note] WSREP: ####### Setting monitor position to 107829441 2019-11-08 1:28:14 0 [Note] WSREP: Lowest cert indnex boundary for CC from group: 107829441 2019-11-08 1:28:14 0 [Note] WSREP: Min available from gcache for CC from group: 107827514 2019-11-08 1:28:14 36 [Note] WSREP: ================================================ View: id: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829441 status: primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: -1 members(4): 0: c108bb1a-01a9-11ea-9105-3730ecd6c800, FLLGaleraDBColo2 1: d929f93e-01cd-11ea-87ce-0f169e037fd4, Galeradbprod02 2: f19a4d7f-01e2-11ea-95f3-5b60e72438da, HQGaleraDBProd1 3: fd37624f-01ca-11ea-9c93-77cc6d9c9f87, GaleraDBColo1 ================================================= 2019-11-08 1:28:14 36 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 1:28:14 0 [Note] WSREP: ####### processing CC 107829444, from IST, ordered 2019-11-08 1:28:14 0 [Note] WSREP: ####### drain monitors upto 107829441 2019-11-08 1:28:14 0 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-08 1:28:14 0 [Note] WSREP: ####### ST not required 2019-11-08 1:28:14 0 [Note] WSREP: Skipping cert index reset 2019-11-08 1:28:14 0 [Note] WSREP: ####### Adjusting cert position: 107829441 -> 107829444 2019-11-08 1:28:14 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 1:28:14 0 [Note] WSREP: ####### Setting monitor position to 107829444 2019-11-08 1:28:14 39 [Note] InnoDB: BF-BF X lock conflict,mode: 1027 supremum: 0conflicts states: my executing locked committing RECORD LOCKS space id 286102 page no 3 n bits 88 index PRIMARY of table `skuvault`.`jobs` trx id 1161492874 lock_mode X locks rec but not gap Record lock, heap no 15 PHYSICAL RECORD: n_fields 15; compact format; info bits 0 0: len 4; hex 80000005; asc ;; 1: len 6; hex 0000453af98a; asc E: ;; 2: len 7; hex 160000021329e1; asc ) ;; 3: len 20; hex 4d6f7665206461746120746f204d617269614442; asc Move data to MariaDB;; 4: len 1; hex 81; asc ;; 5: len 1; hex 80; asc ;; 6: len 1; hex 81; asc ;; 7: len 8; hex 99a4905ac1001388; asc Z ;; 8: len 1; hex 02; asc ;; 9: len 30; hex 656d756e697a4067726561746865616c7468776f726b732e636f6d3b6e67; asc emuniz@greathealthworks.com;ng; (total 202 bytes); 10: len 30; hex 696665726e616e64657a4067726561746865616c7468776f726b732e636f; asc ifernandez@greathealthworks.co; (total 31 bytes); 11: SQL NULL; 12: len 1; hex 80; asc ;; 13: len 4; hex 80000005; asc ;; 14: SQL DEFAULT; 2019-11-08 1:28:14 39 [Note] InnoDB: SQL1: update jobs set executing=1, executing_date='2019-11-08 05:47:01.006' where id=5��] SQL2: 2019-11-08 1:28:14 34 [Note] InnoDB: BF-BF X lock conflict,mode: 1027 supremum: 0conflicts states: my executing locked committing 2019-11-08 1:28:14 0 [Note] WSREP: Lowest cert indnex boundary for CC from group: 107829444 RECORD LOCKS space id 286102 page no 3 n bits 88 index PRIMARY of table `skuvault`.`jobs` trx id 1161492874 lock_mode X locks rec but not gap Record lock, heap no 15 PHYSICAL RECORD: n_fields 15; compact format; info bits 0 0: len 4; hex 80000005; asc ;; 1: len 6; hex 0000453a2019-11-08 1:28:14 0 [Note] WSREP: Min available from gcache for CC from group: 107827514 f9ca; asc E: ;; 2: len 7; hex 51000043df105d; asc Q C ];; 3: len 20; hex 4d6f7665206461746120746f204d617269614442; asc Move data to MariaDB;; 4: len 1; hex 81; asc ;; 5: len 1; hex 80; asc ;; 6: len 1; hex 81; asc ;; 7: len 8; hex 99a4905bc1001770; asc [ p;; 8: len 1; hex 02; asc ;; 9: len 30; hex 656d756e697a4067726561746865616c7468776f726b732e636f6d3b6e67; asc emuniz@greathealthworks.com;ng; (total 202 bytes); 10: len 30; hex 696665726e616e64657a4067726561746865616c7468776f726b732e636f; asc ifernandez@greathealthworks.co; (total 31 bytes); 11: SQL NULL; 12: len 1; hex 802019-11-08 1:28:14 40 [Note] WSREP: ================================================ View: id: 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22:107829444 status: primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: -1 members(3): 0: c108bb1a-01a9-11ea-9105-3730ecd6c800, FLLGaleraDBColo2 1: d929f93e-01cd-11ea-87ce-0f169e037fd4, Galeradbprod02 2: fd37624f-01ca-11ea-9c93-77cc6d9c9f87, GaleraDBColo1 ================================================= 2019-11-08 1:28:14 40 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. ; asc 2019-11-08 1:28:14 0 [Note] WSREP: ####### processing CC 107829462, from IST, ordered ;2019-11-08 1:28:14 0 [Note] WSREP: ####### drain monitors upto 107829444 ; 13: len 4; hex 80000005; asc ;; 14: SQL DEFAULT; 2019-11-08 1:28:14 34 [Note] InnoDB:  SQL2: 2019-11-08 1:28:14 0 [Note] WSREP: REPL Protocols: 10 (5, 3) 2019-11-08 1:28:14 0 [Note] WSREP: ####### ST not required 2019-11-08 1:28:14 0 [Note] WSREP: Skipping cert index reset 2019-11-08 1:28:14 0 [Note] WSREP: ####### Adjusting cert position: 107829444 -> 107829462 2019-11-08 1:28:14 0 [Note] WSREP: Service thread queue flushed. 2019-11-08 1:28:14 0 [Note] WSREP: ####### Setting monitor position to 107829462 2019-11-08 1:28:14 0 [Note] WSREP: Lowest cert indnex boundary for CC from group: 107829462 2019-11-08 1:28:14 0 [Note] WSREP: Min available from gcache for CC from group: 107827514 2019-11-08 1:28:14 1 [Note] InnoDB: BF-BF X lock conflict,mode: 1027 supremum: 0conflicts states: my executing locked committing RECORD LOCKS space id 286102 page no 3 n bits 88 index PRIMARY of table `skuvault`.`jobs` trx id 1161492874 lock_mode X locks rec but not gap Record lock, heap no 15 PHYSICAL RECORD: n_fields 15; compact format; info bits 0 0: len 4; hex 80000005; asc ;; 1: len 6; hex 0000453af9d3; asc E: ;; 2: len 7; hex 590001003811af; asc Y 8 ;; 3: len 20; hex 4d6f7665206461746120746f204d617269614442; asc Move data to MariaDB;; 4: len 1; hex 81; asc ;; 5: len 1; hex 80; asc ;; 6: len 1; hex 80; asc ;; 7: len 8; hex 99a4905c4b07b4a8; asc \K ;; 8: len 1; hex 02; asc ;; 9: len 30; hex 656d756e697a4067726561746865616c7468776f726b732e636f6d3b6e67; asc emuniz@greathealthworks.com;ng; (total 202 bytes); 10: len 30; hex 2019-11-08 1:28:14 0 [Note] WSREP: ####### processing CC 107829467, from IST, ordered 2019-11-08 1:28:14 0 [Note] WSREP: ####### drain monitors upto 107829462 696665726e616e64657a4067726561746865616c7468776f726b732e636f; asc ifernandez@greathealthworks.co; (total 31 bytes); 11: SQL NULL; 12: len 1; hex 80; asc ;; 13: len 4; hex 80000005; asc ;; 14: SQL DEFAULT; 2019-11-08 1:28:14 1 [Note] InnoDB: SQL1: update jobs set executing=1, executing_date='2019-11-08 05:50:01.005' where id=5R�] SQL2: 2019-11-08 1:28:14 1 [Note] InnoDB: BF-BF X lock conflict,mode: 1027 supremum: 0conflicts states: my executing locked committing RECORD LOCKS space id 286102 page no 3 n bits 88 index PRIMARY of table `skuvault`.`jobs` trx id 1161492947 lock_mode X locks rec but not gap Record lock, heap no 15 PHYSICAL RECORD: n_fields 15; compact format; info bits 0 0: len 4; hex 80000005; asc ;; 1: len 6; hex 0000453af9d3; asc E: ;; 2: len 7; hex 590001003811af; asc Y 8 ;; 3: len 20; hex 4d6f7665206461746120746f204d617269614442; asc Move data to MariaDB;; 4: len 1; hex 81; asc ;; 5: len 1; hex 80; asc ;; 6: len 1; hex 80; asc ;; 7: len 8; hex 99a4905c4b07b4a8; asc \K ;; 8: len 1; hex 02; asc ;; 9: len 30; hex 656d756e697a4067726561746865616c7468776f726b732e636f6d3b6e67; asc emuniz@greathealthworks.com;ng; (total 202 bytes); 10: len 30; hex 696665726e616e64657a4067726561746865616c7468776f726b732e636f; asc ifernandez@greathealthworks.co; (total 31 bytes); 11: SQL NULL; 12: len 1; hex 80; asc ;; 13: len 4; hex 80000005; asc ;; 14: SQL DEFAULT; 2019-11-08 1:28:14 1 [Note] InnoDB: SQL1: update jobs set executing=1, executing_date='2019-11-08 05:50:01.005' where id=5R�] SQL2: 2019-11-08 1:28:15 0 [Note] InnoDB: To roll back: 2 transactions, 9456 rows 2019-11-08 1:28:30 0 [Note] InnoDB: To roll back: 2 transactions, 7574 rows 2019-11-08 1:28:45 0 [Note] InnoDB: To roll back: 2 transactions, 5466 rows 2019-11-08 1:29:00 0 [Note] InnoDB: To roll back: 2 transactions, 2516 rows 2019-11-08 1:29:07 0 [Note] InnoDB: Rolled back recovered transaction 1161492834 2019-11-08 1:29:07 0 [Note] InnoDB: Rolled back recovered transaction 1161492832 2019-11-08 1:29:07 0 [Note] InnoDB: Rollback of non-prepared transactions completed 2019-11-08 1:30:17 0 [Note] WSREP: Member 1.0 (Galeradbprod02) desyncs itself from group 2019-11-08 1:30:17 0 [Note] WSREP: Member 1.0 (Galeradbprod02) resyncs itself to group 2019-11-08 1:30:17 0 [Note] WSREP: Member 1.0 (Galeradbprod02) synced with group. 2019-11-08 1:34:05 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://10.20.11.46:57668 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 1:34:05 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to 0602cb8a ssl://10.20.11.46:4567 2019-11-08 1:34:05 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 2019-11-08 1:34:08 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off 2019-11-08 1:35:17 0 [Note] WSREP: Member 1.0 (Galeradbprod02) desyncs itself from group 2019-11-08 1:35:18 0 [Note] WSREP: Member 1.0 (Galeradbprod02) resyncs itself to group 2019-11-08 1:35:18 0 [Note] WSREP: Member 1.0 (Galeradbprod02) synced with group. 2019-11-08 1:40:27 0 [Note] WSREP: SSL handshake successful, remote endpoint ssl://172.25.1.10:51004 local endpoint ssl://10.1.30.60:4567 cipher: ECDHE-RSA-AES256-GCM-SHA384 compression: none 2019-11-08 1:40:27 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting on, nonlive peers: ssl://172.25.1.10:4567 2019-11-08 1:40:27 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') connection established to a67447ac ssl://172.25.1.10:4567 2019-11-08 1:40:27 0 [Note] WSREP: declaring 0602cb8a at ssl://10.20.11.46:4567 stable 2019-11-08 1:40:27 0 [Note] WSREP: declaring 678b492c at ssl://10.1.30.61:4567 stable 2019-11-08 1:40:27 0 [Note] WSREP: declaring a67447ac at ssl://172.25.1.10:4567 stable 2019-11-08 1:40:27 0 [Note] WSREP: declaring c4669934 at ssl://10.20.11.45:4567 stable 2019-11-08 1:40:28 0 [Note] WSREP: Node 0602cb8a state prim 2019-11-08 1:40:28 0 [Note] WSREP: view(view_id(PRIM,0602cb8a,152) memb { 0602cb8a,1 678b492c,0 a67447ac,2 c4669934,1 f19a4d7f,0 } joined { } left { } partitioned { }) 2019-11-08 1:40:28 0 [Note] WSREP: save pc into disk 2019-11-08 1:40:28 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 4, memb_num = 5 2019-11-08 1:40:28 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2019-11-08 1:40:28 0 [Note] WSREP: STATE EXCHANGE: sent state msg: a74acf01-01f2-11ea-942b-5393bb976bf1 2019-11-08 1:40:28 0 [Note] WSREP: STATE EXCHANGE: got state msg: a74acf01-01f2-11ea-942b-5393bb976bf1 from 0 (FLLGaleraDBColo2) 2019-11-08 1:40:28 0 [Note] WSREP: STATE EXCHANGE: got state msg: a74acf01-01f2-11ea-942b-5393bb976bf1 from 1 (Galeradbprod02) 2019-11-08 1:40:28 0 [Note] WSREP: STATE EXCHANGE: got state msg: a74acf01-01f2-11ea-942b-5393bb976bf1 from 4 (HQGaleraDBProd1) 2019-11-08 1:40:28 0 [Note] WSREP: STATE EXCHANGE: got state msg: a74acf01-01f2-11ea-942b-5393bb976bf1 from 3 (GaleraDBColo1) 2019-11-08 1:40:29 0 [Note] WSREP: STATE EXCHANGE: got state msg: a74acf01-01f2-11ea-942b-5393bb976bf1 from 2 (garb) 2019-11-08 1:40:29 0 [Note] WSREP: Quorum results: version = 5, component = PRIMARY, conf_id = 4, members = 3/5 (joined/total), act_id = 107831078, last_appl. = 0, protocols = 1/10/4 (gcs/repl/appl), vote policy= 0, group UUID = 1cdf2a6b-ce84-11e8-88a7-93f3d3b4ed22 2019-11-08 1:40:29 0 [Note] WSREP: Writing down CC checksum: 5223b1c7 177d6ffe a531ea80 e7e58843 at offset 384 2019-11-08 1:40:29 0 [Note] WSREP: Flow-control interval: [286, 358] 2019-11-08 1:40:29 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 1:40:29 0 [Note] WSREP: Member 2.2 (garb) requested state transfer from '*any*'. Selected 3.1 (GaleraDBColo1)(SYNCED) as donor. 2019-11-08 1:40:29 0 [Note] WSREP: 2.2 (garb): State transfer from 3.1 (GaleraDBColo1) complete. 2019-11-08 1:40:29 0 [Note] WSREP: 3.1 (GaleraDBColo1): State transfer to 2.2 (garb) complete. 2019-11-08 1:40:29 0 [Note] WSREP: Member 2.2 (garb) synced with group. 2019-11-08 1:40:29 0 [Note] WSREP: Member 3.1 (GaleraDBColo1) synced with group. 2019-11-08 1:40:30 0 [Note] WSREP: (f19a4d7f, 'ssl://0.0.0.0:4567') turning message relay requesting off 2019-11-08 1:41:12 0 [Note] /usr/sbin/mysqld (initiated by: unknown): Normal shutdown 2019-11-08 1:41:12 0 [Note] WSREP: Shutdown replication 2019-11-08 1:41:12 0 [Note] WSREP: Server status change joined -> disconnecting 2019-11-08 1:41:12 0 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-11-08 1:41:12 0 [Note] WSREP: Closing send monitor... 2019-11-08 1:41:12 0 [Note] WSREP: Closed send monitor. 2019-11-08 1:41:12 0 [Note] WSREP: gcomm: terminating thread 2019-11-08 1:41:12 0 [Note] WSREP: gcomm: joining thread 2019-11-08 1:41:12 0 [Note] WSREP: gcomm: closing backend 2019-11-08 1:41:13 0 [Note] WSREP: view(view_id(NON_PRIM,0602cb8a,152) memb { f19a4d7f,0 } joined { } left { } partitioned { 0602cb8a,1 678b492c,0 a67447ac,2 c4669934,1 }) 2019-11-08 1:41:13 0 [Note] WSREP: PC protocol downgrade 1 -> 0 2019-11-08 1:41:13 0 [Note] WSREP: view((empty)) 2019-11-08 1:41:13 0 [Note] WSREP: gcomm: closed 2019-11-08 1:41:13 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 2019-11-08 1:41:13 0 [Note] WSREP: Writing down CC checksum: 7151e494 5f3cf73f 364e791b 7ef788ff at offset 128 2019-11-08 1:41:13 0 [Note] WSREP: Flow-control interval: [128, 160] 2019-11-08 1:41:13 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 1:41:13 0 [Note] WSREP: Received NON-PRIMARY. 2019-11-08 1:41:13 0 [Note] WSREP: Shifting JOINER -> OPEN (TO: 107831079) 2019-11-08 1:41:13 0 [Note] WSREP: Received self-leave message. 2019-11-08 1:41:13 0 [Note] WSREP: Writing down CC checksum: 4b2707b5 8f7d8a17 d69ecd9d 73b5096e at offset 64 2019-11-08 1:41:13 0 [Note] WSREP: Flow-control interval: [0, 0] 2019-11-08 1:41:13 0 [Note] WSREP: Trying to continue unpaused monitor 2019-11-08 1:41:13 0 [Note] WSREP: Received SELF-LEAVE. Closing connection. 2019-11-08 1:41:13 0 [Note] WSREP: Shifting OPEN -> CLOSED (TO: -1) 2019-11-08 1:41:13 135 [Warning] Aborted connection 135 to db: 'unconnected' user: 'maxscale' host: '10.1.10.219' (Got an error reading communication packets) 2019-11-08 1:41:13 0 [Note] WSREP: RECV thread exiting 0: Success 2019-11-08 1:41:13 0 [Note] WSREP: recv_thread() joined. 2019-11-08 1:41:13 0 [Note] WSREP: Closing replication queue. 2019-11-08 1:41:13 0 [Note] WSREP: Closing slave action queue. 2019-11-08 1:41:13 202 [Warning] Aborted connection 202 to db: 'unconnected' user: 'percona_mon' host: 'localhost' (Got an error writing communication packets)