## // table somedb_staging_api.bookings was myisam // ##

May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table somedb_staging_api.bookings; Duplicate entry '545' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 612, Internal MariaDB error code: 1062
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [Warning] WSREP: RBR event 2 Write_rows_v1 apply warning: 121, 10542475
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [Warning] WSREP: Failed to apply app buffer: seqno: 10542475, status: 1
May 25 11:32:11 wssql-ix01 mysqld: #011 at galera/src/trx_handle.cpp:apply():340
May 25 11:32:11 wssql-ix01 mysqld: Retrying 2th time
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table somedb_staging_api.bookings; Duplicate entry '545' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 612, Internal MariaDB error code: 1062
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [Warning] WSREP: RBR event 2 Write_rows_v1 apply warning: 121, 10542475
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [Warning] WSREP: Failed to apply app buffer: seqno: 10542475, status: 1
May 25 11:32:11 wssql-ix01 mysqld: #011 at galera/src/trx_handle.cpp:apply():340
May 25 11:32:11 wssql-ix01 mysqld: Retrying 3th time
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table somedb_staging_api.bookings; Duplicate entry '545' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 612, Internal MariaDB error code: 1062
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [Warning] WSREP: RBR event 2 Write_rows_v1 apply warning: 121, 10542475
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [Warning] WSREP: Failed to apply app buffer: seqno: 10542475, status: 1
May 25 11:32:11 wssql-ix01 mysqld: #011 at galera/src/trx_handle.cpp:apply():340
May 25 11:32:11 wssql-ix01 mysqld: Retrying 4th time
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table somedb_staging_api.bookings; Duplicate entry '545' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 612, Internal MariaDB error code: 1062
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [Warning] WSREP: RBR event 2 Write_rows_v1 apply warning: 121, 10542475
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [ERROR] WSREP: Failed to apply trx: source: 9df418b3-e288-11e4-841e-c34bc61e6237 version: 3 local: 0 state: APPLYING flags: 1 conn_id: 2813010 trx_id: 27842910 seqnos (l: 9605988, g: 10542475, s: 10542474, d: 10542262, ts: 14855982663430919)
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [ERROR] WSREP: Failed to apply trx 10542475 4 times
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [ERROR] WSREP: Node consistency compromized, aborting...
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [Note] WSREP: Closing send monitor...
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [Note] WSREP: Closed send monitor.
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [Note] WSREP: gcomm: terminating thread
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [Note] WSREP: gcomm: joining thread
May 25 11:32:11 wssql-ix01 mysqld: 150525 11:32:11 [Note] WSREP: gcomm: closing backend
May 25 11:32:12 wssql-ix01 mysqld: 150525 11:32:12 [Note] WSREP: view(view_id(NON_PRIM,4ffae6ea-e288-11e4-bf6c-a7494d4cb905,33) memb {
May 25 11:32:12 wssql-ix01 mysqld: #011cba8d378-e288-11e4-b46a-6ea890248222,0
May 25 11:32:12 wssql-ix01 mysqld: } joined {
May 25 11:32:12 wssql-ix01 mysqld: } left {
May 25 11:32:12 wssql-ix01 mysqld: } partitioned {
May 25 11:32:12 wssql-ix01 mysqld: #0114ffae6ea-e288-11e4-bf6c-a7494d4cb905,0
May 25 11:32:12 wssql-ix01 mysqld: #0119df418b3-e288-11e4-841e-c34bc61e6237,0
May 25 11:32:12 wssql-ix01 mysqld: })
May 25 11:32:12 wssql-ix01 mysqld: 150525 11:32:12 [Note] WSREP: view((empty))
May 25 11:32:12 wssql-ix01 mysqld: 150525 11:32:12 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
May 25 11:32:12 wssql-ix01 mysqld: 150525 11:32:12 [Note] WSREP: gcomm: closed
May 25 11:32:12 wssql-ix01 mysqld: 150525 11:32:12 [Note] WSREP: Flow-control interval: [16, 16]
May 25 11:32:12 wssql-ix01 mysqld: 150525 11:32:12 [Note] WSREP: Received NON-PRIMARY.
May 25 11:32:12 wssql-ix01 mysqld: 150525 11:32:12 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 10542477)
May 25 11:32:12 wssql-ix01 mysqld: 150525 11:32:12 [Note] WSREP: Received self-leave message.
May 25 11:32:12 wssql-ix01 mysqld: 150525 11:32:12 [Note] WSREP: Flow-control interval: [0, 0]
May 25 11:32:12 wssql-ix01 mysqld: 150525 11:32:12 [Note] WSREP: Received SELF-LEAVE. Closing connection.
May 25 11:32:12 wssql-ix01 mysqld: 150525 11:32:12 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 10542477)
May 25 11:32:12 wssql-ix01 mysqld: 150525 11:32:12 [Note] WSREP: RECV thread exiting 0: Success
May 25 11:32:12 wssql-ix01 mysqld: 150525 11:32:12 [Note] WSREP: recv_thread() joined.
May 25 11:32:12 wssql-ix01 mysqld: 150525 11:32:12 [Note] WSREP: Closing replication queue.
May 25 11:32:12 wssql-ix01 mysqld: 150525 11:32:12 [Note] WSREP: Closing slave action queue.
May 25 11:32:12 wssql-ix01 mysqld: 150525 11:32:12 [Note] WSREP: /usr/sbin/mysqld: Terminated.
May 25 11:32:14 wssql-ix01 mysqld_safe: Number of processes running now: 0
May 25 11:32:14 wssql-ix01 mysqld_safe: WSREP: not restarting wsrep node automatically
May 25 11:32:14 wssql-ix01 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
May 25 11:32:26 wssql-ix01 monit[2279]: 'mysql' process is not running
May 25 11:32:26 wssql-ix01 monit[2279]: 'mysql' trying to restart
May 25 11:32:26 wssql-ix01 monit[2279]: 'mysql' start: /etc/init.d/mysql
May 25 11:32:27 wssql-ix01 mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
May 25 11:32:27 wssql-ix01 mysqld_safe: WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.1nP3Ev' --pid-file='/var/lib/mysql/wssql-ix01-recover.pid'
May 25 11:32:56 wssql-ix01 monit[2279]: 'mysql' failed to start
May 25 11:32:57 wssql-ix01 /etc/init.d/mysql[8835]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
May 25 11:32:57 wssql-ix01 /etc/init.d/mysql[8835]: #007/usr/bin/mysqladmin: connect to server at 'localhost' failed
May 25 11:32:57 wssql-ix01 /etc/init.d/mysql[8835]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111 "Connection refused")'
May 25 11:32:57 wssql-ix01 /etc/init.d/mysql[8835]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
May 25 11:32:57 wssql-ix01 /etc/init.d/mysql[8835]:
May 25 11:33:16 wssql-ix01 monit[2279]: 'mysql' process is not running
May 25 11:33:16 wssql-ix01 monit[2279]: 'mysql' trying to restart
May 25 11:33:16 wssql-ix01 monit[2279]: 'mysql' start: /etc/init.d/mysql
May 25 11:33:16 wssql-ix01 mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
May 25 11:33:16 wssql-ix01 mysqld_safe: WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.eLVIxR' --pid-file='/var/lib/mysql/wssql-ix01-recover.pid'
May 25 11:33:46 wssql-ix01 monit[2279]: 'mysql' failed to start
May 25 11:33:46 wssql-ix01 /etc/init.d/mysql[9756]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
May 25 11:33:46 wssql-ix01 /etc/init.d/mysql[9756]: #007/usr/bin/mysqladmin: connect to server at 'localhost' failed
May 25 11:33:46 wssql-ix01 /etc/init.d/mysql[9756]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111 "Connection refused")'
May 25 11:33:46 wssql-ix01 /etc/init.d/mysql[9756]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
May 25 11:33:46 wssql-ix01 /etc/init.d/mysql[9756]:
May 25 11:34:01 wssql-ix01 mysqld_safe: WSREP: Recovered position 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10542474
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: wsrep_start_position var submitted: '5d3b58bf-7bbe-11e4-9047-d6a833168dde:10542474'
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: Read nil XID from storage engines, skipping position init
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: wsrep_load(): Galera 25.3.5-wheezy(rXXXX) by Codership Oy <info@codership.com> loaded successfully.
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: CRC-32C: using hardware acceleration.
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: Passing config to GCS: base_host = 10.3.134.16; base_port = 4567; cert.log_conflicts = no; debug = no; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 1; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 256M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.bootstrap = true; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.version = 0; pc.wait_prim = false; pc.wait
May 25 11:34:01 wssql-ix01 mysqld: _prim_timeout = P30S; p
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: Service thread queue flushed.
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: wsrep_sst_grab()
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: Start replication
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: protonet asio version 0
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: Using CRC-32C (optimized) for message checksums.
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: backend: asio
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: GMCast version 0
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: (2d505405-02c1-11e5-a13f-862cea9319ca, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: (2d505405-02c1-11e5-a13f-862cea9319ca, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: EVS version 0
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: PC version 0
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: gcomm: connecting to group 'websolution', peer 'wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567'
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Warning] WSREP: (2d505405-02c1-11e5-a13f-862cea9319ca, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' points to own listening address, blacklisting
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: (2d505405-02c1-11e5-a13f-862cea9319ca, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid 2d505405-02c1-11e5-a13f-862cea9319ca is blacklisted, skipping
May 25 11:34:01  mysqld: last message repeated 3 times
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: declaring 4ffae6ea-e288-11e4-bf6c-a7494d4cb905 stable
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: declaring 9df418b3-e288-11e4-841e-c34bc61e6237 stable
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: Node 4ffae6ea-e288-11e4-bf6c-a7494d4cb905 state prim
May 25 11:34:01 wssql-ix01 mysqld: 150525 11:34:01 [Note] WSREP: view(view_id(PRIM,2d505405-02c1-11e5-a13f-862cea9319ca,35) memb {
May 25 11:34:01 wssql-ix01 mysqld: #0112d505405-02c1-11e5-a13f-862cea9319ca,0
May 25 11:34:01 wssql-ix01 mysqld: #0114ffae6ea-e288-11e4-bf6c-a7494d4cb905,0
May 25 11:34:01 wssql-ix01 mysqld: #0119df418b3-e288-11e4-841e-c34bc61e6237,0
May 25 11:34:01 wssql-ix01 mysqld: } joined {
May 25 11:34:01 wssql-ix01 mysqld: } left {
May 25 11:34:01 wssql-ix01 mysqld: } partitioned {
May 25 11:34:01 wssql-ix01 mysqld: })
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: gcomm: connected
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: Opened channel 'websolution'
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 3
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: Waiting for SST to complete.
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 2da2ea0a-02c1-11e5-8853-97858826ddaf
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: STATE EXCHANGE: sent state msg: 2da2ea0a-02c1-11e5-8853-97858826ddaf
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: STATE EXCHANGE: got state msg: 2da2ea0a-02c1-11e5-8853-97858826ddaf from 0 (wssql-ix01)
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: STATE EXCHANGE: got state msg: 2da2ea0a-02c1-11e5-8853-97858826ddaf from 2 (wssql-ix02)
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: STATE EXCHANGE: got state msg: 2da2ea0a-02c1-11e5-8853-97858826ddaf from 1 (wssql-ix03)
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: Quorum results:
May 25 11:34:02 wssql-ix01 mysqld: #011version    = 3,
May 25 11:34:02 wssql-ix01 mysqld: #011component  = PRIMARY,
May 25 11:34:02 wssql-ix01 mysqld: #011conf_id    = 34,
May 25 11:34:02 wssql-ix01 mysqld: #011members    = 2/3 (joined/total),
May 25 11:34:02 wssql-ix01 mysqld: #011act_id     = 10542572,
May 25 11:34:02 wssql-ix01 mysqld: #011last_appl. = -1,
May 25 11:34:02 wssql-ix01 mysqld: #011protocols  = 0/5/3 (gcs/repl/appl),
May 25 11:34:02 wssql-ix01 mysqld: #011group UUID = 5d3b58bf-7bbe-11e4-9047-d6a833168dde
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: Flow-control interval: [28, 28]
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 10542572)
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: State transfer required:
May 25 11:34:02 wssql-ix01 mysqld: #011Group state: 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10542572
May 25 11:34:02 wssql-ix01 mysqld: #011Local state: 00000000-0000-0000-0000-000000000000:-1
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: New cluster view: global state: 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10542572, view# 35: Primary, number of nodes: 3, my index: 0, protocol version 3
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Warning] WSREP: Gap in state sequence. Need state transfer.
May 25 11:34:02 wssql-ix01 mysqld: 150525 11:34:02 [Note] WSREP: Running: 'wsrep_sst_xtrabackup --role 'joiner' --address '10.3.134.16' --auth 'replication-user:f02266aaea02' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --parent '9820' --binlog '/var/log/mysql/mariadb-bin' '
May 25 11:34:03 wssql-ix01 mysqld: WSREP_SST: [INFO] Streaming with tar (20150525 11:34:03.208)
May 25 11:34:03 wssql-ix01 mysqld: WSREP_SST: [INFO] Using socat as streamer (20150525 11:34:03.231)
May 25 11:34:03 wssql-ix01 mysqld: WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:4444,reuseaddr stdio | tar xfi - --recursive-unlink -h; RC=( ${PIPESTATUS[@]} ) (20150525 11:34:03.499)
May 25 11:34:04 wssql-ix01 mysqld: 150525 11:34:04 [Note] WSREP: Prepared SST request: xtrabackup|10.3.134.16:4444/xtrabackup_sst
May 25 11:34:04 wssql-ix01 mysqld: 150525 11:34:04 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
May 25 11:34:04 wssql-ix01 mysqld: 150525 11:34:04 [Note] WSREP: REPL Protocols: 5 (3, 1)
May 25 11:34:04 wssql-ix01 mysqld: 150525 11:34:04 [Note] WSREP: Service thread queue flushed.
May 25 11:34:04 wssql-ix01 mysqld: 150525 11:34:04 [Note] WSREP: Assign initial position for certification: 10542572, protocol version: 3
May 25 11:34:04 wssql-ix01 mysqld: 150525 11:34:04 [Note] WSREP: Service thread queue flushed.
May 25 11:34:04 wssql-ix01 mysqld: 150525 11:34:04 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (5d3b58bf-7bbe-11e4-9047-d6a833168dde): 1 (Operation not permitted)
May 25 11:34:04 wssql-ix01 mysqld: #011 at galera/src/replicator_str.cpp:prepare_for_IST():447. IST will be unavailable.
May 25 11:34:04 wssql-ix01 mysqld: 150525 11:34:04 [Note] WSREP: Member 0.0 (wssql-ix01) requested state transfer from '*any*'. Selected 1.0 (wssql-ix03)(SYNCED) as donor.
May 25 11:34:04 wssql-ix01 mysqld: 150525 11:34:04 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 10542572)
May 25 11:34:04 wssql-ix01 mysqld: 150525 11:34:04 [Note] WSREP: Requesting state transfer: success, donor: 1
May 25 11:34:06 wssql-ix01 monit[2279]: 'mysql' process is not running
May 25 11:34:06 wssql-ix01 monit[2279]: 'mysql' trying to restart
May 25 11:34:06 wssql-ix01 monit[2279]: 'mysql' start: /etc/init.d/mysql
May 25 11:34:07 wssql-ix01 mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
May 25 11:34:07 wssql-ix01 mysqld_safe: WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.iK2h1O' --pid-file='/var/lib/mysql/wssql-ix01-recover.pid'
May 25 11:34:10 wssql-ix01 mysqld_safe: WSREP: Recovered position 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10542474
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: wsrep_start_position var submitted: '5d3b58bf-7bbe-11e4-9047-d6a833168dde:10542474'
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: Read nil XID from storage engines, skipping position init
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: wsrep_load(): Galera 25.3.5-wheezy(rXXXX) by Codership Oy <info@codership.com> loaded successfully.
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: CRC-32C: using hardware acceleration.
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: Passing config to GCS: base_host = 10.3.134.16; base_port = 4567; cert.log_conflicts = no; debug = no; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 1; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 256M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.bootstrap = true; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.version = 0; pc.wait_prim = false; pc.wait
May 25 11:34:10 wssql-ix01 mysqld: _prim_timeout = P30S; p
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: Service thread queue flushed.
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: wsrep_sst_grab()
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: Start replication
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: protonet asio version 0
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: Using CRC-32C (optimized) for message checksums.
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: backend: asio
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: GMCast version 0
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: (32daf894-02c1-11e5-b056-469913ec59ce, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: (32daf894-02c1-11e5-b056-469913ec59ce, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: EVS version 0
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: PC version 0
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: gcomm: connecting to group 'websolution', peer 'wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567'
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [ERROR] WSREP: Address already in use
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [ERROR] WSREP: failed to open gcomm backend connection: 98: error while trying to listen 'tcp://0.0.0.0:4567?socket.non_blocking=1', asio error 'Address already in use': 98 (Address already in use)
May 25 11:34:10 wssql-ix01 mysqld: #011 at gcomm/src/asio_tcp.cpp:listen():814
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [ERROR] WSREP: gcs/src/gcs_core.c:gcs_core_open():202: Failed to open backend connection: -98 (Address already in use)
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [ERROR] WSREP: gcs/src/gcs.c:gcs_open():1291: Failed to open channel 'websolution' at 'gcomm://wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567': -98 (Address already in use)
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [ERROR] WSREP: gcs connect failed: Address already in use
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [ERROR] WSREP: wsrep::connect() failed: 7
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [ERROR] Aborting
May 25 11:34:10 wssql-ix01 mysqld:
May 25 11:34:10 wssql-ix01 mysqld: 150525 11:34:10 [Note] WSREP: Service disconnected.
May 25 11:34:11 wssql-ix01 mysqld: 150525 11:34:11 [Note] WSREP: Some threads may fail to exit.
May 25 11:34:11 wssql-ix01 mysqld: 150525 11:34:11 [Note] /usr/sbin/mysqld: Shutdown complete
May 25 11:34:11 wssql-ix01 mysqld:
May 25 11:34:12 wssql-ix01 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
May 25 11:34:14 wssql-ix01 mysqld_safe: WSREP: Recovered position 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10542474
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: wsrep_start_position var submitted: '5d3b58bf-7bbe-11e4-9047-d6a833168dde:10542474'
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: Read nil XID from storage engines, skipping position init
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: wsrep_load(): Galera 25.3.5-wheezy(rXXXX) by Codership Oy <info@codership.com> loaded successfully.
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: CRC-32C: using hardware acceleration.
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: Passing config to GCS: base_host = 10.3.134.16; base_port = 4567; cert.log_conflicts = no; debug = no; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 1; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 256M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.bootstrap = true; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.version = 0; pc.wait_prim = false; pc.wait
May 25 11:34:14 wssql-ix01 mysqld: _prim_timeout = P30S; p
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: Service thread queue flushed.
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: wsrep_sst_grab()
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: Start replication
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: protonet asio version 0
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: Using CRC-32C (optimized) for message checksums.
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: backend: asio
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: GMCast version 0
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: (353755eb-02c1-11e5-a834-33c11b4acf00, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: (353755eb-02c1-11e5-a834-33c11b4acf00, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: EVS version 0
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: PC version 0
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: gcomm: connecting to group 'websolution', peer 'wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567'
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [ERROR] WSREP: Address already in use
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [ERROR] WSREP: failed to open gcomm backend connection: 98: error while trying to listen 'tcp://0.0.0.0:4567?socket.non_blocking=1', asio error 'Address already in use': 98 (Address already in use)
May 25 11:34:14 wssql-ix01 mysqld: #011 at gcomm/src/asio_tcp.cpp:listen():814
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [ERROR] WSREP: gcs/src/gcs_core.c:gcs_core_open():202: Failed to open backend connection: -98 (Address already in use)
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [ERROR] WSREP: gcs/src/gcs.c:gcs_open():1291: Failed to open channel 'websolution' at 'gcomm://wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567': -98 (Address already in use)
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [ERROR] WSREP: gcs connect failed: Address already in use
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [ERROR] WSREP: wsrep::connect() failed: 7
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [ERROR] Aborting
May 25 11:34:14 wssql-ix01 mysqld:
May 25 11:34:14 wssql-ix01 mysqld: 150525 11:34:14 [Note] WSREP: Service disconnected.
May 25 11:34:15 wssql-ix01 mysqld: 150525 11:34:15 [Note] WSREP: Some threads may fail to exit.
May 25 11:34:15 wssql-ix01 mysqld: 150525 11:34:15 [Note] /usr/sbin/mysqld: Shutdown complete
May 25 11:34:15 wssql-ix01 mysqld:
May 25 11:34:15 wssql-ix01 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
May 25 11:34:36 wssql-ix01 monit[2279]: 'mysql' failed to start
May 25 11:34:42 wssql-ix01 /etc/init.d/mysql[10935]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
May 25 11:34:42 wssql-ix01 /etc/init.d/mysql[10935]: #007/usr/bin/mysqladmin: connect to server at 'localhost' failed
May 25 11:34:42 wssql-ix01 /etc/init.d/mysql[10935]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111 "Connection refused")'
May 25 11:34:42 wssql-ix01 /etc/init.d/mysql[10935]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
May 25 11:34:42 wssql-ix01 /etc/init.d/mysql[10935]:
May 25 11:34:56 wssql-ix01 monit[2279]: 'mysql' process is not running
May 25 11:34:56 wssql-ix01 monit[2279]: 'mysql' trying to restart
May 25 11:34:56 wssql-ix01 monit[2279]: 'mysql' start: /etc/init.d/mysql
May 25 11:34:56 wssql-ix01 mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
May 25 11:34:56 wssql-ix01 mysqld_safe: WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.YXVtuK' --pid-file='/var/lib/mysql/wssql-ix01-recover.pid'
May 25 11:35:26 wssql-ix01 monit[2279]: 'mysql' failed to start
May 25 11:35:27 wssql-ix01 /etc/init.d/mysql[12059]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
May 25 11:35:27 wssql-ix01 /etc/init.d/mysql[12059]: #007/usr/bin/mysqladmin: connect to server at 'localhost' failed
May 25 11:35:27 wssql-ix01 /etc/init.d/mysql[12059]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111 "Connection refused")'
May 25 11:35:27 wssql-ix01 /etc/init.d/mysql[12059]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
May 25 11:35:27 wssql-ix01 /etc/init.d/mysql[12059]:
May 25 11:35:46 wssql-ix01 monit[2279]: 'mysql' process is not running
May 25 11:35:46 wssql-ix01 monit[2279]: 'mysql' trying to restart
May 25 11:35:46 wssql-ix01 monit[2279]: 'mysql' start: /etc/init.d/mysql
May 25 11:35:46 wssql-ix01 mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
May 25 11:35:46 wssql-ix01 mysqld_safe: WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.Re3lot' --pid-file='/var/lib/mysql/wssql-ix01-recover.pid'
May 25 11:36:16 wssql-ix01 monit[2279]: 'mysql' failed to start
May 25 11:36:19 wssql-ix01 mysqld_safe: WSREP: Recovered position 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10542576
May 25 11:36:19 wssql-ix01 mysqld: 150525 11:36:19 [Note] WSREP: wsrep_start_position var submitted: '5d3b58bf-7bbe-11e4-9047-d6a833168dde:10542576'
May 25 11:36:19 wssql-ix01 mysqld: 150525 11:36:19 [Note] WSREP: Read nil XID from storage engines, skipping position init
May 25 11:36:19 wssql-ix01 mysqld: 150525 11:36:19 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
May 25 11:36:19 wssql-ix01 mysqld: 150525 11:36:19 [Note] WSREP: wsrep_load(): Galera 25.3.5-wheezy(rXXXX) by Codership Oy <info@codership.com> loaded successfully.
May 25 11:36:19 wssql-ix01 mysqld: 150525 11:36:19 [Note] WSREP: CRC-32C: using hardware acceleration.
May 25 11:36:19 wssql-ix01 mysqld: 150525 11:36:19 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
May 25 11:36:19 wssql-ix01 mysqld: 150525 11:36:19 [Note] WSREP: Passing config to GCS: base_host = 10.3.134.16; base_port = 4567; cert.log_conflicts = no; debug = no; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 1; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 256M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.bootstrap = true; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.version = 0; pc.wait_prim = false; pc.wait
May 25 11:36:19 wssql-ix01 mysqld: _prim_timeout = P30S; p
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [Note] WSREP: Service thread queue flushed.
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [Note] WSREP: wsrep_sst_grab()
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [Note] WSREP: Start replication
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [Note] WSREP: protonet asio version 0
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [Note] WSREP: Using CRC-32C (optimized) for message checksums.
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [Note] WSREP: backend: asio
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [Note] WSREP: GMCast version 0
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [Note] WSREP: (7fc65c8e-02c1-11e5-b95d-3346c582954c, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [Note] WSREP: (7fc65c8e-02c1-11e5-b95d-3346c582954c, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [Note] WSREP: EVS version 0
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [Note] WSREP: PC version 0
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [Note] WSREP: gcomm: connecting to group 'websolution', peer 'wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567'
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [ERROR] WSREP: Address already in use
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [ERROR] WSREP: failed to open gcomm backend connection: 98: error while trying to listen 'tcp://0.0.0.0:4567?socket.non_blocking=1', asio error 'Address already in use': 98 (Address already in use)
May 25 11:36:20 wssql-ix01 mysqld: #011 at gcomm/src/asio_tcp.cpp:listen():814
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [ERROR] WSREP: gcs/src/gcs_core.c:gcs_core_open():202: Failed to open backend connection: -98 (Address already in use)
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [ERROR] WSREP: gcs/src/gcs.c:gcs_open():1291: Failed to open channel 'websolution' at 'gcomm://wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567': -98 (Address already in use)
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [ERROR] WSREP: gcs connect failed: Address already in use
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [ERROR] WSREP: wsrep::connect() failed: 7
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [ERROR] Aborting
May 25 11:36:20 wssql-ix01 mysqld:
May 25 11:36:20 wssql-ix01 mysqld: 150525 11:36:20 [Note] WSREP: Service disconnected.
May 25 11:36:21 wssql-ix01 mysqld: 150525 11:36:21 [Note] WSREP: Some threads may fail to exit.
May 25 11:36:21 wssql-ix01 mysqld: 150525 11:36:21 [Note] /usr/sbin/mysqld: Shutdown complete
May 25 11:36:21 wssql-ix01 mysqld:
May 25 11:36:21 wssql-ix01 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
May 25 11:36:36 wssql-ix01 monit[2279]: 'mysql' process is not running
May 25 11:36:36 wssql-ix01 monit[2279]: 'mysql' trying to restart
May 25 11:36:36 wssql-ix01 monit[2279]: 'mysql' start: /etc/init.d/mysql
May 25 11:36:36 wssql-ix01 mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
May 25 11:36:36 wssql-ix01 mysqld_safe: WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.dKPTns' --pid-file='/var/lib/mysql/wssql-ix01-recover.pid'
May 25 11:36:47 wssql-ix01 /etc/init.d/mysql[13610]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
May 25 11:36:47 wssql-ix01 /etc/init.d/mysql[13610]: #007/usr/bin/mysqladmin: connect to server at 'localhost' failed
May 25 11:36:47 wssql-ix01 /etc/init.d/mysql[13610]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111 "Connection refused")'
May 25 11:36:47 wssql-ix01 /etc/init.d/mysql[13610]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
May 25 11:36:47 wssql-ix01 /etc/init.d/mysql[13610]:
May 25 11:37:00 wssql-ix01 monit[2279]: 'mysql' stop on user request
May 25 11:37:06 wssql-ix01 monit[2279]: 'mysql' failed to start
May 25 11:37:06 wssql-ix01 monit[2279]: 'mysql' stop action done
May 25 11:37:18 wssql-ix01 mysqld_safe: WSREP: Recovered position 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10542576
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: wsrep_start_position var submitted: '5d3b58bf-7bbe-11e4-9047-d6a833168dde:10542576'
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: Read nil XID from storage engines, skipping position init
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: wsrep_load(): Galera 25.3.5-wheezy(rXXXX) by Codership Oy <info@codership.com> loaded successfully.
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: CRC-32C: using hardware acceleration.
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: Passing config to GCS: base_host = 10.3.134.16; base_port = 4567; cert.log_conflicts = no; debug = no; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 1; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 256M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.bootstrap = true; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.version = 0; pc.wait_prim = false; pc.wait
May 25 11:37:18 wssql-ix01 mysqld: _prim_timeout = P30S; p
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: Service thread queue flushed.
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: wsrep_sst_grab()
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: Start replication
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: protonet asio version 0
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: Using CRC-32C (optimized) for message checksums.
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: backend: asio
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: GMCast version 0
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: (a29c8548-02c1-11e5-b275-8eace1a69c61, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: (a29c8548-02c1-11e5-b275-8eace1a69c61, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: EVS version 0
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: PC version 0
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: gcomm: connecting to group 'websolution', peer 'wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567'
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [ERROR] WSREP: Address already in use
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [ERROR] WSREP: failed to open gcomm backend connection: 98: error while trying to listen 'tcp://0.0.0.0:4567?socket.non_blocking=1', asio error 'Address already in use': 98 (Address already in use)
May 25 11:37:18 wssql-ix01 mysqld: #011 at gcomm/src/asio_tcp.cpp:listen():814
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [ERROR] WSREP: gcs/src/gcs_core.c:gcs_core_open():202: Failed to open backend connection: -98 (Address already in use)
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [ERROR] WSREP: gcs/src/gcs.c:gcs_open():1291: Failed to open channel 'websolution' at 'gcomm://wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567': -98 (Address already in use)
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [ERROR] WSREP: gcs connect failed: Address already in use
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [ERROR] WSREP: wsrep::connect() failed: 7
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [ERROR] Aborting
May 25 11:37:18 wssql-ix01 mysqld:
May 25 11:37:18 wssql-ix01 mysqld: 150525 11:37:18 [Note] WSREP: Service disconnected.
May 25 11:37:19 wssql-ix01 mysqld: 150525 11:37:19 [Note] WSREP: Some threads may fail to exit.
May 25 11:37:19 wssql-ix01 mysqld: 150525 11:37:19 [Note] /usr/sbin/mysqld: Shutdown complete
May 25 11:37:19 wssql-ix01 mysqld:
May 25 11:37:19 wssql-ix01 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
May 25 11:37:23 wssql-ix01 /etc/init.d/mysql[14071]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
May 25 11:37:23 wssql-ix01 /etc/init.d/mysql[14071]: #007/usr/bin/mysqladmin: connect to server at 'localhost' failed
May 25 11:37:23 wssql-ix01 /etc/init.d/mysql[14071]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111 "Connection refused")'
May 25 11:37:23 wssql-ix01 /etc/init.d/mysql[14071]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
May 25 11:37:23 wssql-ix01 /etc/init.d/mysql[14071]:
May 25 11:38:18 wssql-ix01 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
May 25 11:39:24 wssql-ix01 mysqld: 150525 11:39:24 [Note] WSREP: 1.0 (wssql-ix03): State transfer to 0.0 (wssql-ix01) complete.
May 25 11:39:24 wssql-ix01 mysqld: WSREP_SST: [INFO] Proceeding with SST (20150525 11:39:24.903)
May 25 11:39:24 wssql-ix01 mysqld: WSREP_SST: [INFO] Removing existing ib_logfile files (20150525 11:39:24.908)
May 25 11:39:25 wssql-ix01 mysqld: WSREP_SST: [INFO] Preparing the backup at /var/lib/mysql/ (20150525 11:39:25.748)
May 25 11:39:25 wssql-ix01 mysqld: WSREP_SST: [INFO] Evaluating innobackupex --defaults-file=/etc/mysql/my.cnf --apply-log $rebuildcmd ${DATA} &>${DATA}/innobackup.prepare.log (20150525 11:39:25.754)
May 25 11:39:58 wssql-ix01 mysqld: 150525 11:39:58 [Note] WSREP: Member 1.0 (wssql-ix03) synced with group.
May 25 11:40:08 wssql-ix01 mysqld: WSREP_SST: [INFO] Total time on joiner: 0 seconds (20150525 11:40:08.704)
May 25 11:40:08 wssql-ix01 mysqld: WSREP_SST: [INFO] Removing the sst_in_progress file (20150525 11:40:08.843)
May 25 11:40:08 wssql-ix01 mysqld: 150525 11:40:08 [Note] WSREP: SST complete, seqno: 10542576
May 25 11:40:08 wssql-ix01 mysqld: 2015-05-25 11:40:08 7f6e2182c780 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html.
May 25 11:40:08 wssql-ix01 mysqld: 150525 11:40:08 [Note] InnoDB: Using mutexes to ref count buffer pool pages
May 25 11:40:08 wssql-ix01 mysqld: 150525 11:40:08 [Note] InnoDB: The InnoDB memory heap is disabled
May 25 11:40:08 wssql-ix01 mysqld: 150525 11:40:08 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
May 25 11:40:08 wssql-ix01 mysqld: 150525 11:40:08 [Note] InnoDB: Memory barrier is not used
May 25 11:40:08 wssql-ix01 mysqld: 150525 11:40:08 [Note] InnoDB: Compressed tables use zlib 1.2.3.4
May 25 11:40:08 wssql-ix01 mysqld: 150525 11:40:08 [Note] InnoDB: Using Linux native AIO
May 25 11:40:08 wssql-ix01 mysqld: 150525 11:40:08 [Note] InnoDB: Using CPU crc32 instructions
May 25 11:40:08 wssql-ix01 mysqld: 150525 11:40:08 [Note] InnoDB: Initializing buffer pool, size = 7.0G
May 25 11:40:09 wssql-ix01 mysqld: 150525 11:40:09 [Note] InnoDB: Completed initialization of buffer pool
May 25 11:40:10 wssql-ix01 mysqld: 150525 11:40:10 [Note] InnoDB: Highest supported file format is Barracuda.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 6 log sequence number 35648491110
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 941 log sequence number 35648491110
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 45 log sequence number 35648491149
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 382 log sequence number 35648491149
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 46 log sequence number 35648491188
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 921 log sequence number 35648491188
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 47 log sequence number 35648491227
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 922 log sequence number 35648491227
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 48 log sequence number 35648491266
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 348 log sequence number 35648491266
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 49 log sequence number 35648491305
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 485 log sequence number 35648491305
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 50 log sequence number 35648491344
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 904 log sequence number 35648491344
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 53 log sequence number 35648491383
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 458 log sequence number 35648491383
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 54 log sequence number 35648491422
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 596 log sequence number 35648491422
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 55 log sequence number 35648491461
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 341 log sequence number 35648491461
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 56 log sequence number 35648491500
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 902 log sequence number 35648491500
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 57 log sequence number 35648491555
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 737 log sequence number 35648491555
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 58 log sequence number 35648491594
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 335 log sequence number 35648491594
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 59 log sequence number 35648491633
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 402 log sequence number 35648491633
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 60 log sequence number 35648491672
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 957 log sequence number 35648491672
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: for more information.
May 25 11:40:10 wssql-ix01 mysqld: 2015-05-25 11:40:10 7f6e2182c780 InnoDB: Error: page 61 log sequence number 35648491711
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: is in the future! Current system log sequence number 35283240302.
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: Your database may be corrupt or you may have copied the InnoDB
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: tablespace but not the InnoDB log files. See
May 25 11:40:10 wssql-ix01 mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
May 25 11:40:14 wssql-ix01 mysqld: 150525 11:40:14 [Note] Plugin 'FEEDBACK' is disabled.
May 25 11:40:14 wssql-ix01 mysqld: 150525 11:40:14 [Note] Server socket created on IP: '0.0.0.0'.
May 25 11:40:14 wssql-ix01 mysqld: 150525 11:40:14 [ERROR] mysqld: Can't create/write to file '/var/run/mysqld/mysqld.pid' (Errcode: 13 "Permission denied")
May 25 11:40:14 wssql-ix01 mysqld: 150525 11:40:14 [ERROR] Can't start server: can't create PID file: Permission denied
May 25 11:40:28 wssql-ix01 monit[2279]: 'mysql' start on user request
May 25 11:40:28 wssql-ix01 monit[2279]: 'mysql' start: /etc/init.d/mysql
May 25 11:40:29 wssql-ix01 mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
May 25 11:40:30 wssql-ix01 mysqld_safe: WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.QG8TbG' --pid-file='/var/lib/mysql/wssql-ix01-recover.pid'
May 25 11:40:58 wssql-ix01 monit[2279]: 'mysql' failed to start
May 25 11:40:58 wssql-ix01 monit[2279]: 'mysql' start action done
May 25 11:40:58 wssql-ix01 mysqld_safe: WSREP: Recovered position 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10542576
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: wsrep_start_position var submitted: '5d3b58bf-7bbe-11e4-9047-d6a833168dde:10542576'
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: Read nil XID from storage engines, skipping position init
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: wsrep_load(): Galera 25.3.5-wheezy(rXXXX) by Codership Oy <info@codership.com> loaded successfully.
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: CRC-32C: using hardware acceleration.
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: Passing config to GCS: base_host = 10.3.134.16; base_port = 4567; cert.log_conflicts = no; debug = no; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 1; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 256M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.bootstrap = true; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.version = 0; pc.wait_prim = false; pc.wait
May 25 11:40:58 wssql-ix01 mysqld: _prim_timeout = P30S; p
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: Service thread queue flushed.
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: wsrep_sst_grab()
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: Start replication
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: protonet asio version 0
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: Using CRC-32C (optimized) for message checksums.
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: backend: asio
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: GMCast version 0
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: (25fc17e7-02c2-11e5-8c5a-7b5b15f1781c, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: (25fc17e7-02c2-11e5-8c5a-7b5b15f1781c, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: EVS version 0
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: PC version 0
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: gcomm: connecting to group 'websolution', peer 'wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567'
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Warning] WSREP: (25fc17e7-02c2-11e5-8c5a-7b5b15f1781c, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' points to own listening address, blacklisting
May 25 11:40:58 wssql-ix01 mysqld: 150525 11:40:58 [Note] WSREP: (25fc17e7-02c2-11e5-8c5a-7b5b15f1781c, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid 25fc17e7-02c2-11e5-8c5a-7b5b15f1781c is blacklisted, skipping
May 25 11:40:59  mysqld: last message repeated 3 times
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: declaring 4ffae6ea-e288-11e4-bf6c-a7494d4cb905 stable
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: declaring 9df418b3-e288-11e4-841e-c34bc61e6237 stable
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: Node 4ffae6ea-e288-11e4-bf6c-a7494d4cb905 state prim
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: view(view_id(PRIM,25fc17e7-02c2-11e5-8c5a-7b5b15f1781c,37) memb {
May 25 11:40:59 wssql-ix01 mysqld: #01125fc17e7-02c2-11e5-8c5a-7b5b15f1781c,0
May 25 11:40:59 wssql-ix01 mysqld: #0114ffae6ea-e288-11e4-bf6c-a7494d4cb905,0
May 25 11:40:59 wssql-ix01 mysqld: #0119df418b3-e288-11e4-841e-c34bc61e6237,0
May 25 11:40:59 wssql-ix01 mysqld: } joined {
May 25 11:40:59 wssql-ix01 mysqld: } left {
May 25 11:40:59 wssql-ix01 mysqld: } partitioned {
May 25 11:40:59 wssql-ix01 mysqld: })
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: gcomm: connected
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: Opened channel 'websolution'
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: Waiting for SST to complete.
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 3
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 26499fdb-02c2-11e5-89df-4b8781ebbf3f
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: STATE EXCHANGE: sent state msg: 26499fdb-02c2-11e5-89df-4b8781ebbf3f
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: STATE EXCHANGE: got state msg: 26499fdb-02c2-11e5-89df-4b8781ebbf3f from 0 (wssql-ix01)
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: STATE EXCHANGE: got state msg: 26499fdb-02c2-11e5-89df-4b8781ebbf3f from 1 (wssql-ix03)
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: STATE EXCHANGE: got state msg: 26499fdb-02c2-11e5-89df-4b8781ebbf3f from 2 (wssql-ix02)
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: Quorum results:
May 25 11:40:59 wssql-ix01 mysqld: #011version    = 3,
May 25 11:40:59 wssql-ix01 mysqld: #011component  = PRIMARY,
May 25 11:40:59 wssql-ix01 mysqld: #011conf_id    = 36,
May 25 11:40:59 wssql-ix01 mysqld: #011members    = 2/3 (joined/total),
May 25 11:40:59 wssql-ix01 mysqld: #011act_id     = 10545541,
May 25 11:40:59 wssql-ix01 mysqld: #011last_appl. = -1,
May 25 11:40:59 wssql-ix01 mysqld: #011protocols  = 0/5/3 (gcs/repl/appl),
May 25 11:40:59 wssql-ix01 mysqld: #011group UUID = 5d3b58bf-7bbe-11e4-9047-d6a833168dde
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: Flow-control interval: [28, 28]
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 10545541)
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: State transfer required:
May 25 11:40:59 wssql-ix01 mysqld: #011Group state: 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10545541
May 25 11:40:59 wssql-ix01 mysqld: #011Local state: 00000000-0000-0000-0000-000000000000:-1
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: New cluster view: global state: 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10545541, view# 37: Primary, number of nodes: 3, my index: 0, protocol version 3
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Warning] WSREP: Gap in state sequence. Need state transfer.
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: Running: 'wsrep_sst_xtrabackup --role 'joiner' --address '10.3.134.16' --auth 'replication-user:f02266aaea02' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --parent '16270' --binlog '/var/log/mysql/mariadb-bin' '
May 25 11:40:59 wssql-ix01 mysqld: WSREP_SST: [INFO] Streaming with tar (20150525 11:40:59.621)
May 25 11:40:59 wssql-ix01 mysqld: WSREP_SST: [INFO] Using socat as streamer (20150525 11:40:59.627)
May 25 11:40:59 wssql-ix01 mysqld: WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:4444,reuseaddr stdio | tar xfi - --recursive-unlink -h; RC=( ${PIPESTATUS[@]} ) (20150525 11:40:59.658)
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: Prepared SST request: xtrabackup|10.3.134.16:4444/xtrabackup_sst
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: REPL Protocols: 5 (3, 1)
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: Service thread queue flushed.
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: Assign initial position for certification: 10545541, protocol version: 3
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: Service thread queue flushed.
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (5d3b58bf-7bbe-11e4-9047-d6a833168dde): 1 (Operation not permitted)
May 25 11:40:59 wssql-ix01 mysqld: #011 at galera/src/replicator_str.cpp:prepare_for_IST():447. IST will be unavailable.
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: Member 0.0 (wssql-ix01) requested state transfer from '*any*'. Selected 1.0 (wssql-ix03)(SYNCED) as donor.
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 10545541)
May 25 11:40:59 wssql-ix01 mysqld: 150525 11:40:59 [Note] WSREP: Requesting state transfer: success, donor: 1
May 25 11:41:05 wssql-ix01 /etc/init.d/mysql[16635]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
May 25 11:41:05 wssql-ix01 /etc/init.d/mysql[16635]: #007/usr/bin/mysqladmin: connect to server at 'localhost' failed
May 25 11:41:05 wssql-ix01 /etc/init.d/mysql[16635]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111 "Connection refused")'
May 25 11:41:05 wssql-ix01 /etc/init.d/mysql[16635]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
May 25 11:41:05 wssql-ix01 /etc/init.d/mysql[16635]:
May 25 11:41:18 wssql-ix01 monit[2279]: 'mysql' process is not running
May 25 11:41:18 wssql-ix01 monit[2279]: 'mysql' trying to restart
May 25 11:41:18 wssql-ix01 monit[2279]: 'mysql' start: /etc/init.d/mysql
May 25 11:41:18 wssql-ix01 mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
May 25 11:41:18 wssql-ix01 mysqld_safe: WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.Yq72T6' --pid-file='/var/lib/mysql/wssql-ix01-recover.pid'
May 25 11:41:44 wssql-ix01 mysqld_safe: WSREP: Recovered position 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10542576
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: wsrep_start_position var submitted: '5d3b58bf-7bbe-11e4-9047-d6a833168dde:10542576'
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: Read nil XID from storage engines, skipping position init
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: wsrep_load(): Galera 25.3.5-wheezy(rXXXX) by Codership Oy <info@codership.com> loaded successfully.
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: CRC-32C: using hardware acceleration.
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: Passing config to GCS: base_host = 10.3.134.16; base_port = 4567; cert.log_conflicts = no; debug = no; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 1; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 256M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.bootstrap = true; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.version = 0; pc.wait_prim = false; pc.wait
May 25 11:41:44 wssql-ix01 mysqld: _prim_timeout = P30S; p
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: Service thread queue flushed.
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: wsrep_sst_grab()
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: Start replication
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: protonet asio version 0
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: Using CRC-32C (optimized) for message checksums.
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: backend: asio
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: GMCast version 0
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: (40ec2a72-02c2-11e5-8b8c-57bda2b28b60, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: (40ec2a72-02c2-11e5-8b8c-57bda2b28b60, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: EVS version 0
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: PC version 0
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: gcomm: connecting to group 'websolution', peer 'wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567'
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [ERROR] WSREP: Address already in use
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [ERROR] WSREP: failed to open gcomm backend connection: 98: error while trying to listen 'tcp://0.0.0.0:4567?socket.non_blocking=1', asio error 'Address already in use': 98 (Address already in use)
May 25 11:41:44 wssql-ix01 mysqld: #011 at gcomm/src/asio_tcp.cpp:listen():814
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [ERROR] WSREP: gcs/src/gcs_core.c:gcs_core_open():202: Failed to open backend connection: -98 (Address already in use)
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [ERROR] WSREP: gcs/src/gcs.c:gcs_open():1291: Failed to open channel 'websolution' at 'gcomm://wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567': -98 (Address already in use)
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [ERROR] WSREP: gcs connect failed: Address already in use
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [ERROR] WSREP: wsrep::connect() failed: 7
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [ERROR] Aborting
May 25 11:41:44 wssql-ix01 mysqld:
May 25 11:41:44 wssql-ix01 mysqld: 150525 11:41:44 [Note] WSREP: Service disconnected.
May 25 11:41:45 wssql-ix01 mysqld: 150525 11:41:45 [Note] WSREP: Some threads may fail to exit.
May 25 11:41:45 wssql-ix01 mysqld: 150525 11:41:45 [Note] /usr/sbin/mysqld: Shutdown complete
May 25 11:41:45 wssql-ix01 mysqld:
May 25 11:41:45 wssql-ix01 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
May 25 11:41:48 wssql-ix01 monit[2279]: 'mysql' failed to start
May 25 11:41:52 wssql-ix01 /etc/init.d/mysql[17453]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
May 25 11:41:52 wssql-ix01 /etc/init.d/mysql[17453]: #007/usr/bin/mysqladmin: connect to server at 'localhost' failed
May 25 11:41:52 wssql-ix01 /etc/init.d/mysql[17453]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111 "Connection refused")'
May 25 11:41:52 wssql-ix01 /etc/init.d/mysql[17453]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
May 25 11:41:52 wssql-ix01 /etc/init.d/mysql[17453]:
May 25 11:42:08 wssql-ix01 monit[2279]: 'mysql' process is not running
May 25 11:42:08 wssql-ix01 monit[2279]: 'mysql' trying to restart
May 25 11:42:08 wssql-ix01 monit[2279]: 'mysql' start: /etc/init.d/mysql
May 25 11:42:08 wssql-ix01 mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
May 25 11:42:08 wssql-ix01 mysqld_safe: WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.JxGO6d' --pid-file='/var/lib/mysql/wssql-ix01-recover.pid'
May 25 11:42:23 wssql-ix01 mysqld_safe: WSREP: Recovered position 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10545549
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: wsrep_start_position var submitted: '5d3b58bf-7bbe-11e4-9047-d6a833168dde:10545549'
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: Read nil XID from storage engines, skipping position init
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: wsrep_load(): Galera 25.3.5-wheezy(rXXXX) by Codership Oy <info@codership.com> loaded successfully.
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: CRC-32C: using hardware acceleration.
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: Passing config to GCS: base_host = 10.3.134.16; base_port = 4567; cert.log_conflicts = no; debug = no; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 1; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 256M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.bootstrap = true; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.version = 0; pc.wait_prim = false; pc.wait
May 25 11:42:23 wssql-ix01 mysqld: _prim_timeout = P30S; p
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: Service thread queue flushed.
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: wsrep_sst_grab()
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: Start replication
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: protonet asio version 0
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: Using CRC-32C (optimized) for message checksums.
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: backend: asio
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: GMCast version 0
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: (5843a171-02c2-11e5-aa0d-9641e172a021, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: (5843a171-02c2-11e5-aa0d-9641e172a021, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: EVS version 0
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: PC version 0
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: gcomm: connecting to group 'websolution', peer 'wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567'
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [ERROR] WSREP: Address already in use
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [ERROR] WSREP: failed to open gcomm backend connection: 98: error while trying to listen 'tcp://0.0.0.0:4567?socket.non_blocking=1', asio error 'Address already in use': 98 (Address already in use)
May 25 11:42:23 wssql-ix01 mysqld: #011 at gcomm/src/asio_tcp.cpp:listen():814
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [ERROR] WSREP: gcs/src/gcs_core.c:gcs_core_open():202: Failed to open backend connection: -98 (Address already in use)
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [ERROR] WSREP: gcs/src/gcs.c:gcs_open():1291: Failed to open channel 'websolution' at 'gcomm://wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567': -98 (Address already in use)
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [ERROR] WSREP: gcs connect failed: Address already in use
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [ERROR] WSREP: wsrep::connect() failed: 7
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [ERROR] Aborting
May 25 11:42:23 wssql-ix01 mysqld:
May 25 11:42:23 wssql-ix01 mysqld: 150525 11:42:23 [Note] WSREP: Service disconnected.
May 25 11:42:24 wssql-ix01 mysqld: 150525 11:42:24 [Note] WSREP: Some threads may fail to exit.
May 25 11:42:24 wssql-ix01 mysqld: 150525 11:42:24 [Note] /usr/sbin/mysqld: Shutdown complete
May 25 11:42:24 wssql-ix01 mysqld:
May 25 11:42:24 wssql-ix01 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
May 25 11:42:31 wssql-ix01 monit[2279]: 'mysql' started
May 25 11:42:45 wssql-ix01 /etc/init.d/mysql[18518]: 1 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
May 25 11:42:45 wssql-ix01 /etc/init.d/mysql[18518]: #007/usr/bin/mysqladmin: connect to server at 'localhost' failed
May 25 11:42:45 wssql-ix01 /etc/init.d/mysql[18518]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111 "Connection refused")'
May 25 11:42:45 wssql-ix01 /etc/init.d/mysql[18518]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
May 25 11:42:45 wssql-ix01 /etc/init.d/mysql[18518]:
May 25 11:42:51 wssql-ix01 monit[2279]: 'mysql' process is running with pid 16270
May 25 11:43:31 wssql-ix01 monit[2279]: 'mysql' stop on user request
May 25 11:43:31 wssql-ix01 monit[2279]: 'mysql' stop: /etc/init.d/mysql
May 25 11:44:01 wssql-ix01 monit[2279]: 'mysql' failed to stop
May 25 11:44:01 wssql-ix01 monit[2279]: 'mysql' stop action done
May 25 11:46:34 wssql-ix01 mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
May 25 11:46:34 wssql-ix01 mysqld_safe: WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.U1lX1I' --pid-file='/var/lib/mysql/wssql-ix01-recover.pid'
May 25 11:46:50 wssql-ix01 mysqld_safe: WSREP: Recovered position 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10545549
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: wsrep_start_position var submitted: '5d3b58bf-7bbe-11e4-9047-d6a833168dde:10545549'
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: Read nil XID from storage engines, skipping position init
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: wsrep_load(): Galera 25.3.5-wheezy(rXXXX) by Codership Oy <info@codership.com> loaded successfully.
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: CRC-32C: using hardware acceleration.
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: Passing config to GCS: base_host = 10.3.134.16; base_port = 4567; cert.log_conflicts = no; debug = no; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 1; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 256M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.bootstrap = true; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.version = 0; pc.wait_prim = false; pc.wait
May 25 11:46:50 wssql-ix01 mysqld: _prim_timeout = P30S; p
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: Service thread queue flushed.
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: wsrep_sst_grab()
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: Start replication
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: protonet asio version 0
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: Using CRC-32C (optimized) for message checksums.
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: backend: asio
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: GMCast version 0
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: (f7a42bca-02c2-11e5-ab7a-cec563816739, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: (f7a42bca-02c2-11e5-ab7a-cec563816739, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: EVS version 0
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: PC version 0
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: gcomm: connecting to group 'websolution', peer 'wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567'
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Warning] WSREP: (f7a42bca-02c2-11e5-ab7a-cec563816739, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' points to own listening address, blacklisting
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: (f7a42bca-02c2-11e5-ab7a-cec563816739, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid f7a42bca-02c2-11e5-ab7a-cec563816739 is blacklisted, skipping
May 25 11:46:50  mysqld: last message repeated 3 times
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: declaring 4ffae6ea-e288-11e4-bf6c-a7494d4cb905 stable
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: declaring 9df418b3-e288-11e4-841e-c34bc61e6237 stable
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: Node 4ffae6ea-e288-11e4-bf6c-a7494d4cb905 state prim
May 25 11:46:50 wssql-ix01 mysqld: 150525 11:46:50 [Note] WSREP: view(view_id(PRIM,4ffae6ea-e288-11e4-bf6c-a7494d4cb905,39) memb {
May 25 11:46:50 wssql-ix01 mysqld: #0114ffae6ea-e288-11e4-bf6c-a7494d4cb905,0
May 25 11:46:50 wssql-ix01 mysqld: #0119df418b3-e288-11e4-841e-c34bc61e6237,0
May 25 11:46:50 wssql-ix01 mysqld: #011f7a42bca-02c2-11e5-ab7a-cec563816739,0
May 25 11:46:50 wssql-ix01 mysqld: } joined {
May 25 11:46:50 wssql-ix01 mysqld: } left {
May 25 11:46:50 wssql-ix01 mysqld: } partitioned {
May 25 11:46:50 wssql-ix01 mysqld: })
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: gcomm: connected
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: Opened channel 'websolution'
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: Waiting for SST to complete.
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 3
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: STATE EXCHANGE: sent state msg: f7acb324-02c2-11e5-9c4e-6ade19b4a603
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: STATE EXCHANGE: got state msg: f7acb324-02c2-11e5-9c4e-6ade19b4a603 from 0 (wssql-ix03)
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: STATE EXCHANGE: got state msg: f7acb324-02c2-11e5-9c4e-6ade19b4a603 from 1 (wssql-ix02)
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: STATE EXCHANGE: got state msg: f7acb324-02c2-11e5-9c4e-6ade19b4a603 from 2 (wssql-ix01)
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: Quorum results:
May 25 11:46:51 wssql-ix01 mysqld: #011version    = 3,
May 25 11:46:51 wssql-ix01 mysqld: #011component  = PRIMARY,
May 25 11:46:51 wssql-ix01 mysqld: #011conf_id    = 38,
May 25 11:46:51 wssql-ix01 mysqld: #011members    = 2/3 (joined/total),
May 25 11:46:51 wssql-ix01 mysqld: #011act_id     = 10545926,
May 25 11:46:51 wssql-ix01 mysqld: #011last_appl. = -1,
May 25 11:46:51 wssql-ix01 mysqld: #011protocols  = 0/5/3 (gcs/repl/appl),
May 25 11:46:51 wssql-ix01 mysqld: #011group UUID = 5d3b58bf-7bbe-11e4-9047-d6a833168dde
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: Flow-control interval: [28, 28]
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 10545926)
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: State transfer required:
May 25 11:46:51 wssql-ix01 mysqld: #011Group state: 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10545926
May 25 11:46:51 wssql-ix01 mysqld: #011Local state: 00000000-0000-0000-0000-000000000000:-1
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: New cluster view: global state: 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10545926, view# 39: Primary, number of nodes: 3, my index: 2, protocol version 3
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Warning] WSREP: Gap in state sequence. Need state transfer.
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: Running: 'wsrep_sst_xtrabackup --role 'joiner' --address '10.3.134.16' --auth 'replication-user:f02266aaea02' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --parent '20680' --binlog '/var/log/mysql/mariadb-bin' '
May 25 11:46:51 wssql-ix01 mysqld: WSREP_SST: [INFO] Streaming with tar (20150525 11:46:51.345)
May 25 11:46:51 wssql-ix01 mysqld: WSREP_SST: [INFO] Using socat as streamer (20150525 11:46:51.349)
May 25 11:46:51 wssql-ix01 mysqld: WSREP_SST: [INFO] Stale sst_in_progress file: /var/lib/mysql//sst_in_progress (20150525 11:46:51.364)
May 25 11:46:51 wssql-ix01 mysqld: WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:4444,reuseaddr stdio | tar xfi - --recursive-unlink -h; RC=( ${PIPESTATUS[@]} ) (20150525 11:46:51.389)
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: Prepared SST request: xtrabackup|10.3.134.16:4444/xtrabackup_sst
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: REPL Protocols: 5 (3, 1)
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: Service thread queue flushed.
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: Assign initial position for certification: 10545926, protocol version: 3
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: Service thread queue flushed.
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (5d3b58bf-7bbe-11e4-9047-d6a833168dde): 1 (Operation not permitted)
May 25 11:46:51 wssql-ix01 mysqld: #011 at galera/src/replicator_str.cpp:prepare_for_IST():447. IST will be unavailable.
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: Member 2.0 (wssql-ix01) requested state transfer from '*any*'. Selected 0.0 (wssql-ix03)(SYNCED) as donor.
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 10545926)
May 25 11:46:51 wssql-ix01 mysqld: 150525 11:46:51 [Note] WSREP: Requesting state transfer: success, donor: 0
May 25 11:47:08 wssql-ix01 /etc/init.d/mysql[21392]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
May 25 11:47:08 wssql-ix01 /etc/init.d/mysql[21392]: #007/usr/bin/mysqladmin: connect to server at 'localhost' failed
May 25 11:47:08 wssql-ix01 /etc/init.d/mysql[21392]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111 "Connection refused")'
May 25 11:47:08 wssql-ix01 /etc/init.d/mysql[21392]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
May 25 11:47:08 wssql-ix01 /etc/init.d/mysql[21392]:
May 25 11:55:22 wssql-ix01 mysqld: 150525 11:55:22 [Note] WSREP: (f7a42bca-02c2-11e5-ab7a-cec563816739, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid f7a42bca-02c2-11e5-ab7a-cec563816739 is blacklisted, skipping
May 25 11:55:23  mysqld: last message repeated 3 times
May 25 11:55:23 wssql-ix01 mysqld: 150525 11:55:23 [Note] WSREP: (f7a42bca-02c2-11e5-ab7a-cec563816739, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid f7a42bca-02c2-11e5-ab7a-cec563816739 is blacklisted, skipping
May 25 11:55:55  mysqld: last message repeated 3 times
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Note] WSREP: (f7a42bca-02c2-11e5-ab7a-cec563816739, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid f7a42bca-02c2-11e5-ab7a-cec563816739 is blacklisted, skipping
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Note] WSREP: (f7a42bca-02c2-11e5-ab7a-cec563816739, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid f7a42bca-02c2-11e5-ab7a-cec563816739 is blacklisted, skipping
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Note] WSREP: (f7a42bca-02c2-11e5-ab7a-cec563816739, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://10.3.134.18:4567
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Note] WSREP: declaring 9df418b3-e288-11e4-841e-c34bc61e6237 stable
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Note] WSREP: Node 9df418b3-e288-11e4-841e-c34bc61e6237 state prim
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Note] WSREP: view(view_id(PRIM,9df418b3-e288-11e4-841e-c34bc61e6237,40) memb {
May 25 11:55:55 wssql-ix01 mysqld: #0119df418b3-e288-11e4-841e-c34bc61e6237,0
May 25 11:55:55 wssql-ix01 mysqld: #011f7a42bca-02c2-11e5-ab7a-cec563816739,0
May 25 11:55:55 wssql-ix01 mysqld: } joined {
May 25 11:55:55 wssql-ix01 mysqld: } left {
May 25 11:55:55 wssql-ix01 mysqld: } partitioned {
May 25 11:55:55 wssql-ix01 mysqld: #0114ffae6ea-e288-11e4-bf6c-a7494d4cb905,0
May 25 11:55:55 wssql-ix01 mysqld: })
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Note] WSREP: forgetting 4ffae6ea-e288-11e4-bf6c-a7494d4cb905 (tcp://10.3.134.18:4567)
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Note] WSREP: (f7a42bca-02c2-11e5-ab7a-cec563816739, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid f7a42bca-02c2-11e5-ab7a-cec563816739 is blacklisted, skipping
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Note] WSREP: (f7a42bca-02c2-11e5-ab7a-cec563816739, 'tcp://0.0.0.0:4567') turning message relay requesting off
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Note] WSREP: STATE EXCHANGE: sent state msg: 3cac9a01-02c4-11e5-9e06-5feb5a9754ca
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Note] WSREP: STATE EXCHANGE: got state msg: 3cac9a01-02c4-11e5-9e06-5feb5a9754ca from 0 (wssql-ix02)
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Note] WSREP: STATE EXCHANGE: got state msg: 3cac9a01-02c4-11e5-9e06-5feb5a9754ca from 1 (wssql-ix01)
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Note] WSREP: Quorum results:
May 25 11:55:55 wssql-ix01 mysqld: #011version    = 3,
May 25 11:55:55 wssql-ix01 mysqld: #011component  = PRIMARY,
May 25 11:55:55 wssql-ix01 mysqld: #011conf_id    = 39,
May 25 11:55:55 wssql-ix01 mysqld: #011members    = 1/2 (joined/total),
May 25 11:55:55 wssql-ix01 mysqld: #011act_id     = 10550291,
May 25 11:55:55 wssql-ix01 mysqld: #011last_appl. = 10550223,
May 25 11:55:55 wssql-ix01 mysqld: #011protocols  = 0/5/3 (gcs/repl/appl),
May 25 11:55:55 wssql-ix01 mysqld: #011group UUID = 5d3b58bf-7bbe-11e4-9047-d6a833168dde
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Warning] WSREP: Donor 4ffae6ea-e288-11e4-bf6c-a7494d4cb905 is no longer in the group. State transfer cannot be completed, need to abort. Aborting...
May 25 11:55:55 wssql-ix01 mysqld: 150525 11:55:55 [Note] WSREP: /usr/sbin/mysqld: Terminated.
May 25 11:55:57 wssql-ix01 mysqld: WSREP_SST: [ERROR] Parent mysqld process (PID:20680) terminated unexpectedly. (20150525 11:55:57.455)
May 25 11:55:57 wssql-ix01 mysqld: WSREP_SST: [ERROR] Cleanup after exit with status:32 (20150525 11:55:57.462)
May 25 11:55:57 wssql-ix01 mysqld: WSREP_SST: [INFO] Removing the sst_in_progress file (20150525 11:55:57.466)
May 25 11:55:57 wssql-ix01 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
May 25 12:13:59 wssql-ix01 monit[2279]: 'mysql' unmonitor on user request
May 25 12:13:59 wssql-ix01 monit[2279]: 'mysql' unmonitor action done
May 25 12:14:16 wssql-ix01 mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
May 25 12:14:16 wssql-ix01 mysqld_safe: WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.qZoBse' --pid-file='/var/lib/mysql/wssql-ix01-recover.pid'
May 25 12:14:49 wssql-ix01 mysqld_safe: WSREP: Recovered position 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10545926
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: wsrep_start_position var submitted: '5d3b58bf-7bbe-11e4-9047-d6a833168dde:10545926'
May 25 12:14:49 wssql-ix01 /etc/init.d/mysql[1178]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
May 25 12:14:49 wssql-ix01 /etc/init.d/mysql[1178]: #007/usr/bin/mysqladmin: connect to server at 'localhost' failed
May 25 12:14:49 wssql-ix01 /etc/init.d/mysql[1178]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111 "Connection refused")'
May 25 12:14:49 wssql-ix01 /etc/init.d/mysql[1178]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
May 25 12:14:49 wssql-ix01 /etc/init.d/mysql[1178]:
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: Read nil XID from storage engines, skipping position init
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: wsrep_load(): Galera 25.3.5-wheezy(rXXXX) by Codership Oy <info@codership.com> loaded successfully.
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: CRC-32C: using hardware acceleration.
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: Passing config to GCS: base_host = 10.3.134.16; base_port = 4567; cert.log_conflicts = no; debug = no; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 1; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 256M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.bootstrap = true; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.version = 0; pc.wait_prim = false; pc.wait
May 25 12:14:49 wssql-ix01 mysqld: _prim_timeout = P30S; p
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: Service thread queue flushed.
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: wsrep_sst_grab()
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: Start replication
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: protonet asio version 0
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: Using CRC-32C (optimized) for message checksums.
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: backend: asio
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: GMCast version 0
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: EVS version 0
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: PC version 0
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: gcomm: connecting to group 'websolution', peer 'wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567'
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Warning] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' points to own listening address, blacklisting
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: declaring 9df418b3-e288-11e4-841e-c34bc61e6237 stable
May 25 12:14:49 wssql-ix01 mysqld: 150525 12:14:49 [Note] WSREP: view(view_id(NON_PRIM,9df418b3-e288-11e4-841e-c34bc61e6237,82) memb {
May 25 12:14:49 wssql-ix01 mysqld: #0119df418b3-e288-11e4-841e-c34bc61e6237,0
May 25 12:14:49 wssql-ix01 mysqld: #011e03bddea-02c6-11e5-a847-963ac599e030,0
May 25 12:14:49 wssql-ix01 mysqld: } joined {
May 25 12:14:49 wssql-ix01 mysqld: } left {
May 25 12:14:49 wssql-ix01 mysqld: } partitioned {
May 25 12:14:49 wssql-ix01 mysqld: #01105e46402-02c6-11e5-809f-062a9119e68b,0
May 25 12:14:49 wssql-ix01 mysqld: #011179158d8-02c5-11e5-9205-f7b74bbf74a4,0
May 25 12:14:49 wssql-ix01 mysqld: #01123b3a88b-02c6-11e5-846b-f227e28cf600,0
May 25 12:14:49 wssql-ix01 mysqld: #01135513378-02c5-11e5-a8f4-5be1d163b26c,0
May 25 12:14:49 wssql-ix01 mysqld: #011418343b8-02c6-11e5-9004-b7ce33220d60,0
May 25 12:14:49 wssql-ix01 mysqld: #0114aced258-02c4-11e5-a01d-53fc78e3f366,0
May 25 12:14:49 wssql-ix01 mysqld: #0115319aa1a-02c5-11e5-91a3-9e2f20af2255,0
May 25 12:14:49 wssql-ix01 mysqld: #0115f5f86d3-02c6-11e5-b13c-e7a2c3b250f0,0
May 25 12:14:49 wssql-ix01 mysqld: #01164c267f3-02c4-11e5-be77-fed94faadcb2,0
May 25 12:14:49 wssql-ix01 mysqld: #01170ef274b-02c5-11e5-9ff2-8b6355487f18,0
May 25 12:14:49 wssql-ix01 mysqld: #0117d273ae0-02c6-11e5-b916-a6ec4c114952,0
May 25 12:14:49 wssql-ix01 mysqld: #011827ab6a5-02c4-11e5-a113-2b623efadeab,0
May 25 12:14:49 wssql-ix01 mysqld: #0118ec47e92-02c5-11e5-8bf1-526fa5022323,0
May 25 12:14:49 wssql-ix01 mysqld: #011a0593a66-02c4-11e5-b9d9-e76e590131d8,0
May 25 12:14:49 wssql-ix01 mysqld: #011ac946227-02c5-11e5-999a-1744c80f3ba8,0
May 25 12:14:49 wssql-ix01 mysqld: #011be3490cf-02c4-11e5-ba10-d22d82e86180,0
May 25 12:14:49 wssql-ix01 mysqld: #011ca55a25e-02c5-11e5-b0e3-c25b536e5d25,0
May 25 12:14:49 wssql-ix01 mysqld: #011dbeeb19e-02c4-11e5-91b5-9e206fb94165,0
May 25 12:14:49 wssql-ix01 mysqld: #011e81bb57b-02c5-11e5-a4cc-f3db0bec71bb,0
May 25 12:14:49 wssql-ix01 mysqld: #011f7a42bca-02c2-11e5-ab7a-cec563816739,0
May 25 12:14:49 wssql-ix01 mysqld: #011f9c6554c-02c4-11e5-84ba-5312d
May 25 12:14:50 wssql-ix01 mysqld: 150525 12:14:50 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:14:52 wssql-ix01 mysqld: 150525 12:14:52 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:14:53 wssql-ix01 mysqld: 150525 12:14:53 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:14:55 wssql-ix01 mysqld: 150525 12:14:55 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:14:56 wssql-ix01 mysqld: 150525 12:14:56 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:14:58 wssql-ix01 mysqld: 150525 12:14:58 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:14:59 wssql-ix01 mysqld: 150525 12:14:59 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:15:01 wssql-ix01 mysqld: 150525 12:15:01 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:15:02 wssql-ix01 mysqld: 150525 12:15:02 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:15:03 wssql-ix01 mysqld: 150525 12:15:03 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:15:05 wssql-ix01 mysqld: 150525 12:15:05 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:15:06 wssql-ix01 mysqld: 150525 12:15:06 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:15:08 wssql-ix01 mysqld: 150525 12:15:08 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:15:09 wssql-ix01 mysqld: 150525 12:15:09 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:15:11 wssql-ix01 mysqld: 150525 12:15:11 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:15:12 wssql-ix01 mysqld: 150525 12:15:12 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:15:14 wssql-ix01 mysqld: 150525 12:15:14 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:15:15 wssql-ix01 mysqld: 150525 12:15:15 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:15:17 wssql-ix01 mysqld: 150525 12:15:17 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:15:18 wssql-ix01 mysqld: 150525 12:15:18 [Note] WSREP: (e03bddea-02c6-11e5-a847-963ac599e030, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid e03bddea-02c6-11e5-a847-963ac599e030 is blacklisted, skipping
May 25 12:15:19 wssql-ix01 mysqld: 150525 12:15:19 [ERROR] WSREP: failed to open gcomm backend connection: 110: failed to reach primary view: 110 (Connection timed out)
May 25 12:15:19 wssql-ix01 mysqld: #011 at gcomm/src/pc.cpp:connect():141
May 25 12:15:19 wssql-ix01 mysqld: 150525 12:15:19 [ERROR] WSREP: gcs/src/gcs_core.c:gcs_core_open():202: Failed to open backend connection: -110 (Connection timed out)
May 25 12:15:19 wssql-ix01 mysqld: 150525 12:15:19 [ERROR] WSREP: gcs/src/gcs.c:gcs_open():1291: Failed to open channel 'websolution' at 'gcomm://wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567': -110 (Connection timed out)
May 25 12:15:19 wssql-ix01 mysqld: 150525 12:15:19 [ERROR] WSREP: gcs connect failed: Connection timed out
May 25 12:15:19 wssql-ix01 mysqld: 150525 12:15:19 [ERROR] WSREP: wsrep::connect() failed: 7
May 25 12:15:19 wssql-ix01 mysqld: 150525 12:15:19 [ERROR] Aborting
May 25 12:15:19 wssql-ix01 mysqld:
May 25 12:15:19 wssql-ix01 mysqld: 150525 12:15:19 [Note] WSREP: Service disconnected.
May 25 12:15:20 wssql-ix01 mysqld: 150525 12:15:20 [Note] WSREP: Some threads may fail to exit.
May 25 12:15:20 wssql-ix01 mysqld: 150525 12:15:20 [Note] /usr/sbin/mysqld: Shutdown complete
May 25 12:15:20 wssql-ix01 mysqld:
May 25 12:15:20 wssql-ix01 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
May 25 12:35:39 wssql-ix01 mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
May 25 12:35:39 wssql-ix01 mysqld_safe: WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.kUyong' --pid-file='/var/lib/mysql/wssql-ix01-recover.pid'
May 25 12:36:05 wssql-ix01 mysqld_safe: WSREP: Recovered position 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10545926
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: wsrep_start_position var submitted: '5d3b58bf-7bbe-11e4-9047-d6a833168dde:10545926'
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: Read nil XID from storage engines, skipping position init
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: wsrep_load(): Galera 25.3.5-wheezy(rXXXX) by Codership Oy <info@codership.com> loaded successfully.
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: CRC-32C: using hardware acceleration.
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: Passing config to GCS: base_host = 10.3.134.16; base_port = 4567; cert.log_conflicts = no; debug = no; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 1; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 256M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.bootstrap = true; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.version = 0; pc.wait_prim = false; pc.wait
May 25 12:36:05 wssql-ix01 mysqld: _prim_timeout = P30S; p
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: Service thread queue flushed.
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: wsrep_sst_grab()
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: Start replication
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: protonet asio version 0
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: Using CRC-32C (optimized) for message checksums.
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: backend: asio
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: GMCast version 0
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: (d8dd9ac4-02c9-11e5-a395-2e2e7d8ad6c0, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: (d8dd9ac4-02c9-11e5-a395-2e2e7d8ad6c0, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: EVS version 0
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: PC version 0
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: gcomm: connecting to group 'websolution', peer 'wssql-ix02.se-ix.delta.prod:4567,wssql-ix03.se-ix.delta.prod:4567,wssql-ix01.se-ix.delta.prod:4567'
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Warning] WSREP: (d8dd9ac4-02c9-11e5-a395-2e2e7d8ad6c0, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' points to own listening address, blacklisting
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: (d8dd9ac4-02c9-11e5-a395-2e2e7d8ad6c0, 'tcp://0.0.0.0:4567') address 'tcp://10.3.134.16:4567' pointing to uuid d8dd9ac4-02c9-11e5-a395-2e2e7d8ad6c0 is blacklisted, skipping
May 25 12:36:05  mysqld: last message repeated 3 times
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: declaring 674568e4-02c9-11e5-aaba-b78d1d81c311 stable
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: declaring 7eb94be8-02c9-11e5-a47a-76cb6536623d stable
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: Node 674568e4-02c9-11e5-aaba-b78d1d81c311 state prim
May 25 12:36:05 wssql-ix01 mysqld: 150525 12:36:05 [Note] WSREP: view(view_id(PRIM,674568e4-02c9-11e5-aaba-b78d1d81c311,3) memb {
May 25 12:36:05 wssql-ix01 mysqld: #011674568e4-02c9-11e5-aaba-b78d1d81c311,0
May 25 12:36:05 wssql-ix01 mysqld: #0117eb94be8-02c9-11e5-a47a-76cb6536623d,0
May 25 12:36:05 wssql-ix01 mysqld: #011d8dd9ac4-02c9-11e5-a395-2e2e7d8ad6c0,0
May 25 12:36:05 wssql-ix01 mysqld: } joined {
May 25 12:36:05 wssql-ix01 mysqld: } left {
May 25 12:36:05 wssql-ix01 mysqld: } partitioned {
May 25 12:36:05 wssql-ix01 mysqld: })
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: gcomm: connected
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: Opened channel 'websolution'
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: Waiting for SST to complete.
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 3
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: STATE EXCHANGE: sent state msg: d90f2e59-02c9-11e5-b8d0-1b53ff8fb83b
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: STATE EXCHANGE: got state msg: d90f2e59-02c9-11e5-b8d0-1b53ff8fb83b from 0 (wssql-ix02)
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: STATE EXCHANGE: got state msg: d90f2e59-02c9-11e5-b8d0-1b53ff8fb83b from 1 (wssql-ix03)
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: STATE EXCHANGE: got state msg: d90f2e59-02c9-11e5-b8d0-1b53ff8fb83b from 2 (wssql-ix01)
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: Quorum results:
May 25 12:36:06 wssql-ix01 mysqld: #011version    = 3,
May 25 12:36:06 wssql-ix01 mysqld: #011component  = PRIMARY,
May 25 12:36:06 wssql-ix01 mysqld: #011conf_id    = 2,
May 25 12:36:06 wssql-ix01 mysqld: #011members    = 1/3 (joined/total),
May 25 12:36:06 wssql-ix01 mysqld: #011act_id     = 10550998,
May 25 12:36:06 wssql-ix01 mysqld: #011last_appl. = -1,
May 25 12:36:06 wssql-ix01 mysqld: #011protocols  = 0/5/3 (gcs/repl/appl),
May 25 12:36:06 wssql-ix01 mysqld: #011group UUID = 5d3b58bf-7bbe-11e4-9047-d6a833168dde
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: Flow-control interval: [28, 28]
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 10550998)
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: State transfer required:
May 25 12:36:06 wssql-ix01 mysqld: #011Group state: 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10550998
May 25 12:36:06 wssql-ix01 mysqld: #011Local state: 00000000-0000-0000-0000-000000000000:-1
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: New cluster view: global state: 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10550998, view# 3: Primary, number of nodes: 3, my index: 2, protocol version 3
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Warning] WSREP: Gap in state sequence. Need state transfer.
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: Running: 'wsrep_sst_xtrabackup --role 'joiner' --address '10.3.134.16' --auth 'replication-user:f02266aaea02' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --parent '10377' --binlog '/var/log/mysql/mariadb-bin' '
May 25 12:36:06 wssql-ix01 mysqld: WSREP_SST: [INFO] Streaming with tar (20150525 12:36:06.718)
May 25 12:36:06 wssql-ix01 mysqld: WSREP_SST: [INFO] Using socat as streamer (20150525 12:36:06.725)
May 25 12:36:06 wssql-ix01 mysqld: WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:4444,reuseaddr stdio | tar xfi - --recursive-unlink -h; RC=( ${PIPESTATUS[@]} ) (20150525 12:36:06.795)
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: Prepared SST request: xtrabackup|10.3.134.16:4444/xtrabackup_sst
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: REPL Protocols: 5 (3, 1)
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: Service thread queue flushed.
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: Assign initial position for certification: 10550998, protocol version: 3
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: Service thread queue flushed.
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (5d3b58bf-7bbe-11e4-9047-d6a833168dde): 1 (Operation not permitted)
May 25 12:36:06 wssql-ix01 mysqld: #011 at galera/src/replicator_str.cpp:prepare_for_IST():447. IST will be unavailable.
May 25 12:36:06 wssql-ix01 mysqld: 150525 12:36:06 [Note] WSREP: Requesting state transfer failed: -11(Resource temporarily unavailable). Will keep retrying every 1 second(s)
May 25 12:36:16 wssql-ix01 /etc/init.d/mysql[10680]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
May 25 12:36:16 wssql-ix01 /etc/init.d/mysql[10680]: #007/usr/bin/mysqladmin: connect to server at 'localhost' failed
May 25 12:36:16 wssql-ix01 /etc/init.d/mysql[10680]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111 "Connection refused")'
May 25 12:36:16 wssql-ix01 /etc/init.d/mysql[10680]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
May 25 12:36:16 wssql-ix01 /etc/init.d/mysql[10680]:
May 25 12:36:45 wssql-ix01 mysqld: 150525 12:36:45 [Warning] WSREP: Could not find peer:
May 25 12:36:45 wssql-ix01 mysqld: 150525 12:36:45 [Note] WSREP: 0.0 (wssql-ix02): State transfer to -1.-1 (left the group) complete.
May 25 12:36:45 wssql-ix01 mysqld: 150525 12:36:45 [Note] WSREP: Member 0.0 (wssql-ix02) synced with group.
May 25 12:36:45 wssql-ix01 mysqld: 150525 12:36:45 [Note] WSREP: Member 2.0 (wssql-ix01) requested state transfer from '*any*'. Selected 0.0 (wssql-ix02)(SYNCED) as donor.
May 25 12:36:45 wssql-ix01 mysqld: 150525 12:36:45 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 10551012)
May 25 12:36:45 wssql-ix01 mysqld: 150525 12:36:45 [Note] WSREP: Requesting state transfer: success after 39 tries, donor: 0
May 25 12:36:54 wssql-ix01 mysqld: 150525 12:36:54 [Warning] WSREP: Could not find peer:
May 25 12:36:54 wssql-ix01 mysqld: 150525 12:36:54 [Note] WSREP: 1.0 (wssql-ix03): State transfer from -1.-1 (left the group) complete.
May 25 12:36:54 wssql-ix01 mysqld: 150525 12:36:54 [Note] WSREP: Member 1.0 (wssql-ix03) synced with group.
May 25 12:41:06 wssql-ix01 mysqld: 150525 12:41:06 [Note] WSREP: 0.0 (wssql-ix02): State transfer to 2.0 (wssql-ix01) complete.
May 25 12:41:06 wssql-ix01 mysqld: 150525 12:41:06 [Note] WSREP: Member 0.0 (wssql-ix02) synced with group.
May 25 12:41:06 wssql-ix01 mysqld: WSREP_SST: [INFO] Proceeding with SST (20150525 12:41:06.982)
May 25 12:41:06 wssql-ix01 mysqld: WSREP_SST: [INFO] Removing existing ib_logfile files (20150525 12:41:06.987)
May 25 12:41:07 wssql-ix01 mysqld: WSREP_SST: [INFO] Preparing the backup at /var/lib/mysql/ (20150525 12:41:07.199)
May 25 12:41:07 wssql-ix01 mysqld: WSREP_SST: [INFO] Evaluating innobackupex --defaults-file=/etc/mysql/my.cnf --apply-log $rebuildcmd ${DATA} &>${DATA}/innobackup.prepare.log (20150525 12:41:07.204)
May 25 12:41:15 wssql-ix01 mysqld: WSREP_SST: [INFO] Total time on joiner: 0 seconds (20150525 12:41:15.367)
May 25 12:41:15 wssql-ix01 mysqld: WSREP_SST: [INFO] Removing the sst_in_progress file (20150525 12:41:15.372)
May 25 12:41:15 wssql-ix01 mysqld: 150525 12:41:15 [Note] WSREP: SST complete, seqno: 10551378
May 25 12:41:15 wssql-ix01 mysqld: 2015-05-25 12:41:15 7fcce1174780 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html.
May 25 12:41:15 wssql-ix01 mysqld: 150525 12:41:15 [Note] InnoDB: Using mutexes to ref count buffer pool pages
May 25 12:41:15 wssql-ix01 mysqld: 150525 12:41:15 [Note] InnoDB: The InnoDB memory heap is disabled
May 25 12:41:15 wssql-ix01 mysqld: 150525 12:41:15 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
May 25 12:41:15 wssql-ix01 mysqld: 150525 12:41:15 [Note] InnoDB: Memory barrier is not used
May 25 12:41:15 wssql-ix01 mysqld: 150525 12:41:15 [Note] InnoDB: Compressed tables use zlib 1.2.3.4
May 25 12:41:15 wssql-ix01 mysqld: 150525 12:41:15 [Note] InnoDB: Using Linux native AIO
May 25 12:41:15 wssql-ix01 mysqld: 150525 12:41:15 [Note] InnoDB: Using CPU crc32 instructions
May 25 12:41:15 wssql-ix01 mysqld: 150525 12:41:15 [Note] InnoDB: Initializing buffer pool, size = 7.0G
May 25 12:41:16 wssql-ix01 mysqld: 150525 12:41:16 [Note] InnoDB: Completed initialization of buffer pool
May 25 12:41:16 wssql-ix01 mysqld: 150525 12:41:16 [Note] InnoDB: Highest supported file format is Barracuda.
May 25 12:41:35 wssql-ix01 mysqld: 150525 12:41:35 [Note] InnoDB: 128 rollback segment(s) are active.
May 25 12:41:35 wssql-ix01 mysqld: 150525 12:41:35 [Note] InnoDB: Waiting for purge to start
May 25 12:41:35 wssql-ix01 mysqld: 150525 12:41:35 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.20-68.0 started; log sequence number 35238298781
May 25 12:41:35 wssql-ix01 mysqld: 150525 12:41:35 [Note] Plugin 'FEEDBACK' is disabled.
May 25 12:41:36 wssql-ix01 mysqld: 150525 12:41:36 [Note] Server socket created on IP: '0.0.0.0'.
May 25 12:41:37 wssql-ix01 mysqld: 150525 12:41:37 [Note] Event Scheduler: Loaded 0 events
May 25 12:41:37 wssql-ix01 mysqld: 150525 12:41:37 [Note] WSREP: Signalling provider to continue.
May 25 12:41:37 wssql-ix01 mysqld: 150525 12:41:37 [Note] WSREP: SST received: 5d3b58bf-7bbe-11e4-9047-d6a833168dde:10551378
May 25 12:41:37 wssql-ix01 mysqld: 150525 12:41:37 [Note] WSREP: 2.0 (wssql-ix01): State transfer from 0.0 (wssql-ix02) complete.
May 25 12:41:37 wssql-ix01 mysqld: 150525 12:41:37 [Note] WSREP: Shifting JOINER -> JOINED (TO: 10551506)
May 25 12:41:37 wssql-ix01 mysqld: 150525 12:41:37 [Note] Reading of all Master_info entries succeded
May 25 12:41:37 wssql-ix01 mysqld: 150525 12:41:37 [Note] Added new Master_info '' to hash table
May 25 12:41:37 wssql-ix01 mysqld: 150525 12:41:37 [Note] /usr/sbin/mysqld: ready for connections.
May 25 12:41:37 wssql-ix01 mysqld: Version: '10.0.14-MariaDB-1~precise-wsrep-log'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution, wsrep_25.10.r4144
May 25 12:41:39 wssql-ix01 mysqld: 150525 12:41:39 [Note] WSREP: Member 2.0 (wssql-ix01) synced with group.
May 25 12:41:39 wssql-ix01 mysqld: 150525 12:41:39 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 10551508)
May 25 12:41:41 wssql-ix01 mysqld: 150525 12:41:41 [Note] WSREP: Synchronized with group, ready for connections
May 25 12:41:41 wssql-ix01 mysqld: 150525 12:41:41 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.