Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-15365

Cannot upgrade MariaDB Galera Cluster to 10.1.31

    XMLWordPrintable

Details

    Description

      OS: Debian amd64

      When upgrading any previous version of MariaDB Galera Cluster to 10.1.31, MariaDB fails to restart. It sees an inexistant gap in the Galera sequence and tries doing a SST that always fails (there is no donor).

      If mariadb-server-10.1-core package is rolled back to 10.1.26, the server will start correctly and do an IST. Note that mysqld is still at version 10.1.31 at this point. Because of this I suspect that the issue has been introduced by a possible change in the SST scripts.

      Please see logs below for an explanation of the issue.

      2018-02-20 16:28:12 139788188453632 [Note] /usr/sbin/mysqld: Normal shutdown
       
      2018-02-20 16:28:12 139788188453632 [Note] WSREP: Stop replication
      2018-02-20 16:28:12 139788188453632 [Note] WSREP: Closing send monitor...
      2018-02-20 16:28:12 139788188453632 [Note] WSREP: Closed send monitor.
      2018-02-20 16:28:12 139788188453632 [Note] WSREP: gcomm: terminating thread
      2018-02-20 16:28:12 139788188453632 [Note] WSREP: gcomm: joining thread
      2018-02-20 16:28:12 139788188453632 [Note] WSREP: gcomm: closing backend
      2018-02-20 16:28:12 139788188453632 [Note] WSREP: view(view_id(NON_PRIM,3cb9412b,39) memb {
      	7315d7e7,0
      } joined {
      } left {
      } partitioned {
      	3cb9412b,0
      	cdf55f2f,0
      })
      2018-02-20 16:28:12 139788188453632 [Note] WSREP: view((empty))
      2018-02-20 16:28:12 139788188453632 [Note] WSREP: gcomm: closed
      2018-02-20 16:28:12 139844101601024 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
      2018-02-20 16:28:12 139844101601024 [Note] WSREP: Flow-control interval: [16, 16]
      2018-02-20 16:28:12 139844101601024 [Note] WSREP: Received NON-PRIMARY.
      2018-02-20 16:28:12 139844101601024 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 1263147016)
      2018-02-20 16:28:12 139844101601024 [Note] WSREP: Received self-leave message.
      2018-02-20 16:28:12 139844101601024 [Note] WSREP: Flow-control interval: [0, 0]
      2018-02-20 16:28:12 139844101601024 [Note] WSREP: Received SELF-LEAVE. Closing connection.
      2018-02-20 16:28:12 139844101601024 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 1263147016)
      2018-02-20 16:28:12 139844101601024 [Note] WSREP: RECV thread exiting 0: Success
      2018-02-20 16:28:12 139848620464896 [Note] WSREP: New cluster view: global state: e5e263a4-08ca-11e7-bb23-225463a7d968:1263147016, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version 3
      2018-02-20 16:28:12 139788188453632 [Note] WSREP: recv_thread() joined.
      2018-02-20 16:28:12 139788188453632 [Note] WSREP: Closing replication queue.
      2018-02-20 16:28:12 139788188453632 [Note] WSREP: Closing slave action queue.
      2018-02-20 16:28:12 139848620464896 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
      2018-02-20 16:28:12 139848620464896 [Note] WSREP: New cluster view: global state: e5e263a4-08ca-11e7-bb23-225463a7d968:1263147016, view# -1: non-Primary, number of nodes: 0, my index: -1, protocol version 3
      2018-02-20 16:28:12 139848620464896 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
      2018-02-20 16:28:12 139848620464896 [Note] WSREP: applier thread exiting (code:0)
      2018-02-20 16:28:12 139848690092800 [Note] WSREP: applier thread exiting (code:6)
      2018-02-20 16:28:12 139848636545792 [Note] WSREP: applier thread exiting (code:6)
      2018-02-20 16:28:12 139848636242688 [Note] WSREP: applier thread exiting (code:6)
      2018-02-20 16:28:12 139848688880384 [Note] WSREP: applier thread exiting (code:6)
      2018-02-20 16:28:12 139848620768000 [Note] WSREP: applier thread exiting (code:6)
      2018-02-20 16:28:12 139848689789696 [Note] WSREP: applier thread exiting (code:6)
      2018-02-20 16:28:12 139848689183488 [Note] WSREP: applier thread exiting (code:6)
      2018-02-20 16:28:12 139848619858688 [Note] WSREP: applier thread exiting (code:6)
      2018-02-20 16:28:12 139848707668736 [Note] WSREP: applier thread exiting (code:6)
      2018-02-20 16:28:12 139848688577280 [Note] WSREP: applier thread exiting (code:6)
      2018-02-20 16:28:12 139848620161792 [Note] WSREP: applier thread exiting (code:6)
      2018-02-20 16:28:12 139848706530048 [Note] WSREP: applier thread exiting (code:6)
      2018-02-20 16:28:12 139848689486592 [Note] WSREP: applier thread exiting (code:6)
      2018-02-20 16:28:12 139848619555584 [Note] WSREP: applier thread exiting (code:6)
      2018-02-20 16:28:12 139848636848896 [Note] WSREP: applier thread exiting (code:6)
      2018-02-20 16:28:14 139848707971840 [Note] WSREP: rollbacker thread exiting
      2018-02-20 16:28:14 139788188453632 [Note] Event Scheduler: Killing the scheduler thread, thread id 1123115
      2018-02-20 16:28:14 139788188453632 [Note] Event Scheduler: Waiting for the scheduler thread to reply
      2018-02-20 16:28:14 139788188453632 [Note] Event Scheduler: Stopped
      2018-02-20 16:28:14 139788188453632 [Note] Event Scheduler: Purging the queue. 0 events
      2018-02-20 16:28:14 139788188453632 [Note] WSREP: dtor state: CLOSED
      2018-02-20 16:28:14 139788188453632 [Note] WSREP: mon: entered 1262898728 oooe fraction 0 oool fraction 2.22504e-07
      2018-02-20 16:28:14 139788188453632 [Note] WSREP: mon: entered 1262898728 oooe fraction 0.0827089 oool fraction 9.7996e-05
      2018-02-20 16:28:14 139788188453632 [Note] WSREP: mon: entered 1290228897 oooe fraction 0 oool fraction 4.03029e-08
      2018-02-20 16:28:14 139788188453632 [Note] WSREP: cert index usage at exit 0
      2018-02-20 16:28:14 139788188453632 [Note] WSREP: cert trx map usage at exit 122
      2018-02-20 16:28:14 139788188453632 [Note] WSREP: deps set usage at exit 0
      2018-02-20 16:28:14 139788188453632 [Note] WSREP: avg deps dist 76.3776
      2018-02-20 16:28:14 139788188453632 [Note] WSREP: avg cert interval 0.306251
      2018-02-20 16:28:14 139788188453632 [Note] WSREP: cert index size 154
      2018-02-20 16:28:14 139844168021760 [Note] WSREP: Service thread queue flushed.
      2018-02-20 16:28:14 139788188453632 [Note] WSREP: wsdb trx map usage 0 conn query map usage 0
      2018-02-20 16:28:14 139788188453632 [Note] WSREP: MemPool(LocalTrxHandle): hit ratio: 0.999984, misses: 884, in use: 0, in pool: 884
      2018-02-20 16:28:14 139788188453632 [Note] WSREP: MemPool(SlaveTrxHandle): hit ratio: 0.999986, misses: 16527, in use: 0, in pool: 2048
      2018-02-20 16:28:14 139788188453632 [Note] WSREP: Shifting CLOSED -> DESTROYED (TO: 1263147016)
      2018-02-20 16:28:14 139788188453632 [Note] WSREP: Flushing memory map to disk...
      2018-02-20 16:28:14 139788459927296 [Note] InnoDB: FTS optimize thread exiting.
      2018-02-20 16:28:14 139788188453632 [Note] InnoDB: Starting shutdown...
      2018-02-20 16:28:14 139788476712704 [Warning] InnoDB: Dumping buffer pool(s) to .//ib_buffer_pool
      2018-02-20 16:28:15 139788476712704 [Warning] InnoDB: Buffer pool(s) dump completed at 180220 16:28:15
      2018-02-20 16:28:15 139788188453632 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool
      2018-02-20 16:28:22 139788188453632 [Note] InnoDB: Shutdown completed; log sequence number 3679987115201
      2018-02-20 16:28:22 139788188453632 [Note] /usr/sbin/mysqld: Shutdown complete
       
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: Read nil XID from storage engines, skipping position init
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: wsrep_load(): Galera 25.3.22(r3764) by Codership Oy <info@codership.com> loaded successfully.
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: CRC-32C: using hardware acceleration.
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: Found saved state: e5e263a4-08ca-11e7-bb23-225463a7d968:1263147016, safe_to_bootstrap: 0
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: Passing config to GCS: base_dir = /data/mysql/; base_host = 10.0.5.12; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /data/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /data/mysql//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 4G; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum =
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: GCache history reset: e5e263a4-08ca-11e7-bb23-225463a7d968:0 -> e5e263a4-08ca-11e7-bb23-225463a7d968:1263147016
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: Assign initial position for certification: 1263147016, protocol version: -1
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: wsrep_sst_grab()
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: Start replication
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: Setting initial position to e5e263a4-08ca-11e7-bb23-225463a7d968:1263147016
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: protonet asio version 0
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: Using CRC-32C for message checksums.
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: backend: asio
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: gcomm thread scheduling priority set to other:0 
      2018-02-20 16:29:29 140719173044160 [Warning] WSREP: access file(/data/mysql//gvwstate.dat) failed(No such file or directory)
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: restore pc from disk failed
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: GMCast version 0
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: (d7511b93, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: (d7511b93, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: EVS version 0
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: gcomm: connecting to group 'pup_pdb', peer '10.0.5.11:,10.0.5.12:,10.0.5.13:'
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: (d7511b93, 'tcp://0.0.0.0:4567') connection established to d7511b93 tcp://10.0.5.12:4567
      2018-02-20 16:29:29 140719173044160 [Warning] WSREP: (d7511b93, 'tcp://0.0.0.0:4567') address 'tcp://10.0.5.12:4567' points to own listening address, blacklisting
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: (d7511b93, 'tcp://0.0.0.0:4567') connection established to 3cb9412b tcp://10.0.5.13:4567
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: (d7511b93, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: (d7511b93, 'tcp://0.0.0.0:4567') connection established to cdf55f2f tcp://10.0.5.11:4567
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: declaring 3cb9412b at tcp://10.0.5.13:4567 stable
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: declaring cdf55f2f at tcp://10.0.5.11:4567 stable
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: Node 3cb9412b state prim
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: view(view_id(PRIM,3cb9412b,41) memb {
      	3cb9412b,0
      	cdf55f2f,0
      	d7511b93,0
      } joined {
      } left {
      } partitioned {
      })
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: save pc into disk
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: gcomm: connected
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: Opened channel 'pup_pdb'
      2018-02-20 16:29:29 140719173044160 [Note] WSREP: Waiting for SST to complete.
      2018-02-20 16:29:29 140714574866176 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 3
      2018-02-20 16:29:29 140714574866176 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
      2018-02-20 16:29:29 140714574866176 [Note] WSREP: STATE EXCHANGE: sent state msg: d7774739-1652-11e8-8826-8fbe6be4743c
      2018-02-20 16:29:29 140714574866176 [Note] WSREP: STATE EXCHANGE: got state msg: d7774739-1652-11e8-8826-8fbe6be4743c from 0 (pdb-3)
      2018-02-20 16:29:29 140714574866176 [Note] WSREP: STATE EXCHANGE: got state msg: d7774739-1652-11e8-8826-8fbe6be4743c from 1 (pdb-1)
      2018-02-20 16:29:29 140714574866176 [Note] WSREP: STATE EXCHANGE: got state msg: d7774739-1652-11e8-8826-8fbe6be4743c from 2 (pdb-2)
      2018-02-20 16:29:29 140714574866176 [Note] WSREP: Quorum results:
      	version    = 4,
      	component  = PRIMARY,
      	conf_id    = 40,
      	members    = 2/3 (joined/total),
      	act_id     = 1263147765,
      	last_appl. = -1,
      	protocols  = 0/7/3 (gcs/repl/appl),
      	group UUID = e5e263a4-08ca-11e7-bb23-225463a7d968
      2018-02-20 16:29:29 140714574866176 [Note] WSREP: Flow-control interval: [28, 28]
      2018-02-20 16:29:29 140714574866176 [Note] WSREP: Trying to continue unpaused monitor
      2018-02-20 16:29:29 140714574866176 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 1263147765)
      2018-02-20 16:29:29 140719172729600 [Note] WSREP: State transfer required: 
      	Group state: e5e263a4-08ca-11e7-bb23-225463a7d968:1263147765
      	Local state: e5e263a4-08ca-11e7-bb23-225463a7d968:1263147016
      2018-02-20 16:29:29 140719172729600 [Note] WSREP: New cluster view: global state: e5e263a4-08ca-11e7-bb23-225463a7d968:1263147765, view# 41: Primary, number of nodes: 3, my index: 2, protocol version 3
      2018-02-20 16:29:29 140719172729600 [Warning] WSREP: Gap in state sequence. Need state transfer.
      2018-02-20 16:29:29 140714545510144 [Note] WSREP: Running: 'wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.0.5.12' --datadir '/data/mysql/'   --parent '1579929' --binlog 'mariadb-bin' '
      WSREP_SST: [INFO] Streaming with xbstream (20180220 16:29:29.624)
      WSREP_SST: [INFO] Using socat as streamer (20180220 16:29:29.625)
      WSREP_SST: [INFO] Evaluating timeout -k 110 100 socat -u TCP-LISTEN:4444,reuseaddr stdio | pv -f  -i 10 -N joiner -F '%N => Rate:%r Avg:%a Elapsed:%t %e Bytes: %b %p'  | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20180220 16:29:29.638)
      2018-02-20 16:29:32 140714583258880 [Note] WSREP: (d7511b93, 'tcp://0.0.0.0:4567') connection to peer d7511b93 with addr tcp://10.0.5.12:4567 timed out, no messages seen in PT3S
      2018-02-20 16:29:32 140714583258880 [Note] WSREP: (d7511b93, 'tcp://0.0.0.0:4567') turning message relay requesting off
         joiner: => Rate:[   0 B/s] Avg:[   0 B/s] Elapsed:0:01:40  Bytes:    0 B 
      WSREP_SST: [INFO] NOTE: Joiner-Recv-gtid took 100 seconds (20180220 16:31:09.644)
      WSREP_SST: [ERROR] Possible timeout in receving first data from donor in gtid stage (20180220 16:31:09.647)
      WSREP_SST: [ERROR] Cleanup after exit with status:32 (20180220 16:31:09.650)
      2018-02-20 16:31:09 140714545510144 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.0.5.12' --datadir '/data/mysql/'   --parent '1579929' --binlog 'mariadb-bin' 
      	Read: '(null)'
      2018-02-20 16:31:09 140714545510144 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.0.5.12' --datadir '/data/mysql/'   --parent '1579929' --binlog 'mariadb-bin' : 32 (Broken pipe)
      2018-02-20 16:31:09 140719172729600 [ERROR] WSREP: Failed to prepare for 'xtrabackup-v2' SST. Unrecoverable.
      2018-02-20 16:31:09 140719172729600 [ERROR] Aborting
       
      Error in my_thread_global_end(): 1 threads didn't exit
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: Read nil XID from storage engines, skipping position init
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: wsrep_load(): Galera 25.3.20(r3703) by Codership Oy <info@codership.com> loaded successfully.
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: CRC-32C: using hardware acceleration.
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: Found saved state: e5e263a4-08ca-11e7-bb23-225463a7d968:-1, safe_to_bootsrap: 0
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: Passing config to GCS: base_dir = /data/mysql/; base_host = 10.0.5.12; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /data/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /data/mysql//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 4G; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum =
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: GCache history reset: old(e5e263a4-08ca-11e7-bb23-225463a7d968:0) -> new(e5e263a4-08ca-11e7-bb23-225463a7d968:1263147016)
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: Assign initial position for certification: 1263147016, protocol version: -1
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: wsrep_sst_grab()
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: Start replication
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: Setting initial position to e5e263a4-08ca-11e7-bb23-225463a7d968:1263147016
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: protonet asio version 0
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: Using CRC-32C for message checksums.
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: backend: asio
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: gcomm thread scheduling priority set to other:0 
      2018-02-20 16:32:55 139736135251904 [Warning] WSREP: access file(/data/mysql//gvwstate.dat) failed(No such file or directory)
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: restore pc from disk failed
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: GMCast version 0
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: (5228f5f8, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: (5228f5f8, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: EVS version 0
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: gcomm: connecting to group 'pup_pdb', peer '10.0.5.11:,10.0.5.12:,10.0.5.13:'
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: (5228f5f8, 'tcp://0.0.0.0:4567') connection established to 5228f5f8 tcp://10.0.5.12:4567
      2018-02-20 16:32:55 139736135251904 [Warning] WSREP: (5228f5f8, 'tcp://0.0.0.0:4567') address 'tcp://10.0.5.12:4567' points to own listening address, blacklisting
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: (5228f5f8, 'tcp://0.0.0.0:4567') connection established to cdf55f2f tcp://10.0.5.11:4567
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: (5228f5f8, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: (5228f5f8, 'tcp://0.0.0.0:4567') connection established to 3cb9412b tcp://10.0.5.13:4567
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: declaring 3cb9412b at tcp://10.0.5.13:4567 stable
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: declaring cdf55f2f at tcp://10.0.5.11:4567 stable
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: Node 3cb9412b state prim
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: view(view_id(PRIM,3cb9412b,43) memb {
      	3cb9412b,0
      	5228f5f8,0
      	cdf55f2f,0
      } joined {
      } left {
      } partitioned {
      })
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: save pc into disk
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: gcomm: connected
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: Opened channel 'pup_pdb'
      2018-02-20 16:32:55 139736135251904 [Note] WSREP: Waiting for SST to complete.
      2018-02-20 16:32:55 139731539060480 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 3
      2018-02-20 16:32:55 139731539060480 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
      2018-02-20 16:32:55 139731539060480 [Note] WSREP: STATE EXCHANGE: sent state msg: 522b3abf-1653-11e8-b7ac-df36163d6d5a
      2018-02-20 16:32:55 139731539060480 [Note] WSREP: STATE EXCHANGE: got state msg: 522b3abf-1653-11e8-b7ac-df36163d6d5a from 0 (pdb-3)
      2018-02-20 16:32:55 139731539060480 [Note] WSREP: STATE EXCHANGE: got state msg: 522b3abf-1653-11e8-b7ac-df36163d6d5a from 2 (pdb-1)
      2018-02-20 16:32:55 139731539060480 [Note] WSREP: STATE EXCHANGE: got state msg: 522b3abf-1653-11e8-b7ac-df36163d6d5a from 1 (pdb-2)
      2018-02-20 16:32:55 139731539060480 [Note] WSREP: Quorum results:
      	version    = 4,
      	component  = PRIMARY,
      	conf_id    = 42,
      	members    = 2/3 (joined/total),
      	act_id     = 1263159927,
      	last_appl. = -1,
      	protocols  = 0/7/3 (gcs/repl/appl),
      	group UUID = e5e263a4-08ca-11e7-bb23-225463a7d968
      2018-02-20 16:32:55 139731539060480 [Note] WSREP: Flow-control interval: [28, 28]
      2018-02-20 16:32:55 139731539060480 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 1263159927)
      2018-02-20 16:32:55 139736134937344 [Note] WSREP: State transfer required: 
      	Group state: e5e263a4-08ca-11e7-bb23-225463a7d968:1263159927
      	Local state: e5e263a4-08ca-11e7-bb23-225463a7d968:1263147016
      2018-02-20 16:32:55 139736134937344 [Note] WSREP: New cluster view: global state: e5e263a4-08ca-11e7-bb23-225463a7d968:1263159927, view# 43: Primary, number of nodes: 3, my index: 1, protocol version 3
      2018-02-20 16:32:55 139736134937344 [Warning] WSREP: Gap in state sequence. Need state transfer.
      2018-02-20 16:32:55 139731509704448 [Note] WSREP: Running: 'wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.0.5.12' --datadir '/data/mysql/'   --parent '1582954' --binlog 'mariadb-bin' '
      WSREP_SST: [INFO] Streaming with xbstream (20180220 16:32:55.744)
      WSREP_SST: [INFO] Using socat as streamer (20180220 16:32:55.745)
      WSREP_SST: [INFO] Stale sst_in_progress file: /data/mysql//sst_in_progress (20180220 16:32:55.748)
      WSREP_SST: [INFO] Evaluating timeout -k 110 100 socat -u TCP-LISTEN:4444,reuseaddr stdio | pv -f  -i 10 -N joiner -F '%N => Rate:%r Avg:%a Elapsed:%t %e Bytes: %b %p'  | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20180220 16:32:55.765)
      2018-02-20 16:32:58 139731547453184 [Note] WSREP: (5228f5f8, 'tcp://0.0.0.0:4567') connection to peer 5228f5f8 with addr tcp://10.0.5.12:4567 timed out, no messages seen in PT3S
      2018-02-20 16:32:58 139731547453184 [Note] WSREP: (5228f5f8, 'tcp://0.0.0.0:4567') turning message relay requesting off
         joiner: => Rate:[   0 B/s] Avg:[   0 B/s] Elapsed:0:01:40  Bytes:    0 B 
      WSREP_SST: [INFO] NOTE: Joiner-Recv-gtid took 100 seconds (20180220 16:34:35.769)
      WSREP_SST: [ERROR] Possible timeout in receving first data from donor in gtid stage (20180220 16:34:35.770)
      WSREP_SST: [ERROR] Cleanup after exit with status:32 (20180220 16:34:35.771)
      2018-02-20 16:34:35 139731509704448 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.0.5.12' --datadir '/data/mysql/'   --parent '1582954' --binlog 'mariadb-bin' 
      	Read: '(null)'
      2018-02-20 16:34:35 139731509704448 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.0.5.12' --datadir '/data/mysql/'   --parent '1582954' --binlog 'mariadb-bin' : 32 (Broken pipe)
      2018-02-20 16:34:35 139736134937344 [ERROR] WSREP: Failed to prepare for 'xtrabackup-v2' SST. Unrecoverable.
      2018-02-20 16:34:35 139736134937344 [ERROR] Aborting
       
      Error in my_thread_global_end(): 1 threads didn't exit
      2018-02-20 16:36:27 139808677193664 [Note] WSREP: Read nil XID from storage engines, skipping position init
      2018-02-20 16:36:27 139808677193664 [Note] WSREP: wsrep_load(): loading provider library 'none'
      2018-02-20 16:36:27 7f27be8a67c0 InnoDB: Warning: Setting innodb_buffer_pool_populate is DEPRECATED and has no effect. This option will be removed in MariaDB 10.2.3.
      2018-02-20 16:36:27 139808677193664 [Note] InnoDB: Using mutexes to ref count buffer pool pages
      2018-02-20 16:36:27 139808677193664 [Note] InnoDB: The InnoDB memory heap is disabled
      2018-02-20 16:36:27 139808677193664 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
      2018-02-20 16:36:27 139808677193664 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
      2018-02-20 16:36:27 139808677193664 [Note] InnoDB: Compressed tables use zlib 1.2.8
      2018-02-20 16:36:27 139808677193664 [Note] InnoDB: Using Linux native AIO
      2018-02-20 16:36:27 139808677193664 [Note] InnoDB: Using SSE crc32 instructions
      2018-02-20 16:36:27 139808677193664 [Note] InnoDB: Initializing buffer pool, size = 47.2G
      2018-02-20 16:36:28 139808677193664 [Note] InnoDB: Completed initialization of buffer pool
      2018-02-20 16:36:28 139808677193664 [Note] InnoDB: Highest supported file format is Barracuda.
      2018-02-20 16:36:29 139808677193664 [Note] InnoDB: 128 rollback segment(s) are active.
      2018-02-20 16:36:29 139808677193664 [Note] InnoDB: Waiting for purge to start
      2018-02-20 16:36:29 139808677193664 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.36-83.0 started; log sequence number 3679987115221
      2018-02-20 16:36:29 139752929982208 [Note] InnoDB: Dumping buffer pool(s) not yet started
      2018-02-20 16:36:29 7f1ac3bf7700 InnoDB: Loading buffer pool(s) from .//ib_buffer_pool
      2018-02-20 16:36:29 139808677193664 [Note] Plugin 'FEEDBACK' is disabled.
      2018-02-20 16:36:30 7f1ac3bf7700 InnoDB: Buffer pool(s) load completed at 180220 16:36:30
      2018-02-20 16:36:30 139752929982208 [Warning] InnoDB: Dumping buffer pool(s) to .//ib_buffer_pool
      2018-02-20 16:36:30 139752929982208 [Warning] InnoDB: Buffer pool(s) dump completed at 180220 16:36:30
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: Read nil XID from storage engines, skipping position init
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: wsrep_load(): Galera 25.3.20(r3703) by Codership Oy <info@codership.com> loaded successfully.
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: CRC-32C: using hardware acceleration.
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: Found saved state: e5e263a4-08ca-11e7-bb23-225463a7d968:-1, safe_to_bootsrap: 0
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: Passing config to GCS: base_dir = /data/mysql/; base_host = 10.0.5.12; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /data/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /data/mysql//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 4G; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum =
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: GCache history reset: old(e5e263a4-08ca-11e7-bb23-225463a7d968:0) -> new(e5e263a4-08ca-11e7-bb23-225463a7d968:1263147016)
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: Assign initial position for certification: 1263147016, protocol version: -1
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: wsrep_sst_grab()
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: Start replication
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: Setting initial position to e5e263a4-08ca-11e7-bb23-225463a7d968:1263147016
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: protonet asio version 0
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: Using CRC-32C for message checksums.
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: backend: asio
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: gcomm thread scheduling priority set to other:0 
      2018-02-20 16:37:15 140165551499200 [Warning] WSREP: access file(/data/mysql//gvwstate.dat) failed(No such file or directory)
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: restore pc from disk failed
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: GMCast version 0
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: (ed617f57, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: (ed617f57, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: EVS version 0
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: gcomm: connecting to group 'pup_pdb', peer '10.0.5.11:,10.0.5.12:,10.0.5.13:'
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: (ed617f57, 'tcp://0.0.0.0:4567') connection established to ed617f57 tcp://10.0.5.12:4567
      2018-02-20 16:37:15 140165551499200 [Warning] WSREP: (ed617f57, 'tcp://0.0.0.0:4567') address 'tcp://10.0.5.12:4567' points to own listening address, blacklisting
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: (ed617f57, 'tcp://0.0.0.0:4567') connection established to cdf55f2f tcp://10.0.5.11:4567
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: (ed617f57, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 
      2018-02-20 16:37:15 140165551499200 [Note] WSREP: (ed617f57, 'tcp://0.0.0.0:4567') connection established to 3cb9412b tcp://10.0.5.13:4567
      2018-02-20 16:37:16 140165551499200 [Note] WSREP: declaring 3cb9412b at tcp://10.0.5.13:4567 stable
      2018-02-20 16:37:16 140165551499200 [Note] WSREP: declaring cdf55f2f at tcp://10.0.5.11:4567 stable
      2018-02-20 16:37:16 140165551499200 [Note] WSREP: Node 3cb9412b state prim
      2018-02-20 16:37:16 140165551499200 [Note] WSREP: view(view_id(PRIM,3cb9412b,45) memb {
      	3cb9412b,0
      	cdf55f2f,0
      	ed617f57,0
      } joined {
      } left {
      } partitioned {
      })
      2018-02-20 16:37:16 140165551499200 [Note] WSREP: save pc into disk
      2018-02-20 16:37:16 140165551499200 [Note] WSREP: gcomm: connected
      2018-02-20 16:37:16 140165551499200 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
      2018-02-20 16:37:16 140165551499200 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
      2018-02-20 16:37:16 140165551499200 [Note] WSREP: Opened channel 'pup_pdb'
      2018-02-20 16:37:16 140165551499200 [Note] WSREP: Waiting for SST to complete.
      2018-02-20 16:37:16 140160956098304 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 3
      2018-02-20 16:37:16 140160956098304 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
      2018-02-20 16:37:16 140160956098304 [Note] WSREP: STATE EXCHANGE: sent state msg: edae4b13-1653-11e8-88a0-ceb7bd2e0241
      2018-02-20 16:37:16 140160956098304 [Note] WSREP: STATE EXCHANGE: got state msg: edae4b13-1653-11e8-88a0-ceb7bd2e0241 from 0 (pdb-3)
      2018-02-20 16:37:16 140160956098304 [Note] WSREP: STATE EXCHANGE: got state msg: edae4b13-1653-11e8-88a0-ceb7bd2e0241 from 1 (pdb-1)
      2018-02-20 16:37:16 140160956098304 [Note] WSREP: STATE EXCHANGE: got state msg: edae4b13-1653-11e8-88a0-ceb7bd2e0241 from 2 (pdb-2)
      2018-02-20 16:37:16 140160956098304 [Note] WSREP: Quorum results:
      	version    = 4,
      	component  = PRIMARY,
      	conf_id    = 44,
      	members    = 2/3 (joined/total),
      	act_id     = 1263168047,
      	last_appl. = -1,
      	protocols  = 0/7/3 (gcs/repl/appl),
      	group UUID = e5e263a4-08ca-11e7-bb23-225463a7d968
      2018-02-20 16:37:16 140160956098304 [Note] WSREP: Flow-control interval: [28, 28]
      2018-02-20 16:37:16 140160956098304 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 1263168047)
      2018-02-20 16:37:16 140165551184640 [Note] WSREP: State transfer required: 
      	Group state: e5e263a4-08ca-11e7-bb23-225463a7d968:1263168047
      	Local state: e5e263a4-08ca-11e7-bb23-225463a7d968:1263147016
      2018-02-20 16:37:16 140165551184640 [Note] WSREP: New cluster view: global state: e5e263a4-08ca-11e7-bb23-225463a7d968:1263168047, view# 45: Primary, number of nodes: 3, my index: 2, protocol version 3
      2018-02-20 16:37:16 140165551184640 [Warning] WSREP: Gap in state sequence. Need state transfer.
      2018-02-20 16:37:16 140160926742272 [Note] WSREP: Running: 'wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.0.5.12' --datadir '/data/mysql/'   --parent '1587409' --binlog 'mariadb-bin' '
      WSREP_SST: [INFO] Streaming with xbstream (20180220 16:37:16.661)
      WSREP_SST: [INFO] Using socat as streamer (20180220 16:37:16.663)
      WSREP_SST: [INFO] Stale sst_in_progress file: /data/mysql//sst_in_progress (20180220 16:37:16.667)
      WSREP_SST: [INFO] Evaluating timeout -k 110 100 socat -u TCP-LISTEN:4444,reuseaddr stdio | pv -f  -i 10 -N joiner -F '%N => Rate:%r Avg:%a Elapsed:%t %e Bytes: %b %p'  | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20180220 16:37:16.691)
      2018-02-20 16:37:16 140165551184640 [Note] WSREP: Prepared SST request: xtrabackup-v2|10.0.5.12:4444/xtrabackup_sst//1
      2018-02-20 16:37:16 140165551184640 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
      2018-02-20 16:37:16 140165551184640 [Note] WSREP: REPL Protocols: 7 (3, 2)
      2018-02-20 16:37:16 140165551184640 [Note] WSREP: Assign initial position for certification: 1263168047, protocol version: 3
      2018-02-20 16:37:16 140161014134528 [Note] WSREP: Service thread queue flushed.
      2018-02-20 16:37:16 140165551184640 [Note] WSREP: IST receiver addr using tcp://10.0.5.12:4568
      2018-02-20 16:37:16 140165551184640 [Note] WSREP: Prepared IST receiver, listening at: tcp://10.0.5.12:4568
      2018-02-20 16:37:16 140160956098304 [Note] WSREP: Member 2.0 (pdb-2) requested state transfer from '*any*'. Selected 0.0 (pdb-3)(SYNCED) as donor.
      2018-02-20 16:37:16 140160956098304 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 1263168074)
      2018-02-20 16:37:16 140165551184640 [Note] WSREP: Requesting state transfer: success, donor: 0
      2018-02-20 16:37:16 140165551184640 [Note] WSREP: GCache history reset: old(e5e263a4-08ca-11e7-bb23-225463a7d968:0) -> new(e5e263a4-08ca-11e7-bb23-225463a7d968:1263168047)
         joiner: => Rate:[ 600 B/s] Avg:[ 600 B/s] Elapsed:0:00:00  Bytes:  220 B 
      2018-02-20 16:37:17 140160956098304 [Note] WSREP: 0.0 (pdb-3): State transfer to 2.0 (pdb-2) complete.
      2018-02-20 16:37:17 140160956098304 [Note] WSREP: Member 0.0 (pdb-3) synced with group.
      WSREP_SST: [INFO] NOTE: Joiner-Recv-gtid took 1 seconds (20180220 16:37:17.089)
      WSREP_SST: [INFO] xtrabackup_ist received from donor: Running IST (20180220 16:37:17.097)
      WSREP_SST: [INFO] Galera co-ords from recovery: e5e263a4-08ca-11e7-bb23-225463a7d968:1263147016 1 (20180220 16:37:17.098)
      WSREP_SST: [INFO] Total time on joiner: 1 seconds (20180220 16:37:17.100)
      WSREP_SST: [INFO] Removing the sst_in_progress file (20180220 16:37:17.101)
      2018-02-20 16:37:17 140165551499200 [Note] WSREP: SST complete, seqno: 1263147016
      2018-02-20 16:37:17 140165551499200 [ERROR] Can't open shared library 'metadata_lock_info.so' (errno: 8, API version for INFORMATION SCHEMA plugin METADATA_LOCK_INFO not supported by this version of the server)
      2018-02-20 16:37:17 7f7ad5e827c0 InnoDB: Warning: Setting innodb_buffer_pool_populate is DEPRECATED and has no effect. This option will be removed in MariaDB 10.2.3.
      2018-02-20 16:37:17 140165551499200 [Note] InnoDB: Using mutexes to ref count buffer pool pages
      2018-02-20 16:37:17 140165551499200 [Note] InnoDB: The InnoDB memory heap is disabled
      2018-02-20 16:37:17 140165551499200 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
      2018-02-20 16:37:17 140165551499200 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
      2018-02-20 16:37:17 140165551499200 [Note] InnoDB: Compressed tables use zlib 1.2.8
      2018-02-20 16:37:17 140165551499200 [Note] InnoDB: Using Linux native AIO
      2018-02-20 16:37:17 140165551499200 [Note] InnoDB: Using SSE crc32 instructions
      2018-02-20 16:37:17 140165551499200 [Note] InnoDB: Initializing buffer pool, size = 47.2G
      2018-02-20 16:37:18 140165551499200 [Note] InnoDB: Completed initialization of buffer pool
      2018-02-20 16:37:18 140165551499200 [Note] InnoDB: Highest supported file format is Barracuda.
      2018-02-20 16:37:18 140160964491008 [Note] WSREP: (ed617f57, 'tcp://0.0.0.0:4567') connection to peer ed617f57 with addr tcp://10.0.5.12:4567 timed out, no messages seen in PT3S
      2018-02-20 16:37:18 140165551499200 [Note] InnoDB: 128 rollback segment(s) are active.
      2018-02-20 16:37:18 140165551499200 [Note] InnoDB: Waiting for purge to start
      2018-02-20 16:37:18 140165551499200 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.36-83.0 started; log sequence number 3679987115241
      2018-02-20 16:37:19 140105385735936 [Note] InnoDB: Dumping buffer pool(s) not yet started
      2018-02-20 16:37:19 7f6cd3bf7700 InnoDB: Loading buffer pool(s) from .//ib_buffer_pool
      2018-02-20 16:37:19 140165551499200 [Note] Plugin 'FEEDBACK' is disabled.
      2018-02-20 16:37:19 140160964491008 [Note] WSREP: (ed617f57, 'tcp://0.0.0.0:4567') turning message relay requesting off
      2018-02-20 16:37:19 140165551499200 [Note] Server socket created on IP: '10.0.5.12'.
      2018-02-20 16:37:19 140165551499200 [Note] Server socket created on IP: '10.0.5.12'.
      2018-02-20 16:37:19 140165551499200 [Warning] 'user' entry 'root@platform-node1' ignored in --skip-name-resolve mode.
      2018-02-20 16:37:19 140165551499200 [Warning] 'proxies_priv' entry '@% root@platform-node1' ignored in --skip-name-resolve mode.
      2018-02-20 16:37:19 140165551499200 [Note] WSREP: Signalling provider to continue.
      2018-02-20 16:37:19 140165551499200 [Note] WSREP: SST received: e5e263a4-08ca-11e7-bb23-225463a7d968:1263147016
      2018-02-20 16:37:19 140165551184640 [Note] WSREP: Receiving IST: 21031 writesets, seqnos 1263147016-1263168047
      2018-02-20 16:37:19 140165550050048 [Note] Event Scheduler: scheduler thread started with id 4
      2018-02-20 16:37:19 140165551499200 [Note] Reading of all Master_info entries succeded
      2018-02-20 16:37:19 140165551499200 [Note] Added new Master_info '' to hash table
      2018-02-20 16:37:19 140165551499200 [Note] /usr/sbin/mysqld: ready for connections.
      Version: '10.1.31-MariaDB-1~jessie'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
      2018-02-20 16:37:19 140165533276928 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table backend-1.pds_site_channel_file; Cannot add or update a child row: a foreign key constraint fails (`backend-1`.`pds_site_channel_file`, CONSTRAINT `pds_site_channel_file_ibfk_1` FOREIGN KEY (`site_channel_id`) REFERENCES `pds_site_channel` (`id`) ON DELETE CASCADE ON UPDATE CASCADE), Error_code: 1452; handler error HA_ERR_NO_REFERENCED_ROW; the event's master log FIRST, end_log_pos 258, Internal MariaDB error code: 1452
      2018-02-20 16:37:19 140165533276928 [Warning] WSREP: RBR event 3 Write_rows_v1 apply warning: 151, 1263152029
      2018-02-20 16:37:19 140165533276928 [Warning] WSREP: Failed to apply app buffer: seqno: 1263152029, status: 1
      	 at galera/src/trx_handle.cpp:apply():351
      Retrying 2th time
      2018-02-20 16:37:19 7f6cd3bf7700 InnoDB: Buffer pool(s) load completed at 180220 16:37:19
      2018-02-20 16:37:20 140165551184640 [Note] WSREP: IST received: e5e263a4-08ca-11e7-bb23-225463a7d968:1263168047
      2018-02-20 16:37:20 140160956098304 [Note] WSREP: 2.0 (pdb-2): State transfer from 0.0 (pdb-3) complete.
      2018-02-20 16:37:20 140160956098304 [Note] WSREP: Shifting JOINER -> JOINED (TO: 1263168143)
      2018-02-20 16:37:20 140160956098304 [Note] WSREP: Member 2.0 (pdb-2) synced with group.
      2018-02-20 16:37:20 140160956098304 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 1263168143)
      2018-02-20 16:37:20 140165465668352 [Note] WSREP: Synchronized with group, ready for connections
      2018-02-20 16:37:20 140165465668352 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
      2018-02-20 16:38:15 140165321456384 [Note] Start binlog_dump to slave_server(101), pos(mariadb-bin.018981, 38776402)
      

      Attachments

        Issue Links

          Activity

            People

              sysprg Julius Goryavsky
              tanj Guillaume Lefranc
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.