[MDEV-15517] Problem add node into galera cluster using xtrabackup-v2 - Jira

XML

Word

Printable

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Not a Bug
Affects Version/s: 10.1.31
Fix Version/s: N/A
Component/s: Galera, Galera SST
Labels:
None
Environment:
Centos 7

Description

Mar 8 17:00:14 mysql systemd: Starting MariaDB 10.1.31 database server...
Mar 8 17:00:18 mysql sh: WSREP: Recovered position 00000000-0000-0000-0000-000000000000:-1
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Warning] 'THREAD_CONCURRENCY' is deprecated and will be removed in a future release.
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] /usr/sbin/mysqld (mysqld 10.1.31-MariaDB) starting as process 20780 ...
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: Read nil XID from storage engines, skipping position init
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera/libgalera_smm.so'
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: wsrep_load(): Galera 25.3.22(r3764) by Codership Oy <info@codership.com> loaded successfully.
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: CRC-32C: using hardware acceleration.
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: galera/src/saved_state.cpp:SavedState():115: Found saved state: 00000000-0000-0000-0000-000000000000:-1, safe_to_bootstrap: 1
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: galera/src/galera_gcs.hpp:Gcs():89: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 149.56.18.138; base_port = 4567; cert.log_conflicts = no; debug = true; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT10S; evs.inactive_timeout = PT1M; evs.install_timeout = PT1M; evs.join_retrans_period = PT1S; evs.keepalive_period = PT3S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT30S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /tmp/galera.cache; gcache.page_size = 100M; gcache.recover = no; gcache.size = 512M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.se
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcache/src/GCache_seqno.cpp:seqno_reset():28: GCache history reset: 00000000-0000-0000-0000-000000000000:0 -> 00000000-0000-0000-0000-000000000000:-1
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: galera/src/certification.cpp:assign_initial_position():905: Assign initial position for certification: -1, protocol version: -1
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: wsrep_sst_grab()
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: Start replication
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: galera/src/replicator_smm.cpp:connect():294: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcomm/src/protonet.cpp:create():71: protonet asio version 0
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcomm/src/datagram.cpp:checksum_type():26: Using CRC-32C for message checksums.
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcs/src/gcs_gcomm.cpp:GCommConn():197: backend: asio
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcs/src/gcs_gcomm.cpp:connect():245: gcomm thread scheduling priority set to other:0
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Warning] WSREP: gcomm/src/view.cpp:read_file():311: access file(/var/lib/mysql//gvwstate.dat) failed(No such file or directory)
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcomm/src/pc.cpp:PC():256: restore pc from disk failed
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcomm/src/gmcast.cpp:GMCast():98: GMCast version 0
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcomm/src/gmcast.cpp:GMCast():191: (53c3494f, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcomm/src/gmcast.cpp:GMCast():192: (53c3494f, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcomm/src/evs_proto.cpp:Proto():199: EVS version 0
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcs/src/gcs_gcomm.cpp:connect():275: gcomm: connecting to group 'jmvdbcluster', peer '23.111.157.10:,149.56.18.138:'
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcomm/src/gmcast.cpp:handle_established():551: (53c3494f, 'tcp://0.0.0.0:4567') connection established to 53c3494f tcp://149.56.18.138:4567
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Warning] WSREP: gcomm/src/gmcast.cpp:handle_established():576: (53c3494f, 'tcp://0.0.0.0:4567') address 'tcp://149.56.18.138:4567' points to own listening address, blacklisting
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcomm/src/gmcast.cpp:handle_established():551: (53c3494f, 'tcp://0.0.0.0:4567') connection established to aea4215b tcp://23.111.157.10:4567
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcomm/src/gmcast.cpp:check_liveness():1143: (53c3494f, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers:
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcomm/src/gmcast.cpp:handle_stable_view():1593: declaring aea4215b at tcp://23.111.157.10:4567 stable
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcomm/src/pc_proto.cpp:is_prim():796: Node aea4215b state prim
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcomm/src/pc_proto.cpp:deliver_view():229: view(view_id(PRIM,53c3494f,2) memb

{ Mar 8 17:00:19 mysql mysqld: 53c3494f,0 Mar 8 17:00:19 mysql mysqld: aea4215b,0 Mar 8 17:00:19 mysql mysqld: }

joined

{ Mar 8 17:00:19 mysql mysqld: }

left

{ Mar 8 17:00:19 mysql mysqld: }

partitioned

{ Mar 8 17:00:19 mysql mysqld: }

)
Mar 8 17:00:19 mysql mysqld: 2018-03-08 17:00:19 111511155497216 [Note] WSREP: gcomm/src/pc.cpp:handle_up():30: save pc into disk
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111511155497216 [Note] WSREP: gcs/src/gcs_gcomm.cpp:connect():285: gcomm: connected
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111511155497216 [Note] WSREP: gcs/src/gcs_core.cpp:gcs_core_set_pkt_size():1254: Changing maximum packet size to 64500, resulting msg size: 32636
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111511155497216 [Note] WSREP: gcs/src/gcs.cpp:gcs_shift_state():635: Shifting CLOSED -> OPEN (TO: 0)
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111511155497216 [Note] WSREP: gcs/src/gcs.cpp:gcs_open():1445: Opened channel 'jmvdbcluster'
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111511155497216 [Note] WSREP: Waiting for SST to complete.
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111510449239808 [Note] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_comp_msg():409: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111510449239808 [Note] WSREP: gcs/src/gcs_core.cpp:core_handle_comp_msg():767: STATE_EXCHANGE: sent state UUID: 545c2997-230b-11e8-8961-fba151d00f7d
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111510449239808 [Note] WSREP: gcs/src/gcs_core.cpp:core_handle_uuid_msg():862: STATE EXCHANGE: sent state msg: 545c2997-230b-11e8-8961-fba151d00f7d
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111510449239808 [Note] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_state_msg():582: STATE EXCHANGE: got state msg: 545c2997-230b-11e8-8961-fba151d00f7d from 0 (galera4)
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111510449239808 [Note] WSREP: gcs/src/gcs_group.cpp:group_print_state_debug():564:
Mar 8 17:00:20 mysql mysqld: Version : 4
Mar 8 17:00:20 mysql mysqld: Flags : 0x1
Mar 8 17:00:20 mysql mysqld: Protocols : 0 / 7 / 3
Mar 8 17:00:20 mysql mysqld: State : NON-PRIMARY
Mar 8 17:00:20 mysql mysqld: Desync count : 0
Mar 8 17:00:20 mysql mysqld: Prim state : NON-PRIMARY
Mar 8 17:00:20 mysql mysqld: Prim UUID : 00000000-0000-0000-0000-000000000000
Mar 8 17:00:20 mysql mysqld: Prim seqno : -1
Mar 8 17:00:20 mysql mysqld: First seqno : -1
Mar 8 17:00:20 mysql mysqld: Last seqno : -1
Mar 8 17:00:20 mysql mysqld: Prim JOINED : 0
Mar 8 17:00:20 mysql mysqld: State UUID : 545c2997-230b-11e8-8961-fba151d00f7d
Mar 8 17:00:20 mysql mysqld: Group UUID : 00000000-0000-0000-0000-000000000000
Mar 8 17:00:20 mysql mysqld: Name : 'galera4'
Mar 8 17:00:20 mysql mysqld: Incoming addr: '149.56.18.138:3306'
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111510449239808 [Note] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_state_msg():582: STATE EXCHANGE: got state msg: 545c2997-230b-11e8-8961-fba151d00f7d from 1 (galera1)
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111510449239808 [Note] WSREP: gcs/src/gcs_group.cpp:group_print_state_debug():564:
Mar 8 17:00:20 mysql mysqld: Version : 4
Mar 8 17:00:20 mysql mysqld: Flags : 0x2
Mar 8 17:00:20 mysql mysqld: Protocols : 0 / 7 / 3
Mar 8 17:00:20 mysql mysqld: State : SYNCED
Mar 8 17:00:20 mysql mysqld: Desync count : 0
Mar 8 17:00:20 mysql mysqld: Prim state : SYNCED
Mar 8 17:00:20 mysql mysqld: Prim UUID : aea4650d-230b-11e8-b226-3e129a191949
Mar 8 17:00:20 mysql mysqld: Prim seqno : 1
Mar 8 17:00:20 mysql mysqld: First seqno : -1
Mar 8 17:00:20 mysql mysqld: Last seqno : 0
Mar 8 17:00:20 mysql mysqld: Prim JOINED : 1
Mar 8 17:00:20 mysql mysqld: State UUID : 545c2997-230b-11e8-8961-fba151d00f7d
Mar 8 17:00:20 mysql mysqld: Group UUID : 4f8aa364-2306-11e8-b094-3a03f9b52e07
Mar 8 17:00:20 mysql mysqld: Name : 'galera1'
Mar 8 17:00:20 mysql mysqld: Incoming addr: '23.111.157.10:3306'
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111510449239808 [Note] WSREP: gcs/src/gcs_group.cpp:group_post_state_exchange():368: Quorum results:
Mar 8 17:00:20 mysql mysqld: version = 4,
Mar 8 17:00:20 mysql mysqld: component = PRIMARY,
Mar 8 17:00:20 mysql mysqld: conf_id = 1,
Mar 8 17:00:20 mysql mysqld: members = 1/2 (joined/total),
Mar 8 17:00:20 mysql mysqld: act_id = 0,
Mar 8 17:00:20 mysql mysqld: last_appl. = -1,
Mar 8 17:00:20 mysql mysqld: protocols = 0/7/3 (gcs/repl/appl),
Mar 8 17:00:20 mysql mysqld: group UUID = 4f8aa364-2306-11e8-b094-3a03f9b52e07
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111510449239808 [Note] WSREP: gcs/src/gcs.cpp:_set_fc_limits():835: Flow-control interval: [23, 23]
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111510449239808 [Note] WSREP: gcs/src/gcs_sm.hpp:gcs_sm_continue():452: Trying to continue unpaused monitor
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111510449239808 [Note] WSREP: gcs/src/gcs.cpp:gcs_shift_state():635: Shifting OPEN -> PRIMARY (TO: 0)
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111511076341504 [Note] WSREP: galera/src/replicator_smm.cpp:process_conf_change():1405: State transfer required:
Mar 8 17:00:20 mysql mysqld: Group state: 4f8aa364-2306-11e8-b094-3a03f9b52e07:0
Mar 8 17:00:20 mysql mysqld: Local state: 00000000-0000-0000-0000-000000000000:-1
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111511076341504 [Note] WSREP: New cluster view: global state: 4f8aa364-2306-11e8-b094-3a03f9b52e07:0, view# 2: Primary, number of nodes: 2, my index: 0, protocol version 3
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111511076341504 [Warning] WSREP: Gap in state sequence. Need state transfer.
Mar 8 17:00:20 mysql mysqld: 2018-03-08 17:00:20 111510419597056 [Note] WSREP: Running: 'wsrep_sst_xtrabackup-v2 --role 'joiner' --address '149.56.18.138' --datadir '/var/lib/mysql/' --parent '20780' --binlog 'mysql-bin' '
Mar 8 17:00:20 mysql mysqld: WSREP_SST: [INFO] Streaming with xbstream (20180308 17:00:20.383)
Mar 8 17:00:20 mysql mysqld: WSREP_SST: [INFO] Using socat as streamer (20180308 17:00:20.384)
Mar 8 17:00:20 mysql mysqld: WSREP_SST: [INFO] Stale sst_in_progress file: /var/lib/mysql//sst_in_progress (20180308 17:00:20.387)
Mar 8 17:00:20 mysql mysqld: WSREP_SST: [INFO] Evaluating timeout -k 110 100 socat -u TCP-LISTEN:4444,reuseaddr stdio | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20180308 17:00:20.404)
Mar 8 17:00:22 mysql mysqld: 2018-03-08 17:00:22 111510457947904 [Note] WSREP: gcomm/src/gmcast.cpp:check_liveness():1084: (53c3494f, 'tcp://0.0.0.0:4567') connection to peer 53c3494f with addr tcp://149.56.18.138:4567 timed out, no messages seen in PT3S
Mar 8 17:00:22 mysql mysqld: 2018-03-08 17:00:22 111510457947904 [Note] WSREP: gcomm/src/gmcast.cpp:check_liveness():1184: (53c3494f, 'tcp://0.0.0.0:4567') turning message relay requesting off
Mar 8 17:00:36 mysql kernel: Firewall: TCP_IN Blocked IN=eth0 OUT= MAC=0c:c4:7a:c3:4f:18:00:ff:ff:ff:ff:fd:08:00 SRC=211.157.226.130 DST=149.56.18.138 LEN=40 TOS=0x00 PREC=0xE0 TTL=233 ID=11253 PROTO=TCP SPT=11980 DPT=1433 WINDOW=1024 RES=0x00 SYN URGP=0
Mar 8 17:00:40 mysql kernel: Firewall: TCP_IN Blocked IN=eth0 OUT= MAC=0c:c4:7a:c3:4f:18:00:ff:ff:ff:ff:fd:08:00 SRC=218.70.218.46 DST=149.56.18.138 LEN=40 TOS=0x00 PREC=0xE0 TTL=237 ID=13413 PROTO=TCP SPT=50135 DPT=1433 WINDOW=1024 RES=0x00 SYN URGP=0
Mar 8 17:00:58 mysql kernel: Firewall: TCP_IN Blocked IN=eth0 OUT= MAC=0c:c4:7a:c3:4f:18:00:ff:ff:ff:ff:fd:08:00 SRC=185.169.230.3 DST=149.56.18.138 LEN=40 TOS=0x00 PREC=0x00 TTL=243 ID=54321 PROTO=TCP SPT=48099 DPT=8545 WINDOW=65535 RES=0x00 SYN URGP=0
Mar 8 17:00:58 mysql kernel: Firewall: TCP_IN Blocked IN=eth0 OUT= MAC=0c:c4:7a:c3:4f:18:00:ff:ff:ff:ff:fd:08:00 SRC=185.169.230.3 DST=149.56.18.138 LEN=40 TOS=0x00 PREC=0x00 TTL=243 ID=54321 PROTO=TCP SPT=48100 DPT=8545 WINDOW=65535 RES=0x00 SYN URGP=0
Mar 8 17:00:58 mysql kernel: Firewall: TCP_IN Blocked IN=eth0 OUT= MAC=0c:c4:7a:c3:4f:18:00:ff:ff:ff:ff:fd:08:00 SRC=185.169.230.3 DST=149.56.18.138 LEN=40 TOS=0x00 PREC=0x00 TTL=243 ID=54321 PROTO=TCP SPT=48101 DPT=8545 WINDOW=65535 RES=0x00 SYN URGP=0
Mar 8 17:01:01 mysql systemd: Started Session c698059 of user root.
Mar 8 17:01:01 mysql systemd: Starting Session c698059 of user root.
Mar 8 17:01:02 mysql systemd: Started Session c698060 of user root.
Mar 8 17:01:02 mysql systemd: Starting Session c698060 of user root.
Mar 8 17:01:02 mysql kernel: Firewall: UDP_OUT Blocked IN= OUT=eth0 SRC=149.56.18.138 DST=37.187.231.251 LEN=211 TOS=0x00 PREC=0x00 TTL=64 ID=42212 DF PROTO=UDP SPT=57074 DPT=6114 LEN=191 UID=0 GID=0
Mar 8 17:01:02 mysql kernel: Firewall: UDP_OUT Blocked IN= OUT=eth0 SRC=149.56.18.138 DST=37.187.231.251 LEN=217 TOS=0x00 PREC=0x00 TTL=64 ID=42214 DF PROTO=UDP SPT=46266 DPT=6168 LEN=197 UID=0 GID=0
Mar 8 17:01:02 mysql kernel: Firewall: UDP_OUT Blocked IN= OUT=eth0 SRC=149.56.18.138 DST=37.187.231.251 LEN=199 TOS=0x00 PREC=0x00 TTL=64 ID=42215 DF PROTO=UDP SPT=41947 DPT=6155 LEN=179 UID=0 GID=0
Mar 8 17:01:02 mysql kernel: Firewall: UDP_OUT Blocked IN= OUT=eth0 SRC=149.56.18.138 DST=37.187.231.251 LEN=212 TOS=0x00 PREC=0x00 TTL=64 ID=42216 DF PROTO=UDP SPT=55692 DPT=6188 LEN=192 UID=0 GID=0
Mar 8 17:01:02 mysql kernel: Firewall: UDP_OUT Blocked IN= OUT=eth0 SRC=149.56.18.138 DST=37.187.231.251 LEN=226 TOS=0x00 PREC=0x00 TTL=64 ID=42217 DF PROTO=UDP SPT=52963 DPT=6131 LEN=206 UID=0 GID=0
Mar 8 17:01:22 mysql mysqld: 2018-03-08 17:01:22 111511076341504 [Note] WSREP: Prepared SST request: xtrabackup-v2|149.56.18.138:4444/xtrabackup_sst//149.56.18.138
Mar 8 17:01:22 mysql mysqld: 2018-03-08 17:01:22 111511076341504 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
Mar 8 17:01:22 mysql mysqld: 2018-03-08 17:01:22 111511076341504 [Note] WSREP: galera/src/replicator_smm.cpp:establish_protocol_versions():1325: REPL Protocols: 7 (3, 2)
Mar 8 17:01:22 mysql mysqld: 2018-03-08 17:01:22 111511076341504 [Note] WSREP: galera/src/certification.cpp:assign_initial_position():905: Assign initial position for certification: 0, protocol version: 3
Mar 8 17:01:22 mysql mysqld: 2018-03-08 17:01:22 111510507751168 [Note] WSREP: galera/src/galera_service_thd.cpp:thd_func():39: Service thread queue flushed.
Mar 8 17:01:22 mysql mysqld: 2018-03-08 17:01:22 111511076341504 [Warning] WSREP: galera/src/replicator_str.cpp:prepare_state_request():539: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (4f8aa364-2306-11e8-b094-3a03f9b52e07): 1 (Operation not permitted)
Mar 8 17:01:22 mysql mysqld: at galera/src/replicator_str.cpp:prepare_for_IST():482. IST will be unavailable.
Mar 8 17:01:22 mysql mysqld: 2018-03-08 17:01:22 111510449239808 [Note] WSREP: gcs/src/gcs_group.cpp:group_select_donor():1268: Member 0.0 (galera4) requested state transfer from 'galera1'. Selected 1.0 (galera1)(SYNCED) as donor.
Mar 8 17:01:22 mysql mysqld: 2018-03-08 17:01:22 111510449239808 [Note] WSREP: gcs/src/gcs.cpp:gcs_shift_state():635: Shifting PRIMARY -> JOINER (TO: 0)
Mar 8 17:01:22 mysql mysqld: 2018-03-08 17:01:22 111511076341504 [Note] WSREP: galera/src/replicator_str.cpp:send_state_request():638: Requesting state transfer: success, donor: 1
Mar 8 17:01:22 mysql mysqld: 2018-03-08 17:01:22 111511076341504 [Note] WSREP: gcache/src/GCache_seqno.cpp:seqno_reset():28: GCache history reset: 00000000-0000-0000-0000-000000000000:0 -> 4f8aa364-2306-11e8-b094-3a03f9b52e07:0
Mar 8 17:01:22 mysql mysqld: WSREP_SST: [INFO] WARNING: Stale temporary SST directory: /var/lib/mysql//.sst from previous state transfer. Removing (20180308 17:01:22.526)
Mar 8 17:01:22 mysql mysqld: WSREP_SST: [INFO] Proceeding with SST (20180308 17:01:22.530)
Mar 8 17:01:22 mysql mysqld: WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:4444,reuseaddr stdio | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20180308 17:01:22.530)
Mar 8 17:01:22 mysql mysqld: WSREP_SST: [INFO] Cleaning the existing datadir and innodb-data/log directories (20180308 17:01:22.531)
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/performance_schema/db.opt’
Mar 8 17:01:22 mysql mysqld: removed directory: ‘/var/lib/mysql/performance_schema’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql-bin.000001’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql.sock’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql-bin.000003’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql-bin.state’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/ib_logfile0’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/ib_logfile1’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/ibdata1’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql-bin.000004’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/multi-master.info’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/aria_log_control’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/aria_log.00000001’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/tables_priv.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/time_zone.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/help_keyword.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/host.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/time_zone_name.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/db.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/func.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/help_relation.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/event.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/index_stats.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/time_zone_transition.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/tables_priv.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/time_zone.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/roles_mapping.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/gtid_slave_pos.ibd’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/gtid_slave_pos.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/columns_priv.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/innodb_index_stats.ibd’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/time_zone_name.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/time_zone_transition_type.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/help_topic.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/db.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/time_zone_transition_type.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/event.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/help_topic.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/table_stats.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/help_category.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/help_topic.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/roles_mapping.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/index_stats.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/proxies_priv.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/innodb_table_stats.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/time_zone.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/proc.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/host.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/column_stats.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/time_zone_leap_second.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/plugin.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/proc.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/servers.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/proxies_priv.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/help_relation.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/column_stats.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/help_category.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/innodb_table_stats.ibd’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/tables_priv.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/time_zone_transition.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/slow_log.CSM’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/help_relation.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/slow_log.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/servers.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/columns_priv.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/event.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/time_zone_leap_second.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/func.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/help_category.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/innodb_index_stats.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/time_zone_name.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/general_log.CSV’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/procs_priv.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/procs_priv.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/procs_priv.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/time_zone_transition.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/general_log.CSM’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/user.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/user.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/time_zone_leap_second.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/plugin.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/func.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/index_stats.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/user.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/help_keyword.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/plugin.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/columns_priv.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/general_log.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/db.MYI’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/table_stats.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/column_stats.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/help_keyword.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/time_zone_transition_type.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/roles_mapping.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/servers.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/table_stats.frm’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/proxies_priv.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/proc.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/host.MYD’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql/slow_log.CSV’
Mar 8 17:01:22 mysql mysqld: removed directory: ‘/var/lib/mysql/mysql’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql-bin.000002’
Mar 8 17:01:22 mysql mysqld: removed ‘/var/lib/mysql/mysql-bin.index’
Mar 8 17:01:22 mysql mysqld: WSREP_SST: [INFO] Waiting for SST streaming to complete! (20180308 17:01:22.562)
Mar 8 17:01:32 mysql mysqld: WSREP_SST: [ERROR] xtrabackup_checkpoints missing, failed innobackupex/SST on donor (20180308 17:01:32.566)
Mar 8 17:01:32 mysql mysqld: WSREP_SST: [ERROR] Cleanup after exit with status:2 (20180308 17:01:32.568)
Mar 8 17:01:32 mysql mysqld: 2018-03-08 17:01:32 111510419597056 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '149.56.18.138' --datadir '/var/lib/mysql/' --parent '20780' --binlog 'mysql-bin' : 2 (No such file or directory)
Mar 8 17:01:32 mysql mysqld: 2018-03-08 17:01:32 111510419597056 [ERROR] WSREP: Failed to read uuid:seqno and wsrep_gtid_domain_id from joiner script.
Mar 8 17:01:32 mysql mysqld: 2018-03-08 17:01:32 111511155497216 [ERROR] WSREP: SST failed: 2 (No such file or directory)
Mar 8 17:01:32 mysql mysqld: 2018-03-08 17:01:32 111511155497216 [ERROR] Aborting
Mar 8 17:01:32 mysql mysqld: 2018-03-08 17:01:32 111510449239808 [Warning] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():730: 1.0 (galera1): State transfer to 0.0 (galera4) failed: -22 (Invalid argument)
Mar 8 17:01:32 mysql mysqld: 2018-03-08 17:01:32 111510449239808 [ERROR] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():736: Will never receive state. Need to abort.

Attachments

Activity

People

Assignee:: Jan Lindström (Inactive)

Reporter:: Janiel Madureira Oliveira

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 2018-03-08 20:08

Updated:: 2019-05-20 05:35

Resolved:: 2019-05-20 05:35

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.