[ec2-user@ip-<node1_ip> my.cnf.d]$ sudo mysqld --user=root --wsrep-new-cluster
2016-08-01 12:20:41 139906927581312 [Note] mysqld (mysqld 10.1.16-MariaDB) starting as process 20834 ...
2016-08-01 12:20:41 139906927581312 [Note] WSREP: Read nil XID from storage engines, skipping position init
2016-08-01 12:20:41 139906927581312 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera/libgalera_smm.so'
2016-08-01 12:20:41 139906927581312 [Note] WSREP: wsrep_load(): Galera 25.3.15(r3578) by Codership Oy <info@codership.com> loaded successfully.
2016-08-01 12:20:41 139906927581312 [Note] WSREP: CRC-32C: using hardware acceleration.
2016-08-01 12:20:41 139906927581312 [Note] WSREP: Found saved state: 3a3e195f-57f1-11e6-92fd-5a5896716e64:0
2016-08-01 12:20:41 139906927581312 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = <node1_ip>; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 128M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; p
2016-08-01 12:20:41 139906688612096 [Note] WSREP: Service thread queue flushed.
2016-08-01 12:20:41 139906927581312 [Note] WSREP: Assign initial position for certification: 0, protocol version: -1
2016-08-01 12:20:41 139906927581312 [Note] WSREP: wsrep_sst_grab()
2016-08-01 12:20:41 139906927581312 [Note] WSREP: Start replication
2016-08-01 12:20:41 139906927581312 [Note] WSREP: 'wsrep-new-cluster' option used, bootstrapping the cluster
2016-08-01 12:20:41 139906927581312 [Note] WSREP: Setting initial position to 3a3e195f-57f1-11e6-92fd-5a5896716e64:0
2016-08-01 12:20:41 139906927581312 [Note] WSREP: protonet asio version 0
2016-08-01 12:20:41 139906927581312 [Note] WSREP: Using CRC-32C for message checksums.
2016-08-01 12:20:41 139906927581312 [Note] WSREP: backend: asio
2016-08-01 12:20:41 139906927581312 [Warning] WSREP: access file(/var/lib/mysql//gvwstate.dat) failed(No such file or directory)
2016-08-01 12:20:41 139906927581312 [Note] WSREP: restore pc from disk failed
2016-08-01 12:20:41 139906927581312 [Note] WSREP: GMCast version 0
2016-08-01 12:20:41 139906927581312 [Note] WSREP: (e3cbb6e5, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
2016-08-01 12:20:41 139906927581312 [Note] WSREP: (e3cbb6e5, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
2016-08-01 12:20:41 139906927581312 [Note] WSREP: EVS version 0
2016-08-01 12:20:41 139906927581312 [Note] WSREP: gcomm: bootstrapping new group 'istcdh_mariadb_cluster'
2016-08-01 12:20:41 139906927581312 [Note] WSREP: start_prim is enabled, turn off pc_recovery
2016-08-01 12:20:41 139906927581312 [Note] WSREP: Node e3cbb6e5 state prim
2016-08-01 12:20:41 139906927581312 [Note] WSREP: view(view_id(PRIM,e3cbb6e5,1) memb {
        e3cbb6e5,0
} joined {
} left {
} partitioned {
})
2016-08-01 12:20:41 139906927581312 [Note] WSREP: save pc into disk
2016-08-01 12:20:41 139906927581312 [Note] WSREP: discarding pending addr without UUID: tcp://<node1_ip>:4567
2016-08-01 12:20:41 139906927581312 [Note] WSREP: discarding pending addr proto entry 0x7f3e99ce0f00
2016-08-01 12:20:41 139906927581312 [Note] WSREP: discarding pending addr without UUID: tcp://<node2_ip>:4567
2016-08-01 12:20:41 139906927581312 [Note] WSREP: discarding pending addr proto entry 0x7f3e99ce0fc0
2016-08-01 12:20:41 139906927581312 [Note] WSREP: gcomm: connected
2016-08-01 12:20:41 139906927581312 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
2016-08-01 12:20:41 139906927581312 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
2016-08-01 12:20:41 139906927581312 [Note] WSREP: Opened channel 'istcdh_mariadb_cluster'
2016-08-01 12:20:41 139906621892352 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1
2016-08-01 12:20:41 139906927581312 [Note] WSREP: Waiting for SST to complete.
2016-08-01 12:20:41 139906621892352 [Note] WSREP: STATE_EXCHANGE: sent state UUID: e3cc5b2a-5803-11e6-b741-57a2d7eb870c
2016-08-01 12:20:41 139906621892352 [Note] WSREP: STATE EXCHANGE: sent state msg: e3cc5b2a-5803-11e6-b741-57a2d7eb870c
2016-08-01 12:20:41 139906621892352 [Note] WSREP: STATE EXCHANGE: got state msg: e3cc5b2a-5803-11e6-b741-57a2d7eb870c from 0 (ip-<node1_ip>.eu-west-1.compute.internal)
2016-08-01 12:20:41 139906621892352 [Note] WSREP: Quorum results:
        version    = 3,
        component  = PRIMARY,
        conf_id    = 0,
        members    = 1/1 (joined/total),
        act_id     = 0,
        last_appl. = -1,
        protocols  = 0/7/3 (gcs/repl/appl),
        group UUID = 3a3e195f-57f1-11e6-92fd-5a5896716e64
2016-08-01 12:20:41 139906621892352 [Note] WSREP: Flow-control interval: [16, 16]
2016-08-01 12:20:41 139906621892352 [Note] WSREP: Restored state OPEN -> JOINED (0)
2016-08-01 12:20:41 139906621892352 [Note] WSREP: Member 0.0 (ip-<node1_ip>.eu-west-1.compute.internal) synced with group.
2016-08-01 12:20:41 139906621892352 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 0)
2016-08-01 12:20:41 139906927266560 [Note] WSREP: New cluster view: global state: 3a3e195f-57f1-11e6-92fd-5a5896716e64:0, view# 1: Primary, number of nodes: 1, my index: 0, protocol version 3
2016-08-01 12:20:41 139906927581312 [Note] WSREP: SST complete, seqno: 0
2016-08-01 12:20:41 139906927581312 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2016-08-01 12:20:41 139906927581312 [Note] InnoDB: The InnoDB memory heap is disabled
2016-08-01 12:20:41 139906927581312 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2016-08-01 12:20:41 139906927581312 [Note] InnoDB: Memory barrier is not used
2016-08-01 12:20:41 139906927581312 [Note] InnoDB: Compressed tables use zlib 1.2.7
2016-08-01 12:20:41 139906927581312 [Note] InnoDB: Using Linux native AIO
2016-08-01 12:20:41 139906927581312 [Note] InnoDB: Using SSE crc32 instructions
2016-08-01 12:20:41 139906927581312 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2016-08-01 12:20:41 139906927581312 [Note] InnoDB: Completed initialization of buffer pool
2016-08-01 12:20:41 139906927581312 [Note] InnoDB: Highest supported file format is Barracuda.
2016-08-01 12:20:41 139906927581312 [Note] InnoDB: 128 rollback segment(s) are active.
2016-08-01 12:20:41 139906927581312 [Note] InnoDB: Waiting for purge to start
2016-08-01 12:20:41 139906927581312 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.30-76.3 started; log sequence number 1618269
2016-08-01 12:20:41 139905954981632 [Note] InnoDB: Dumping buffer pool(s) not yet started
2016-08-01 12:20:41 139906927581312 [Note] Plugin 'FEEDBACK' is disabled.
2016-08-01 12:20:41 139906927581312 [Note] Server socket created on IP: '::'.
2016-08-01 12:20:41 139906927581312 [Note] mysqld: ready for connections.
Version: '10.1.16-MariaDB'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MariaDB Server
2016-08-01 12:20:41 139906927266560 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2016-08-01 12:20:41 139906927266560 [Note] WSREP: REPL Protocols: 7 (3, 2)
2016-08-01 12:20:41 139906688612096 [Note] WSREP: Service thread queue flushed.
2016-08-01 12:20:41 139906927266560 [Note] WSREP: Assign initial position for certification: 0, protocol version: 3
2016-08-01 12:20:41 139906688612096 [Note] WSREP: Service thread queue flushed.
2016-08-01 12:20:41 139906927266560 [Note] WSREP: Synchronized with group, ready for connections
2016-08-01 12:20:41 139906927266560 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2016-08-01 12:21:11 139906630285056 [Note] WSREP: (e3cbb6e5, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers:
2016-08-01 12:21:12 139906630285056 [Note] WSREP: declaring f5f23753 at tcp://<node2_ip>:4567 stable
2016-08-01 12:21:12 139906630285056 [Note] WSREP: Node e3cbb6e5 state prim
2016-08-01 12:21:12 139906630285056 [Note] WSREP: view(view_id(PRIM,e3cbb6e5,2) memb {
        e3cbb6e5,0
        f5f23753,0
} joined {
} left {
} partitioned {
})
2016-08-01 12:21:12 139906630285056 [Note] WSREP: save pc into disk
2016-08-01 12:21:12 139906621892352 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2
2016-08-01 12:21:12 139906621892352 [Note] WSREP: STATE_EXCHANGE: sent state UUID: f63f1911-5803-11e6-93b7-5660a04a755c
2016-08-01 12:21:12 139906621892352 [Note] WSREP: STATE EXCHANGE: sent state msg: f63f1911-5803-11e6-93b7-5660a04a755c
2016-08-01 12:21:12 139906621892352 [Note] WSREP: STATE EXCHANGE: got state msg: f63f1911-5803-11e6-93b7-5660a04a755c from 0 (ip-<node1_ip>.eu-west-1.compute.internal)
2016-08-01 12:21:12 139906621892352 [Note] WSREP: STATE EXCHANGE: got state msg: f63f1911-5803-11e6-93b7-5660a04a755c from 1 (ip-<node1_ip>.eu-west-1.compute.internal)
2016-08-01 12:21:12 139906621892352 [Note] WSREP: Quorum results:
        version    = 3,
        component  = PRIMARY,
        conf_id    = 1,
        members    = 1/2 (joined/total),
        act_id     = 0,
        last_appl. = 0,
        protocols  = 0/7/3 (gcs/repl/appl),
        group UUID = 3a3e195f-57f1-11e6-92fd-5a5896716e64
2016-08-01 12:21:12 139906621892352 [Note] WSREP: Flow-control interval: [23, 23]
2016-08-01 12:21:12 139906927266560 [Note] WSREP: New cluster view: global state: 3a3e195f-57f1-11e6-92fd-5a5896716e64:0, view# 2: Primary, number of nodes: 2, my index: 0, protocol version 3
2016-08-01 12:21:12 139906927266560 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2016-08-01 12:21:12 139906927266560 [Note] WSREP: REPL Protocols: 7 (3, 2)
2016-08-01 12:21:12 139906688612096 [Note] WSREP: Service thread queue flushed.
2016-08-01 12:21:12 139906927266560 [Note] WSREP: Assign initial position for certification: 0, protocol version: 3
2016-08-01 12:21:12 139906688612096 [Note] WSREP: Service thread queue flushed.
2016-08-01 12:21:12 139906621892352 [Note] WSREP: Member 1.0 (ip-<node1_ip>.eu-west-1.compute.internal) requested state transfer from '*any*'. Selected 0.0 (ip-<node1_ip>.eu-west-1.compute.internal)(SYNCED) as donor.
2016-08-01 12:21:12 139906621892352 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 0)
2016-08-01 12:21:12 139906927266560 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2016-08-01 12:21:12 139905883698944 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'donor' --address '<node2_ip>:4444/rsync_sst' --socket '/var/lib/mysql/mysql.sock' --datadir '/var/lib/mysql/'     '' --gtid '3a3e195f-57f1-11e6-92fd-5a5896716e64:0' --gtid-domain-id '0''
2016-08-01 12:21:12 139906927266560 [Note] WSREP: sst_donor_thread signaled with 0
2016-08-01 12:21:12 139905883698944 [Note] WSREP: Flushing tables for SST...
2016-08-01 12:21:12 139905883698944 [Note] WSREP: Provider paused at 3a3e195f-57f1-11e6-92fd-5a5896716e64:0 (5)
2016-08-01 12:21:12 139905883698944 [Note] WSREP: Tables flushed.
rsync: open "ibdata1" (in rsync_sst) failed: Permission denied (13)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9]
WSREP_SST: [ERROR] rsync returned code 23: (20160801 12:21:13.022)
2016-08-01 12:21:13 139905883698944 [ERROR] WSREP: Failed to read from: wsrep_sst_rsync --role 'donor' --address '<node2_ip>:4444/rsync_sst' --socket '/var/lib/mysql/mysql.sock' --datadir '/var/lib/mysql/'     '' --gtid '3a3e195f-57f1-11e6-92fd-5a5896716e64:0' --gtid-domain-id '0'
2016-08-01 12:21:13 139905883698944 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'donor' --address '<node2_ip>:4444/rsync_sst' --socket '/var/lib/mysql/mysql.sock' --datadir '/var/lib/mysql/'     '' --gtid '3a3e195f-57f1-11e6-92fd-5a5896716e64:0' --gtid-domain-id '0': 255 (Unknown error 255)
2016-08-01 12:21:13 139905883698944 [Note] WSREP: resuming provider at 5
2016-08-01 12:21:13 139905883698944 [Note] WSREP: Provider resumed.
2016-08-01 12:21:13 139905883698944 [ERROR] WSREP: Command did not run: wsrep_sst_rsync --role 'donor' --address '<node2_ip>:4444/rsync_sst' --socket '/var/lib/mysql/mysql.sock' --datadir '/var/lib/mysql/'     '' --gtid '3a3e195f-57f1-11e6-92fd-5a5896716e64:0' --gtid-domain-id '0'
2016-08-01 12:21:13 139906621892352 [Warning] WSREP: 0.0 (ip-<node1_ip>.eu-west-1.compute.internal): State transfer to 1.0 (ip-<node1_ip>.eu-west-1.compute.internal) failed: -255 (Unknown error 255)
2016-08-01 12:21:13 139906621892352 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 0)
2016-08-01 12:21:14 139906621892352 [Note] WSREP: Member 0.0 (ip-<node1_ip>.eu-west-1.compute.internal) synced with group.
2016-08-01 12:21:14 139906621892352 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 0)
2016-08-01 12:21:14 139906630285056 [Note] WSREP: forgetting f5f23753 (tcp://<node2_ip>:4567)
2016-08-01 12:21:14 139906927266560 [Note] WSREP: Synchronized with group, ready for connections
2016-08-01 12:21:14 139906927266560 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2016-08-01 12:21:14 139906630285056 [Note] WSREP: Node e3cbb6e5 state prim
2016-08-01 12:21:14 139906630285056 [Note] WSREP: view(view_id(PRIM,e3cbb6e5,3) memb {
        e3cbb6e5,0
} joined {
} left {
} partitioned {
        f5f23753,0
})
2016-08-01 12:21:14 139906630285056 [Note] WSREP: save pc into disk
2016-08-01 12:21:14 139906630285056 [Note] WSREP: forgetting f5f23753 (tcp://<node2_ip>:4567)
2016-08-01 12:21:14 139906621892352 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1
2016-08-01 12:21:14 139906621892352 [Note] WSREP: STATE_EXCHANGE: sent state UUID: f766fc04-5803-11e6-8647-c7bd6d93bff8
2016-08-01 12:21:14 139906621892352 [Note] WSREP: STATE EXCHANGE: sent state msg: f766fc04-5803-11e6-8647-c7bd6d93bff8
2016-08-01 12:21:14 139906621892352 [Note] WSREP: STATE EXCHANGE: got state msg: f766fc04-5803-11e6-8647-c7bd6d93bff8 from 0 (ip-<node1_ip>.eu-west-1.compute.internal)
2016-08-01 12:21:14 139906621892352 [Note] WSREP: Quorum results:
        version    = 3,
        component  = PRIMARY,
        conf_id    = 2,
        members    = 1/1 (joined/total),
        act_id     = 0,
        last_appl. = 0,
        protocols  = 0/7/3 (gcs/repl/appl),
        group UUID = 3a3e195f-57f1-11e6-92fd-5a5896716e64
2016-08-01 12:21:14 139906621892352 [Note] WSREP: Flow-control interval: [16, 16]
2016-08-01 12:21:14 139906927266560 [Note] WSREP: New cluster view: global state: 3a3e195f-57f1-11e6-92fd-5a5896716e64:0, view# 3: Primary, number of nodes: 1, my index: 0, protocol version 3
2016-08-01 12:21:14 139906927266560 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2016-08-01 12:21:14 139906927266560 [Note] WSREP: REPL Protocols: 7 (3, 2)
2016-08-01 12:21:14 139906688612096 [Note] WSREP: Service thread queue flushed.
2016-08-01 12:21:14 139906927266560 [Note] WSREP: Assign initial position for certification: 0, protocol version: 3
2016-08-01 12:21:14 139906688612096 [Note] WSREP: Service thread queue flushed.
2016-08-01 12:21:14 139906630285056 [Note] WSREP: (e3cbb6e5, 'tcp://0.0.0.0:4567') turning message relay requesting off
2016-08-01 12:21:19 139906630285056 [Note] WSREP:  cleaning up f5f23753 (tcp://<node2_ip>:4567)