[MDEV-8967] If wsrep_on=OFF on a node, there might still be some Galera Cluster communications - Jira

XML

Word

Printable

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Not a Bug
Affects Version/s: 10.0.21-galera
Fix Version/s: 10.0.22-galera
Component/s: Documentation, Galera
Labels:
- galera

Description

It seems that if you set wsrep_on=OFF, some Galera Cluster communications can still happen.

To demonstrate this, let's start with a 2-node cluster.

MariaDB [(none)]> show global status where Variable_name IN('wsrep_cluster_size', 'wsrep_cluster_status', 'wsrep_incoming_addresses');

+--------------------------+-------------------------------------+

| Variable_name            | Value                               |

+--------------------------+-------------------------------------+

| wsrep_incoming_addresses | 172.31.25.43:3306,172.31.25.44:3306 |

| wsrep_cluster_size       | 2                                   |

| wsrep_cluster_status     | Primary                             |

+--------------------------+-------------------------------------+

3 rows in set (0.00 sec)

Then let's kill the mysqld process on one node:

sudo kill -9 $(sudo pidof mysqld)

This will cause the cluster to become a 1-node cluster, and it will be non-Primary, assuming that the nodes use the default values of most wsrep_provider_option settings.

MariaDB [(none)]> show global status where Variable_name IN('wsrep_cluster_size', 'wsrep_cluster_status', 'wsrep_incoming_addresses');

+--------------------------+-------------------+

| Variable_name            | Value             |

+--------------------------+-------------------+

| wsrep_incoming_addresses | 172.31.25.44:3306 |

| wsrep_cluster_size       | 1                 |

| wsrep_cluster_status     | non-Primary       |

+--------------------------+-------------------+

3 rows in set (0.00 sec)

On the remaining node, let's set:

SET GLOBAL wsrep_on=OFF

Now, let's bring the killed node back online:

sudo service mysql start

Even though wsrep_on=OFF, a cluster does seem to get formed:

MariaDB [(none)]> show global status where Variable_name IN('wsrep_cluster_size', 'wsrep_cluster_status', 'wsrep_incoming_addresses');

+--------------------------+-------------------------------------+

| Variable_name            | Value                               |

+--------------------------+-------------------------------------+

| wsrep_incoming_addresses | 172.31.25.43:3306,172.31.25.44:3306 |

| wsrep_cluster_size       | 2                                   |

| wsrep_cluster_status     | Primary                             |

+--------------------------+-------------------------------------+

3 rows in set (0.00 sec)

The log contents for the node with wsrep_on=OFF, are:

151019 12:51:09 mysqld_safe mysqld from pid file /var/lib/mysql/ip-172-31-25-44.us-west-2.compute.internal.pid ended

151019 12:51:12 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql

151019 12:51:12 mysqld_safe WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.GXlNPF' --pid-file='/var/lib/mysql/ip-172-31-25-44.us-west-2.compute.internal-recover.pid'

151019 12:51:12 [Note] /usr/sbin/mysqld (mysqld 10.0.21-MariaDB-wsrep) starting as process 2659 ...

151019 12:51:15 mysqld_safe WSREP: Recovered position 0c761b78-767e-11e5-9a39-3be96e837b66:3

151019 12:51:15 [Note] /usr/sbin/mysqld (mysqld 10.0.21-MariaDB-wsrep) starting as process 2694 ...

151019 12:51:15 [Note] WSREP: Read nil XID from storage engines, skipping position init

151019 12:51:15 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera/libgalera_smm.so'

151019 12:51:15 [Note] WSREP: wsrep_load(): Galera 25.3.9(r3387) by Codership Oy <info@codership.com> loaded successfully.

151019 12:51:15 [Note] WSREP: CRC-32C: using hardware acceleration.

151019 12:51:15 [Note] WSREP: Found saved state: 0c761b78-767e-11e5-9a39-3be96e837b66:3

151019 12:51:15 [Note] WSREP: Passing config to GCS: base_host = 172.31.25.44; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 128M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.recover

151019 12:51:15 [Note] WSREP: Service thread queue flushed.

151019 12:51:15 [Note] WSREP: Assign initial position for certification: 3, protocol version: -1

151019 12:51:15 [Note] WSREP: wsrep_sst_grab()

151019 12:51:15 [Note] WSREP: Start replication

151019 12:51:15 [Note] WSREP: 'wsrep-new-cluster' option used, bootstrapping the cluster

151019 12:51:15 [Note] WSREP: Setting initial position to 0c761b78-767e-11e5-9a39-3be96e837b66:3

151019 12:51:15 [Note] WSREP: protonet asio version 0

151019 12:51:15 [Note] WSREP: Using CRC-32C for message checksums.

151019 12:51:15 [Note] WSREP: backend: asio

151019 12:51:15 [Warning] WSREP: access file(gvwstate.dat) failed(No such file or directory)

151019 12:51:15 [Note] WSREP: restore pc from disk failed

151019 12:51:15 [Note] WSREP: GMCast version 0

151019 12:51:15 [Note] WSREP: (9c6a5002, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567

151019 12:51:15 [Note] WSREP: (9c6a5002, 'tcp://0.0.0.0:4567') multicast: , ttl: 1

151019 12:51:15 [Note] WSREP: EVS version 0

151019 12:51:15 [Note] WSREP: gcomm: bootstrapping new group 'my_wsrep_cluster'

151019 12:51:15 [Note] WSREP: start_prim is enabled, turn off pc_recovery

151019 12:51:15 [Note] WSREP: Node 9c6a5002 state prim

151019 12:51:15 [Note] WSREP: view(view_id(PRIM,9c6a5002,1) memb {

        9c6a5002,0

} joined {

} left {

} partitioned {

})

151019 12:51:15 [Note] WSREP: save pc into disk

151019 12:51:15 [Note] WSREP: discarding pending addr without UUID: tcp://172.31.25.42:4567

151019 12:51:15 [Note] WSREP: discarding pending addr proto entry 0x7f284a4d82c0

151019 12:51:15 [Note] WSREP: discarding pending addr without UUID: tcp://172.31.25.43:4567

151019 12:51:15 [Note] WSREP: discarding pending addr proto entry 0x7f284a4d8500

151019 12:51:15 [Note] WSREP: discarding pending addr without UUID: tcp://172.31.25.44:4567

151019 12:51:15 [Note] WSREP: discarding pending addr proto entry 0x7f284a4d85c0

151019 12:51:15 [Note] WSREP: gcomm: connected

151019 12:51:15 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636

151019 12:51:15 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)

151019 12:51:15 [Note] WSREP: Opened channel 'my_wsrep_cluster'

151019 12:51:15 [Note] WSREP: Waiting for SST to complete.

151019 12:51:15 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1

151019 12:51:15 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 9c6ac139-7681-11e5-a736-1b0dd4380317

151019 12:51:15 [Note] WSREP: STATE EXCHANGE: sent state msg: 9c6ac139-7681-11e5-a736-1b0dd4380317

151019 12:51:15 [Note] WSREP: STATE EXCHANGE: got state msg: 9c6ac139-7681-11e5-a736-1b0dd4380317 from 0 (ip-172-31-25-44.us-west-2.compute.internal)

151019 12:51:15 [Note] WSREP: Quorum results:

        version    = 3,

        component  = PRIMARY,

        conf_id    = 0,

        members    = 1/1 (joined/total),

        act_id     = 3,

        last_appl. = -1,

        protocols  = 0/7/3 (gcs/repl/appl),

        group UUID = 0c761b78-767e-11e5-9a39-3be96e837b66

151019 12:51:15 [Note] WSREP: Flow-control interval: [16, 16]

151019 12:51:15 [Note] WSREP: Restored state OPEN -> JOINED (3)

151019 12:51:15 [Note] WSREP: Member 0.0 (ip-172-31-25-44.us-west-2.compute.internal) synced with group.

151019 12:51:15 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 3)

151019 12:51:15 [Note] WSREP: New cluster view: global state: 0c761b78-767e-11e5-9a39-3be96e837b66:3, view# 1: Primary, number of nodes: 1, my index: 0, protocol version 3

151019 12:51:15 [Note] WSREP: SST complete, seqno: 3

151019 12:51:15 [Note] InnoDB: Using mutexes to ref count buffer pool pages

151019 12:51:15 [Note] InnoDB: The InnoDB memory heap is disabled

151019 12:51:15 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins

151019 12:51:15 [Note] InnoDB: Memory barrier is not used

151019 12:51:15 [Note] InnoDB: Compressed tables use zlib 1.2.7

151019 12:51:15 [Note] InnoDB: Using Linux native AIO

151019 12:51:15 [Note] InnoDB: Using CPU crc32 instructions

151019 12:51:15 [Note] InnoDB: Initializing buffer pool, size = 128.0M

151019 12:51:15 [Note] InnoDB: Completed initialization of buffer pool

151019 12:51:15 [Note] InnoDB: Highest supported file format is Barracuda.

151019 12:51:15 [Note] InnoDB: 128 rollback segment(s) are active.

151019 12:51:15 [Note] InnoDB: Waiting for purge to start

151019 12:51:15 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.25-73.1 started; log sequence number 1622291

151019 12:51:15 [Note] Plugin 'FEEDBACK' is disabled.

151019 12:51:15 [Note] Server socket created on IP: '::'.

151019 12:51:15 [Note] Event Scheduler: Loaded 0 events

151019 12:51:15 [Note] /usr/sbin/mysqld: ready for connections.

Version: '10.0.21-MariaDB-wsrep'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MariaDB Server, wsrep_25.10.r4144

151019 12:51:15 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

151019 12:51:15 [Note] WSREP: REPL Protocols: 7 (3, 2)

151019 12:51:15 [Note] WSREP: Service thread queue flushed.

151019 12:51:15 [Note] WSREP: Assign initial position for certification: 3, protocol version: 3

151019 12:51:15 [Note] WSREP: Service thread queue flushed.

151019 12:51:15 [Note] WSREP: Synchronized with group, ready for connections

151019 12:51:15 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

151019 12:51:30 [Note] WSREP: (9c6a5002, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers:

151019 12:51:30 [Note] WSREP: declaring 2a4ff567 at tcp://172.31.25.43:4567 stable

151019 12:51:30 [Note] WSREP: Node 9c6a5002 state prim

151019 12:51:30 [Note] WSREP: view(view_id(PRIM,2a4ff567,9) memb {

        2a4ff567,0

        9c6a5002,0

} joined {

} left {

} partitioned {

})

151019 12:51:30 [Note] WSREP: save pc into disk

151019 12:51:30 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2

151019 12:51:30 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.

151019 12:51:31 [Note] WSREP: STATE EXCHANGE: sent state msg: a5fa0a0f-7681-11e5-8d08-0e195450fcbe

151019 12:51:31 [Note] WSREP: STATE EXCHANGE: got state msg: a5fa0a0f-7681-11e5-8d08-0e195450fcbe from 0 (ip-172-31-25-43.us-west-2.compute.internal)

151019 12:51:31 [Note] WSREP: STATE EXCHANGE: got state msg: a5fa0a0f-7681-11e5-8d08-0e195450fcbe from 1 (ip-172-31-25-44.us-west-2.compute.internal)

151019 12:51:31 [Note] WSREP: Quorum results:

        version    = 3,

        component  = PRIMARY,

        conf_id    = 1,

        members    = 2/2 (joined/total),

        act_id     = 3,

        last_appl. = 0,

        protocols  = 0/7/3 (gcs/repl/appl),

        group UUID = 0c761b78-767e-11e5-9a39-3be96e837b66

151019 12:51:31 [Note] WSREP: Flow-control interval: [23, 23]

151019 12:51:31 [Note] WSREP: New cluster view: global state: 0c761b78-767e-11e5-9a39-3be96e837b66:3, view# 2: Primary, number of nodes: 2, my index: 1, protocol version 3

151019 12:51:31 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

151019 12:51:31 [Note] WSREP: REPL Protocols: 7 (3, 2)

151019 12:51:31 [Note] WSREP: Service thread queue flushed.

151019 12:51:31 [Note] WSREP: Assign initial position for certification: 3, protocol version: 3

151019 12:51:31 [Note] WSREP: Service thread queue flushed.

151019 12:51:31 [Note] WSREP: Member 0.0 (ip-172-31-25-43.us-west-2.compute.internal) synced with group.

151019 12:51:33 [Note] WSREP: (9c6a5002, 'tcp://0.0.0.0:4567') turning message relay requesting off

151019 12:53:12 [Note] WSREP: (9c6a5002, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://172.31.25.43:4567

151019 12:53:13 [Note] WSREP: (9c6a5002, 'tcp://0.0.0.0:4567') reconnecting to 2a4ff567 (tcp://172.31.25.43:4567), attempt 0

151019 12:53:16 [Note] WSREP: evs::proto(9c6a5002, OPERATIONAL, view_id(REG,2a4ff567,9)) suspecting node: 2a4ff567

151019 12:53:16 [Note] WSREP: evs::proto(9c6a5002, OPERATIONAL, view_id(REG,2a4ff567,9)) suspected node without join message, declaring inactive

151019 12:53:17 [Note] WSREP: view(view_id(NON_PRIM,2a4ff567,9) memb {

        9c6a5002,0

} joined {

} left {

} partitioned {

        2a4ff567,0

})

151019 12:53:17 [Note] WSREP: view(view_id(NON_PRIM,9c6a5002,10) memb {

        9c6a5002,0

} joined {

} left {

} partitioned {

        2a4ff567,0

})

151019 12:53:17 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1

151019 12:53:17 [Note] WSREP: Flow-control interval: [16, 16]

151019 12:53:17 [Note] WSREP: Received NON-PRIMARY.

151019 12:53:17 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 3)

151019 12:53:17 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1

151019 12:53:17 [Note] WSREP: Flow-control interval: [16, 16]

151019 12:53:17 [Note] WSREP: Received NON-PRIMARY.

151019 12:53:17 [Note] WSREP: New cluster view: global state: 0c761b78-767e-11e5-9a39-3be96e837b66:3, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version 3

151019 12:53:17 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

151019 12:53:17 [Note] WSREP: New cluster view: global state: 0c761b78-767e-11e5-9a39-3be96e837b66:3, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version 3

151019 12:53:17 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

151019 12:53:58 [Note] WSREP: (9c6a5002, 'tcp://0.0.0.0:4567') reconnecting to 2a4ff567 (tcp://172.31.25.43:4567), attempt 30

151019 12:54:43 [Note] WSREP: (9c6a5002, 'tcp://0.0.0.0:4567') reconnecting to 2a4ff567 (tcp://172.31.25.43:4567), attempt 60

151019 12:54:55 [Note] WSREP: declaring 2a4ff567 at tcp://172.31.25.43:4567 stable

151019 12:54:55 [Note] WSREP: re-bootstrapping prim from partitioned components

151019 12:54:55 [Note] WSREP: view(view_id(PRIM,2a4ff567,11) memb {

        2a4ff567,0

        9c6a5002,0

} joined {

} left {

} partitioned {

})

151019 12:54:55 [Note] WSREP: save pc into disk

151019 12:54:55 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2

151019 12:54:55 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.

151019 12:54:56 [Note] WSREP: STATE EXCHANGE: sent state msg: 20212a96-7682-11e5-abbd-4e0226d29d1c

151019 12:54:56 [Note] WSREP: STATE EXCHANGE: got state msg: 20212a96-7682-11e5-abbd-4e0226d29d1c from 0 (ip-172-31-25-43.us-west-2.compute.internal)

151019 12:54:56 [Note] WSREP: STATE EXCHANGE: got state msg: 20212a96-7682-11e5-abbd-4e0226d29d1c from 1 (ip-172-31-25-44.us-west-2.compute.internal)

151019 12:54:56 [Warning] WSREP: Quorum: No node with complete state:

        Version      : 3

        Flags        : 0x1

        Protocols    : 0 / 7 / 3

        State        : NON-PRIMARY

        Prim state   : NON-PRIMARY

        Prim UUID    : 00000000-0000-0000-0000-000000000000

        Prim  seqno  : -1

        First seqno  : -1

        Last  seqno  : 3

        Prim JOINED  : 0

        State UUID   : 20212a96-7682-11e5-abbd-4e0226d29d1c

        Group UUID   : 0c761b78-767e-11e5-9a39-3be96e837b66

        Name         : 'ip-172-31-25-43.us-west-2.compute.internal'

        Incoming addr: '172.31.25.43:3306'

        Version      : 3

        Flags        : 0x2

        Protocols    : 0 / 7 / 3

        State        : NON-PRIMARY

        Prim state   : SYNCED

        Prim UUID    : a5fa0a0f-7681-11e5-8d08-0e195450fcbe

        Prim  seqno  : 2

        First seqno  : -1

        Last  seqno  : 3

        Prim JOINED  : 2

        State UUID   : 20212a96-7682-11e5-abbd-4e0226d29d1c

        Group UUID   : 0c761b78-767e-11e5-9a39-3be96e837b66

        Name         : 'ip-172-31-25-44.us-west-2.compute.internal'

        Incoming addr: '172.31.25.44:3306'

151019 12:54:56 [Note] WSREP: Partial re-merge of primary a5fa0a0f-7681-11e5-8d08-0e195450fcbe found: 1 of 2.

151019 12:54:56 [Note] WSREP: Quorum results:

        version    = 3,

        component  = PRIMARY,

        conf_id    = 2,

        members    = 2/2 (joined/total),

        act_id     = 3,

        last_appl. = 0,

        protocols  = 0/7/3 (gcs/repl/appl),

        group UUID = 0c761b78-767e-11e5-9a39-3be96e837b66

151019 12:54:56 [Note] WSREP: Flow-control interval: [23, 23]

151019 12:54:56 [Note] WSREP: Restored state OPEN -> SYNCED (3)

151019 12:54:56 [Note] WSREP: New cluster view: global state: 0c761b78-767e-11e5-9a39-3be96e837b66:3, view# 3: Primary, number of nodes: 2, my index: 1, protocol version 3

151019 12:54:56 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

151019 12:54:56 [Note] WSREP: REPL Protocols: 7 (3, 2)

151019 12:54:56 [Note] WSREP: Service thread queue flushed.

151019 12:54:56 [Note] WSREP: Assign initial position for certification: 3, protocol version: 3

151019 12:54:56 [Note] WSREP: Service thread queue flushed.

151019 12:54:56 [Note] WSREP: Synchronized with group, ready for connections

151019 12:54:56 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

151019 12:54:56 [Note] WSREP: Member 0.0 (ip-172-31-25-43.us-west-2.compute.internal) synced with group.

151019 12:54:58 [Note] WSREP: (9c6a5002, 'tcp://0.0.0.0:4567') turning message relay requesting off

The log contents for the second node are:

151019 12:51:27 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql

151019 12:51:27 mysqld_safe WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.8OcDPr' --pid-file='/var/lib/mysql/ip-172-31-25-43.us-west-2.compute.internal-recover.pid'

151019 12:51:27 [Note] /usr/sbin/mysqld (mysqld 10.0.21-MariaDB-wsrep) starting as process 5713 ...

151019 12:51:30 mysqld_safe WSREP: Recovered position 0c761b78-767e-11e5-9a39-3be96e837b66:3

151019 12:51:30 [Note] /usr/sbin/mysqld (mysqld 10.0.21-MariaDB-wsrep) starting as process 5748 ...

151019 12:51:30 [Note] WSREP: Read nil XID from storage engines, skipping position init

151019 12:51:30 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera/libgalera_smm.so'

151019 12:51:30 [Note] WSREP: wsrep_load(): Galera 25.3.9(r3387) by Codership Oy <info@codership.com> loaded successfully.

151019 12:51:30 [Note] WSREP: CRC-32C: using hardware acceleration.

151019 12:51:30 [Note] WSREP: Found saved state: 0c761b78-767e-11e5-9a39-3be96e837b66:3

151019 12:51:30 [Note] WSREP: Passing config to GCS: base_host = 172.31.25.43; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 128M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.recover

151019 12:51:30 [Note] WSREP: Service thread queue flushed.

151019 12:51:30 [Note] WSREP: Assign initial position for certification: 3, protocol version: -1

151019 12:51:30 [Note] WSREP: wsrep_sst_grab()

151019 12:51:30 [Note] WSREP: Start replication

151019 12:51:30 [Note] WSREP: Setting initial position to 0c761b78-767e-11e5-9a39-3be96e837b66:3

151019 12:51:30 [Note] WSREP: protonet asio version 0

151019 12:51:30 [Note] WSREP: Using CRC-32C for message checksums.

151019 12:51:30 [Note] WSREP: backend: asio

151019 12:51:30 [Note] WSREP: restore pc from disk successfully

151019 12:51:30 [Note] WSREP: GMCast version 0

151019 12:51:30 [Note] WSREP: (2a4ff567, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567

151019 12:51:30 [Note] WSREP: (2a4ff567, 'tcp://0.0.0.0:4567') multicast: , ttl: 1

151019 12:51:30 [Note] WSREP: EVS version 0

151019 12:51:30 [Note] WSREP: gcomm: connecting to group 'my_wsrep_cluster', peer '172.31.25.44:,172.31.25.43:,172.31.25.42:'

151019 12:51:30 [Warning] WSREP: (2a4ff567, 'tcp://0.0.0.0:4567') address 'tcp://172.31.25.43:4567' points to own listening address, blacklisting

151019 12:51:30 [Note] WSREP: (2a4ff567, 'tcp://0.0.0.0:4567') address 'tcp://172.31.25.43:4567' pointing to uuid 2a4ff567 is blacklisted, skipping

151019 12:51:30 [Note] WSREP: (2a4ff567, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers:

151019 12:51:30 [Note] WSREP: declaring 9c6a5002 at tcp://172.31.25.44:4567 stable

151019 12:51:30 [Note] WSREP: Node 9c6a5002 state prim

151019 12:51:30 [Note] WSREP: view(view_id(PRIM,2a4ff567,9) memb {

        2a4ff567,0

        9c6a5002,0

} joined {

} left {

} partitioned {

})

151019 12:51:30 [Note] WSREP: save pc into disk

151019 12:51:30 [Note] WSREP: discarding pending addr without UUID: tcp://172.31.25.42:4567

151019 12:51:30 [Note] WSREP: clear restored view

151019 12:51:31 [Note] WSREP: gcomm: connected

151019 12:51:31 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636

151019 12:51:31 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)

151019 12:51:31 [Note] WSREP: Opened channel 'my_wsrep_cluster'

151019 12:51:31 [Note] WSREP: Waiting for SST to complete.

151019 12:51:31 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2

151019 12:51:31 [Note] WSREP: STATE_EXCHANGE: sent state UUID: a5fa0a0f-7681-11e5-8d08-0e195450fcbe

151019 12:51:31 [Note] WSREP: STATE EXCHANGE: sent state msg: a5fa0a0f-7681-11e5-8d08-0e195450fcbe

151019 12:51:31 [Note] WSREP: STATE EXCHANGE: got state msg: a5fa0a0f-7681-11e5-8d08-0e195450fcbe from 0 (ip-172-31-25-43.us-west-2.compute.internal)

151019 12:51:31 [Note] WSREP: STATE EXCHANGE: got state msg: a5fa0a0f-7681-11e5-8d08-0e195450fcbe from 1 (ip-172-31-25-44.us-west-2.compute.internal)

151019 12:51:31 [Note] WSREP: Quorum results:

        version    = 3,

        component  = PRIMARY,

        conf_id    = 1,

        members    = 2/2 (joined/total),

        act_id     = 3,

        last_appl. = -1,

        protocols  = 0/7/3 (gcs/repl/appl),

        group UUID = 0c761b78-767e-11e5-9a39-3be96e837b66

151019 12:51:31 [Note] WSREP: Flow-control interval: [23, 23]

151019 12:51:31 [Note] WSREP: Restored state OPEN -> JOINED (3)

151019 12:51:31 [Note] WSREP: New cluster view: global state: 0c761b78-767e-11e5-9a39-3be96e837b66:3, view# 2: Primary, number of nodes: 2, my index: 0, protocol version 3

151019 12:51:31 [Note] WSREP: SST complete, seqno: 3

151019 12:51:31 [Note] InnoDB: Using mutexes to ref count buffer pool pages

151019 12:51:31 [Note] InnoDB: The InnoDB memory heap is disabled

151019 12:51:31 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins

151019 12:51:31 [Note] InnoDB: Memory barrier is not used

151019 12:51:31 [Note] InnoDB: Compressed tables use zlib 1.2.7

151019 12:51:31 [Note] InnoDB: Using Linux native AIO

151019 12:51:31 [Note] InnoDB: Using CPU crc32 instructions

151019 12:51:31 [Note] InnoDB: Initializing buffer pool, size = 128.0M

151019 12:51:31 [Note] WSREP: Member 0.0 (ip-172-31-25-43.us-west-2.compute.internal) synced with group.

151019 12:51:31 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 3)

151019 12:51:31 [Note] InnoDB: Completed initialization of buffer pool

151019 12:51:31 [Note] InnoDB: Highest supported file format is Barracuda.

151019 12:51:31 [Note] InnoDB: 128 rollback segment(s) are active.

151019 12:51:31 [Note] InnoDB: Waiting for purge to start

151019 12:51:31 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.25-73.1 started; log sequence number 1622809

151019 12:51:31 [Note] Plugin 'FEEDBACK' is disabled.

151019 12:51:31 [Note] Server socket created on IP: '::'.

151019 12:51:31 [Note] Event Scheduler: Loaded 0 events

151019 12:51:31 [Note] /usr/sbin/mysqld: ready for connections.

Version: '10.0.21-MariaDB-wsrep'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MariaDB Server, wsrep_25.10.r4144

151019 12:51:31 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

151019 12:51:31 [Note] WSREP: REPL Protocols: 7 (3, 2)

151019 12:51:31 [Note] WSREP: Service thread queue flushed.

151019 12:51:31 [Note] WSREP: Assign initial position for certification: 3, protocol version: 3

151019 12:51:31 [Note] WSREP: Service thread queue flushed.

151019 12:51:31 [Note] WSREP: Synchronized with group, ready for connections

151019 12:51:31 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

151019 12:51:33 [Note] WSREP: (2a4ff567, 'tcp://0.0.0.0:4567') turning message relay requesting off

151019 12:53:12 mysqld_safe Number of processes running now: 0

151019 12:53:12 mysqld_safe WSREP: not restarting wsrep node automatically

151019 12:53:12 mysqld_safe mysqld from pid file /var/lib/mysql/ip-172-31-25-43.us-west-2.compute.internal.pid ended

151019 12:54:52 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql

151019 12:54:52 mysqld_safe WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.R2ZH5R' --pid-file='/var/lib/mysql/ip-172-31-25-43.us-west-2.compute.internal-recover.pid'

151019 12:54:52 [Note] /usr/sbin/mysqld (mysqld 10.0.21-MariaDB-wsrep) starting as process 5916 ...

151019 12:54:55 mysqld_safe WSREP: Recovered position 0c761b78-767e-11e5-9a39-3be96e837b66:3

151019 12:54:55 [Note] /usr/sbin/mysqld (mysqld 10.0.21-MariaDB-wsrep) starting as process 5952 ...

151019 12:54:55 [Note] WSREP: Read nil XID from storage engines, skipping position init

151019 12:54:55 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera/libgalera_smm.so'

151019 12:54:55 [Note] WSREP: wsrep_load(): Galera 25.3.9(r3387) by Codership Oy <info@codership.com> loaded successfully.

151019 12:54:55 [Note] WSREP: CRC-32C: using hardware acceleration.

151019 12:54:55 [Note] WSREP: Found saved state: 0c761b78-767e-11e5-9a39-3be96e837b66:-1

151019 12:54:55 [Note] WSREP: Passing config to GCS: base_host = 172.31.25.43; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 128M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.recover

151019 12:54:55 [Note] WSREP: Service thread queue flushed.

151019 12:54:55 [Note] WSREP: Assign initial position for certification: 3, protocol version: -1

151019 12:54:55 [Note] WSREP: wsrep_sst_grab()

151019 12:54:55 [Note] WSREP: Start replication

151019 12:54:55 [Note] WSREP: Setting initial position to 0c761b78-767e-11e5-9a39-3be96e837b66:3

151019 12:54:55 [Note] WSREP: protonet asio version 0

151019 12:54:55 [Note] WSREP: Using CRC-32C for message checksums.

151019 12:54:55 [Note] WSREP: backend: asio

151019 12:54:55 [Note] WSREP: restore pc from disk successfully

151019 12:54:55 [Note] WSREP: GMCast version 0

151019 12:54:55 [Note] WSREP: (2a4ff567, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567

151019 12:54:55 [Note] WSREP: (2a4ff567, 'tcp://0.0.0.0:4567') multicast: , ttl: 1

151019 12:54:55 [Note] WSREP: EVS version 0

151019 12:54:55 [Note] WSREP: gcomm: connecting to group 'my_wsrep_cluster', peer '172.31.25.44:,172.31.25.43:,172.31.25.42:'

151019 12:54:55 [Warning] WSREP: (2a4ff567, 'tcp://0.0.0.0:4567') address 'tcp://172.31.25.43:4567' points to own listening address, blacklisting

151019 12:54:55 [Note] WSREP: (2a4ff567, 'tcp://0.0.0.0:4567') address 'tcp://172.31.25.43:4567' pointing to uuid 2a4ff567 is blacklisted, skipping

151019 12:54:55 [Note] WSREP: (2a4ff567, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers:

151019 12:54:55 [Note] WSREP: declaring 9c6a5002 at tcp://172.31.25.44:4567 stable

151019 12:54:55 [Note] WSREP: re-bootstrapping prim from partitioned components

151019 12:54:55 [Note] WSREP: view(view_id(PRIM,2a4ff567,11) memb {

        2a4ff567,0

        9c6a5002,0

} joined {

} left {

} partitioned {

})

151019 12:54:55 [Note] WSREP: save pc into disk

151019 12:54:55 [Note] WSREP: discarding pending addr without UUID: tcp://172.31.25.42:4567

151019 12:54:55 [Note] WSREP: clear restored view

151019 12:54:56 [Note] WSREP: gcomm: connected

151019 12:54:56 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636

151019 12:54:56 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)

151019 12:54:56 [Note] WSREP: Opened channel 'my_wsrep_cluster'

151019 12:54:56 [Note] WSREP: Waiting for SST to complete.

151019 12:54:56 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2

151019 12:54:56 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 20212a96-7682-11e5-abbd-4e0226d29d1c

151019 12:54:56 [Note] WSREP: STATE EXCHANGE: sent state msg: 20212a96-7682-11e5-abbd-4e0226d29d1c

151019 12:54:56 [Note] WSREP: STATE EXCHANGE: got state msg: 20212a96-7682-11e5-abbd-4e0226d29d1c from 0 (ip-172-31-25-43.us-west-2.compute.internal)

151019 12:54:56 [Note] WSREP: STATE EXCHANGE: got state msg: 20212a96-7682-11e5-abbd-4e0226d29d1c from 1 (ip-172-31-25-44.us-west-2.compute.internal)

151019 12:54:56 [Warning] WSREP: Quorum: No node with complete state:

        Version      : 3

        Flags        : 0x1

        Protocols    : 0 / 7 / 3

        State        : NON-PRIMARY

        Prim state   : NON-PRIMARY

        Prim UUID    : 00000000-0000-0000-0000-000000000000

        Prim  seqno  : -1

        First seqno  : -1

        Last  seqno  : 3

        Prim JOINED  : 0

        State UUID   : 20212a96-7682-11e5-abbd-4e0226d29d1c

        Group UUID   : 0c761b78-767e-11e5-9a39-3be96e837b66

        Name         : 'ip-172-31-25-43.us-west-2.compute.internal'

        Incoming addr: '172.31.25.43:3306'

        Version      : 3

        Flags        : 0x2

        Protocols    : 0 / 7 / 3

        State        : NON-PRIMARY

        Prim state   : SYNCED

        Prim UUID    : a5fa0a0f-7681-11e5-8d08-0e195450fcbe

        Prim  seqno  : 2

        First seqno  : -1

        Last  seqno  : 3

        Prim JOINED  : 2

        State UUID   : 20212a96-7682-11e5-abbd-4e0226d29d1c

        Group UUID   : 0c761b78-767e-11e5-9a39-3be96e837b66

        Name         : 'ip-172-31-25-44.us-west-2.compute.internal'

        Incoming addr: '172.31.25.44:3306'

151019 12:54:56 [Note] WSREP: Partial re-merge of primary a5fa0a0f-7681-11e5-8d08-0e195450fcbe found: 1 of 2.

151019 12:54:56 [Note] WSREP: Quorum results:

        version    = 3,

        component  = PRIMARY,

        conf_id    = 2,

        members    = 2/2 (joined/total),

        act_id     = 3,

        last_appl. = -1,

        protocols  = 0/7/3 (gcs/repl/appl),

        group UUID = 0c761b78-767e-11e5-9a39-3be96e837b66

151019 12:54:56 [Note] WSREP: Flow-control interval: [23, 23]

151019 12:54:56 [Note] WSREP: Restored state OPEN -> JOINED (3)

151019 12:54:56 [Note] WSREP: New cluster view: global state: 0c761b78-767e-11e5-9a39-3be96e837b66:3, view# 3: Primary, number of nodes: 2, my index: 0, protocol version 3

151019 12:54:56 [Note] WSREP: SST complete, seqno: 3

151019 12:54:56 [Note] InnoDB: Using mutexes to ref count buffer pool pages

151019 12:54:56 [Note] InnoDB: The InnoDB memory heap is disabled

151019 12:54:56 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins

151019 12:54:56 [Note] InnoDB: Memory barrier is not used

151019 12:54:56 [Note] InnoDB: Compressed tables use zlib 1.2.7

151019 12:54:56 [Note] InnoDB: Using Linux native AIO

151019 12:54:56 [Note] InnoDB: Using CPU crc32 instructions

151019 12:54:56 [Note] InnoDB: Initializing buffer pool, size = 128.0M

151019 12:54:56 [Note] WSREP: Member 0.0 (ip-172-31-25-43.us-west-2.compute.internal) synced with group.

151019 12:54:56 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 3)

151019 12:54:56 [Note] InnoDB: Completed initialization of buffer pool

151019 12:54:56 [Note] InnoDB: Highest supported file format is Barracuda.

151019 12:54:56 [Note] InnoDB: 128 rollback segment(s) are active.

151019 12:54:56 [Note] InnoDB: Waiting for purge to start

151019 12:54:56 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.25-73.1 started; log sequence number 1622983

151019 12:54:56 [Note] Plugin 'FEEDBACK' is disabled.

151019 12:54:56 [Note] Server socket created on IP: '::'.

151019 12:54:56 [Note] Event Scheduler: Loaded 0 events

151019 12:54:56 [Note] /usr/sbin/mysqld: ready for connections.

Version: '10.0.21-MariaDB-wsrep'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MariaDB Server, wsrep_25.10.r4144

151019 12:54:56 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

151019 12:54:56 [Note] WSREP: REPL Protocols: 7 (3, 2)

151019 12:54:56 [Note] WSREP: Service thread queue flushed.

151019 12:54:56 [Note] WSREP: Assign initial position for certification: 3, protocol version: 3

151019 12:54:56 [Note] WSREP: Service thread queue flushed.

151019 12:54:56 [Note] WSREP: Synchronized with group, ready for connections

151019 12:54:56 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

151019 12:54:58 [Note] WSREP: (2a4ff567, 'tcp://0.0.0.0:4567') turning message relay requesting off

Attachments

Issue Links

links to

Codership issue - If wsrep_on=OFF on a node, there might still be some Galera Cluster communications

Activity

People

Assignee:: Nirbhay Choubey (Inactive)

Reporter:: Geoff Montee (Inactive)

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 2015-10-19 20:05

Updated:: 2015-10-27 15:26

Resolved:: 2015-10-27 15:26

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.