Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Not a Bug
-
10.1.38, 10.2.23, 10.3.14
-
None
Description
galera_new_cluster bootstraps a new cluster. However, when this happens, the cluster keeps its old state. For example:
[ec2-user@ip-172-30-0-249 ~]$ sudo galera_new_cluster
|
[ec2-user@ip-172-30-0-249 ~]$ sudo mysql -u root
|
Welcome to the MariaDB monitor. Commands end with ; or \g.
|
Your MariaDB connection id is 8
|
Server version: 10.1.38-MariaDB MariaDB Server
|
|
|
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
|
|
|
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
|
|
|
MariaDB [(none)]> SHOW GLOBAL VARIABLES WHERE Variable_name IN('wsrep_start_position');
|
+----------------------+----------------------------------------+
|
| Variable_name | Value |
|
+----------------------+----------------------------------------+
|
| wsrep_start_position | 0d1774c1-2ce4-11e9-9071-fe16986f98a1:6 |
|
+----------------------+----------------------------------------+
|
1 row in set (0.00 sec)
|
|
|
MariaDB [(none)]> SHOW GLOBAL STATUS WHERE Variable_name IN('wsrep_cluster_state_uuid', 'wsrep_last_committed');
|
+--------------------------+--------------------------------------+
|
| Variable_name | Value |
|
+--------------------------+--------------------------------------+
|
| wsrep_cluster_state_uuid | 0d1774c1-2ce4-11e9-9071-fe16986f98a1 |
|
| wsrep_last_committed | 6 |
|
+--------------------------+--------------------------------------+
|
2 rows in set (0.00 sec)
|
|
|
MariaDB [(none)]> \q
|
Bye
|
[ec2-user@ip-172-30-0-249 ~]$ sudo systemctl stop mariadb
|
[ec2-user@ip-172-30-0-249 ~]$ sudo galera_new_cluster
|
[ec2-user@ip-172-30-0-249 ~]$ sudo mysql -u root
|
Welcome to the MariaDB monitor. Commands end with ; or \g.
|
Your MariaDB connection id is 8
|
Server version: 10.1.38-MariaDB MariaDB Server
|
|
|
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
|
|
|
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
|
|
|
MariaDB [(none)]> SHOW GLOBAL VARIABLES WHERE Variable_name IN('wsrep_start_position');
|
+----------------------+----------------------------------------+
|
| Variable_name | Value |
|
+----------------------+----------------------------------------+
|
| wsrep_start_position | 0d1774c1-2ce4-11e9-9071-fe16986f98a1:6 |
|
+----------------------+----------------------------------------+
|
1 row in set (0.00 sec)
|
|
|
MariaDB [(none)]> SHOW GLOBAL STATUS WHERE Variable_name IN('wsrep_cluster_state_uuid', 'wsrep_last_committed');
|
+--------------------------+--------------------------------------+
|
| Variable_name | Value |
|
+--------------------------+--------------------------------------+
|
| wsrep_cluster_state_uuid | 0d1774c1-2ce4-11e9-9071-fe16986f98a1 |
|
| wsrep_last_committed | 6 |
|
+--------------------------+--------------------------------------+
|
2 rows in set (0.00 sec)
|
|
|
MariaDB [(none)]> \q
|
Bye
|
If it's a "new" cluster, wouldn't it make sense to start with a new UUID and seqno?
galera_new_cluster starts "mysqld --wsrep-new-cluster" using systemd under the hood. I don't believe that this is a systemd or galera_new_cluster problem, because this still happens if you bypass systemd and start "mysqld --wsrep-new-cluster" directly. For example:
[ec2-user@ip-172-30-0-249 ~]$ sudo mysqld --user=mysql --wsrep-new-cluster &
|
[1] 4753
|
2019-05-02 17:16:42 140682328529088 [Note] mysqld (mysqld 10.1.39-MariaDB) starting as process 4754 ...
|
[ec2-user@ip-172-30-0-249 ~]$ sudo mysql -u root
|
Welcome to the MariaDB monitor. Commands end with ; or \g.
|
Your MariaDB connection id is 8
|
Server version: 10.1.39-MariaDB MariaDB Server
|
|
|
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
|
|
|
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
|
|
|
MariaDB [(none)]> SHOW GLOBAL VARIABLES WHERE Variable_name IN('wsrep_start_position');
|
+----------------------+-----------------------------------------+
|
| Variable_name | Value |
|
+----------------------+-----------------------------------------+
|
| wsrep_start_position | 00000000-0000-0000-0000-000000000000:-1 |
|
+----------------------+-----------------------------------------+
|
1 row in set (0.00 sec)
|
|
|
MariaDB [(none)]> SHOW GLOBAL STATUS WHERE Variable_name IN('wsrep_cluster_state_uuid', 'wsrep_last_committed');
|
+--------------------------+--------------------------------------+
|
| Variable_name | Value |
|
+--------------------------+--------------------------------------+
|
| wsrep_cluster_state_uuid | 22f991fd-6d1f-11e9-ac6a-b392983e8673 |
|
| wsrep_last_committed | 0 |
|
+--------------------------+--------------------------------------+
|
2 rows in set (0.01 sec)
|
|
|
MariaDB [(none)]> \q
|
Bye
|
[ec2-user@ip-172-30-0-249 ~]$ sudo mysqladmin -u root shutdown
|
[1]+ Done sudo mysqld -u mysql --wsrep-new-cluster
|
[ec2-user@ip-172-30-0-249 ~]$ sudo mysqld --user=mysql --wsrep-new-cluster &
|
[1] 4799
|
2019-05-02 17:18:15 140331403352256 [Note] mysqld (mysqld 10.1.39-MariaDB) starting as process 4800 ...
|
[ec2-user@ip-172-30-0-249 ~]$ sudo mysql -u root
|
Welcome to the MariaDB monitor. Commands end with ; or \g.
|
Your MariaDB connection id is 8
|
Server version: 10.1.39-MariaDB MariaDB Server
|
|
|
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
|
|
|
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
|
|
|
MariaDB [(none)]> SHOW GLOBAL VARIABLES WHERE Variable_name IN('wsrep_start_position');
|
+----------------------+-----------------------------------------+
|
| Variable_name | Value |
|
+----------------------+-----------------------------------------+
|
| wsrep_start_position | 00000000-0000-0000-0000-000000000000:-1 |
|
+----------------------+-----------------------------------------+
|
1 row in set (0.00 sec)
|
|
|
MariaDB [(none)]> SHOW GLOBAL STATUS WHERE Variable_name IN('wsrep_cluster_state_uuid', 'wsrep_last_committed');
|
+--------------------------+--------------------------------------+
|
| Variable_name | Value |
|
+--------------------------+--------------------------------------+
|
| wsrep_cluster_state_uuid | 22f991fd-6d1f-11e9-ac6a-b392983e8673 |
|
| wsrep_last_committed | 0 |
|
+--------------------------+--------------------------------------+
|
2 rows in set (0.01 sec)
|
|
|
MariaDB [(none)]> \q
|
Bye
|
[ec2-user@ip-172-30-0-249 ~]$ sudo mysqladmin -u root shutdown
|
[1]+ Done sudo mysqld -u mysql --wsrep-new-cluster
|
In order to reset 'wsrep_cluster_state_uuid' and 'wsrep_last_committed', you have to manually delete grastate.dat:
[ec2-user@ip-172-30-0-249 ~]$ sudo galera_new_cluster
|
[ec2-user@ip-172-30-0-249 ~]$ sudo mysql -u root
|
Welcome to the MariaDB monitor. Commands end with ; or \g.
|
Your MariaDB connection id is 9
|
Server version: 10.1.38-MariaDB MariaDB Server
|
|
|
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
|
|
|
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
|
|
|
MariaDB [(none)]> SHOW GLOBAL VARIABLES WHERE Variable_name IN('wsrep_start_position');
|
+----------------------+----------------------------------------+
|
| Variable_name | Value |
|
+----------------------+----------------------------------------+
|
| wsrep_start_position | 0d1774c1-2ce4-11e9-9071-fe16986f98a1:6 |
|
+----------------------+----------------------------------------+
|
1 row in set (0.00 sec)
|
|
|
MariaDB [(none)]> SHOW GLOBAL STATUS WHERE Variable_name IN('wsrep_cluster_state_uuid', 'wsrep_last_committed');
|
+--------------------------+--------------------------------------+
|
| Variable_name | Value |
|
+--------------------------+--------------------------------------+
|
| wsrep_cluster_state_uuid | 0d1774c1-2ce4-11e9-9071-fe16986f98a1 |
|
| wsrep_last_committed | 6 |
|
+--------------------------+--------------------------------------+
|
2 rows in set (0.00 sec)
|
|
|
MariaDB [(none)]> \q
|
Bye
|
[ec2-user@ip-172-30-0-249 ~]$ sudo systemctl stop mariadb
|
[ec2-user@ip-172-30-0-249 ~]$ sudo rm /var/lib/mysql/grastate.dat
|
[ec2-user@ip-172-30-0-249 ~]$ sudo galera_new_cluster
|
[ec2-user@ip-172-30-0-249 ~]$ sudo mysql -u root
|
Welcome to the MariaDB monitor. Commands end with ; or \g.
|
Your MariaDB connection id is 8
|
Server version: 10.1.38-MariaDB MariaDB Server
|
|
|
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
|
|
|
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
|
|
|
MariaDB [(none)]> SHOW GLOBAL VARIABLES WHERE Variable_name IN('wsrep_start_position');
|
+----------------------+----------------------------------------+
|
| Variable_name | Value |
|
+----------------------+----------------------------------------+
|
| wsrep_start_position | 0d1774c1-2ce4-11e9-9071-fe16986f98a1:6 |
|
+----------------------+----------------------------------------+
|
1 row in set (0.00 sec)
|
|
|
MariaDB [(none)]> SHOW GLOBAL STATUS WHERE Variable_name IN('wsrep_cluster_state_uuid', 'wsrep_last_committed');
|
+--------------------------+--------------------------------------+
|
| Variable_name | Value |
|
+--------------------------+--------------------------------------+
|
| wsrep_cluster_state_uuid | 1ff475b4-57d2-11e9-80ac-077936e51cdd |
|
| wsrep_last_committed | 0 |
|
+--------------------------+--------------------------------------+
|
2 rows in set (0.00 sec)
|
|
|
MariaDB [(none)]> \q
|
Bye
|
You can see from above that wsrep_start_position is still not reset when mysqld is started with systemd. This is because MariaDB's systemd unit file has an ExecStartPre option defined that recovers this position using the galera_recovery wrapper script:
https://github.com/MariaDB/server/blob/mariadb-10.1.39/support-files/mariadb.service.in#L72
https://github.com/MariaDB/server/blob/mariadb-10.1.39/scripts/galera_recovery.sh
Attachments
Issue Links
- relates to
-
-
- Closed
-
Activity
I do not think that MDEV-17458, which was a fix for a regression introduced by MDEV-15158, can possibly fix this. This bug is reported to affect the 10.1 and 10.2 series, and the MDEV-17458 fix only applies to 10.3 and later.
sciascid Can you please check 10.1 and 10.2 case and is there still something needed for 10.3->10.4
GeoffMontee This is not a bug.
Option --wsrep-new-cluster simply signals galera that the starting node is alone in the cluster, and will bootstrap.
This option is used in two cases:
1) When you create a cluster from scratch, and a new UUID is allocated for it
2) When restarting a cluster that already existed, which has a UUID already assigned
What you are doing in your examples above, falls in the second case: you are basically restarting the first node of your cluster. And because the cluster existed, its old state is preserved.
There currently is no option to reset galera state. As you mentioned, you can manually delete the old state in the datadir.
Let me know if that clarifies.
Thanks, sciascid. I expected this was the case. I think the new cluster in the --wsrep-new-cluster option name and the galera_new_cluster script name is a bit of a misnomer then. In hindsight, they probably should have been called something like --wsrep-bootstrap-cluster and galera_boostrap_cluster. But it probably doesn't matter.
https://github.com/MariaDB/server/pull/1305 should fix this also.