Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Won't Fix
-
10.3.6
-
None
-
Galera Master-Master 2 Nodes ;CentOS 7.4.
Description
Data inconsistency is observed on the cluster after waiting for table metadata lock occurred on Galera Node
the wsrep status does not show any indication about the occurred inconsistency ,
cluster state is given operational and Synced;
On Node 1 occurred waiting for table metadata lock
After that next received DDL operations on Node1 stuck .
On Node 2 however are observed completed operations , Node1 and Node2
return different results .
Below on Node1 table CUSTOMER returns results while on Node2
table CUSTOMER does not exists.
Node2
|
|
MariaDB [(none)]> select count(*) from tpcc.CUSTOMER ;
|
ERROR 1146 (42S02): Table 'tpcc.CUSTOMER' doesn't exist
|
|
Node1
MariaDB [(none)]> select count(*) from tpcc.CUSTOMER ;
|
+----------+
|
| count(*) |
|
+----------+
|
| 30000 |
|
+----------+
|
1 row in set (0.030 sec)
|
|
|
|
MariaDB [(none)]> show processlist ;
|
+----+-------------+-----------------+------+------------+-------+-------------------------------------------+-------------------------------------------+----------+
|
| Id | User | Host | db | Command | Time | State | Info | Progress |
|
+----+-------------+-----------------+------+------------+-------+-------------------------------------------+-------------------------------------------+----------+
|
| 1 | system user | | NULL | Sleep | 70918 | Committed 26860 | NULL | 0.000 |
|
| 2 | system user | | NULL | Sleep | 86903 | WSREP aborter idle | NULL | 0.000 |
|
| 3 | system user | | NULL | Daemon | NULL | InnoDB purge worker | NULL | 0.000 |
|
| 4 | system user | | NULL | Daemon | NULL | InnoDB purge worker | NULL | 0.000 |
|
| 5 | system user | | NULL | Daemon | NULL | InnoDB purge coordinator | NULL | 0.000 |
|
| 6 | system user | | NULL | Daemon | NULL | InnoDB purge worker | NULL | 0.000 |
|
| 7 | system user | | NULL | Daemon | NULL | InnoDB shutdown handler | NULL | 0.000 |
|
| 13 | root | localhost | cctt | Killed | 62724 | Init | CREATE OR REPLACE TABLE t1 AS SELECT f1() | 0.000 |
|
| 14 | root | localhost | cctt | Field List | 5423 | Waiting for table metadata lock | | 0.000 |
|
| 25 | root | localhost | NULL | Query | 690 | Waiting for stored function metadata lock | drop function cctt.f1 | 0.000 |
|
| 27 | root | localhost:40292 | tpcc | Query | 209 | Checking permissions | DROP TABLE IF EXISTS CUSTOMER | 0.000 |
|
| 28 | root | localhost | NULL | Query | 0 | Init | show processlist | 0.000 |
|
+----+-------------+-----------------+------+------------+-------+-------------------------------------------+-------------------------------------------+----------+
|
12 rows in set (0.000 sec)
|
|
|
|
\
MariaDB [(none)]> show status like '%wsrep%' ;
|
+------------------------------+-------------------------------------------+
|
| Variable_name | Value |
|
+------------------------------+-------------------------------------------+
|
| wsrep_apply_oooe | 0.016129 |
|
| wsrep_apply_oool | 0.000000 |
|
| wsrep_apply_window | 1.024194 |
|
| wsrep_causal_reads | 0 |
|
| wsrep_cert_deps_distance | 1.661290 |
|
| wsrep_cert_index_size | 34 |
|
| wsrep_cert_interval | 0.024194 |
|
| wsrep_cluster_conf_id | 88 |
|
| wsrep_cluster_size | 2 |
|
| wsrep_cluster_state_uuid | 3c15149f-5766-11e8-9a99-22bc53d40581 |
|
| wsrep_cluster_status | Primary |
|
| wsrep_commit_oooe | 0.000000 |
|
| wsrep_commit_oool | 0.000000 |
|
| wsrep_commit_window | 1.000000 |
|
| wsrep_connected | ON |
|
| wsrep_desync_count | 0 |
|
| wsrep_evs_delayed | |
|
| wsrep_evs_evict_list | |
|
| wsrep_evs_repl_latency | 0/0/0/0/0 |
|
| wsrep_evs_state | OPERATIONAL |
|
| wsrep_flow_control_paused | 0.000000 |
|
| wsrep_flow_control_paused_ns | 0 |
|
| wsrep_flow_control_recv | 0 |
|
| wsrep_flow_control_sent | 0 |
|
| wsrep_gcomm_uuid | d09be1b9-58f0-11e8-a6c7-de268e185d14 |
|
| wsrep_incoming_addresses | 192.168.104.195:3306,192.168.104.196:3306 |
|
| wsrep_last_committed | 26892 |
|
| wsrep_local_bf_aborts | 0 |
|
| wsrep_local_cached_downto | 26772 |
|
| wsrep_local_cert_failures | 0 |
|
| wsrep_local_commits | 55 |
|
| wsrep_local_index | 1 |
|
| wsrep_local_recv_queue | 0 |
|
| wsrep_local_recv_queue_avg | 0.004484 |
|
| wsrep_local_recv_queue_max | 2 |
|
| wsrep_local_recv_queue_min | 0 |
|
| wsrep_local_replays | 0 |
|
| wsrep_local_send_queue | 0 |
|
| wsrep_local_send_queue_avg | 0.000000 |
|
| wsrep_local_send_queue_max | 1 |
|
| wsrep_local_send_queue_min | 0 |
|
| wsrep_local_state | 4 |
|
| wsrep_local_state_comment | Synced |
|
| wsrep_local_state_uuid | 3c15149f-5766-11e8-9a99-22bc53d40581 |
|
| wsrep_protocol_version | 7 |
|
| wsrep_provider_name | Galera |
|
| wsrep_provider_vendor | Codership Oy <info@codership.com> |
|
| wsrep_provider_version | 25.3.23(r3789) |
|
| wsrep_ready | ON |
|
| wsrep_received | 223 |
|
| wsrep_received_bytes | 20873 |
|
| wsrep_repl_data_bytes | 36695 |
|
| wsrep_repl_keys | 280 |
|
| wsrep_repl_keys_bytes | 4633 |
|
| wsrep_repl_other_bytes | 0 |
|
| wsrep_replicated | 119 |
|
| wsrep_replicated_bytes | 48944 |
|
| wsrep_thread_count | 2 |
|
+------------------------------+-------------------------------------------+
|
|
It appears consecutively that Node1 hangs on 'drop' and 'create' statements
at 'Checking permissions' state while Node2 is processing all statements .
Node 1
|
MariaDB [(none)]> create database eee
|
-> ;
|
use eee;
|
Command hangs
|
Node 1 processlist
|
28 root localhost NULL Query 21258 Checking permissions create database eee
|
Node 1
|
MariaDB [(none)]> show databases like '%eee%' ;
|
Empty set (0.001 sec)
|
|
|
|
|
|
|
Node 2
|
MariaDB [(none)]> show databases like '%eee%' ;
|
+------------------+
|
| Database (%eee%) |
|
+------------------+
|
| eee |
|
+------------------+
|
1 row in set (0.000 sec)
|
|
|
MariaDB [(none)]>
|
|
Attachments
Issue Links
- relates to
-
-
- Closed
-