[MDEV-10835] Denial Of Service - Crash Any Galera Node - Race Condition: CREATE TABLE IF NOT EXISTS / INSERT Created: 2016-09-19 Updated: 2016-09-29 Resolved: 2016-09-29 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Galera |
| Affects Version/s: | 10.1.17 |
| Fix Version/s: | 10.1.18 |
| Type: | Bug | Priority: | Critical |
| Reporter: | Rob Brown | Assignee: | Nirbhay Choubey (Inactive) |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | None | ||
| Environment: |
Occurs on CentOS 6 and CentOS 7 64-bit |
||
| Issue Links: |
|
||||||||
| Description |
|
WARNING! This will crash any Galera Node 100% of the time. It's very easy to replicate this Race Condition vulnerability. |
| Comments |
| Comment by Rob Brown [ 2016-09-19 ] |
|
Here are the steps to duplicate: 1. Setup galera cluster with at least two nodes. For example, let's say "node1" and "node2". 2. On node1, run this command: [root@node1 ~]# perl -e 'for (1..100) { print qq {CREATE TABLE IF NOT EXISTS test.foo (id INT AUTO_INCREMENT, p INT, PRIMARY KEY (id));\n}; }' | mysql -f 3. While that "CREATE" grinder is running on node1, quickly run this on node2: [root@node2 ~]# perl -e 'for (1..100) { print qq {INSERT INTO test.foo VALUES (NULL, $$)\n;}; }' | mysql -f BOOM!!! And node2 will surely crash. |
| Comment by Rob Brown [ 2016-09-19 ] |
|
The mysqld.log on node2 suddenly shows the following message: 160919 16:41:39 [ERROR] mysqld got signal 11 ; To report this bug, see https://mariadb.com/kb/en/reporting-bugs We will try our best to scrape up some info that will hopefully help Server version: 10.1.17-MariaDB Thread pointer: 0x0x7f683236a008 Trying to get some variables. Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains |
| Comment by Rob Brown [ 2016-09-19 ] |
|
The node receiving the "REPLACE INTO" or "INSERT INTO" command from a MySQL client is the only node that crashes. The node directly receiving the "CREATE TABLE" command from a MySQL client never crashes even though the mysqld.log file on the crashing node always mentions the "CREATE TABLE" query (that it must have received via sync communication from the other node). |
| Comment by Rob Brown [ 2016-09-19 ] |
|
The only reason I suspect this is only pertinent to Galera is because when the nodes are configured using normal master ring, (where each node is a slave of the other instead of in a galera cluster), then none of the nodes will crash, regardless of which one receives the CREATE TABLE and which one receives the INSERT. Both servers will stay alive and strong. |