[MXS-4750] dynamic_node_detection=false does not work with server hostname, needs IP Created: 2023-09-12  Updated: 2023-09-29  Resolved: 2023-09-29

Status: Closed
Project: MariaDB MaxScale
Component/s: xpandmon
Affects Version/s: 23.08.0
Fix Version/s: 23.08.2

Type: Bug Priority: Major
Reporter: Rahul Joshi (Inactive) Assignee: Johan Wikman
Resolution: Fixed Votes: 0
Labels: None

Sprint: MXS-SPRINT-191

 Description   

I was trying to send the loads only to some of nodes by using dynamic_node_detection=false.
It kept giving me "Access denied for user " error where as user has access and when using dynamic_node_detection=true, things work fine and loads go to all nodes.
Had to switch from using hostnames to IP in the server entries to get it working.
This is easily reproducible.

[maxscale]
 
#log_info=1
 
#logdir=/data/clustrix/log
 
threads=auto
 
 
 
######## Glutton karma050 #######
 
 
 
[vqc007c-karma050]
 
type=server
 
#address=10.2.15.127
 
address=karma050
 
port=3306
 
protocol=mariadbbackend
 
 
 
[vqc007c-karma058]
 
type=server
 
#address=10.2.15.147
 
address=karma058
 
port=3306
 
protocol=mariadbbackend
 
 
 
[vqc007c-karma053]
 
type=server
 
#address=10.2.15.142
 
address=karma053
 
port=3306
 
protocol=mariadbbackend
 
 
 
[Gluttonmon-vqc007c]
 
type=monitor
 
module=xpandmon
 
servers=vqc007c-karma050, vqc007c-karma058, vqc007c-karma053
 
user=maxscale
 
password=maxscale_pw
 
cluster_monitor_interval=10000ms
 
dynamic_node_detection=false
 
health_check_port=3581
 
 
 
[GluttonRCR-vqc007c]
 
type=service
 
router=readconnroute
 
user=maxscale
 
password=maxscale_pw
 
cluster=Gluttonmon-vqc007c
 
 
 
[GluttonRCR-Listener-vqc007c]
 
type=listener
 
service=GluttonRCR-vqc007c
 
protocol=MariaDBClient
 
port=4010
 



[root@vqc007c ~]# mysql -A -h karma172 -P 4010 -u maxscale -pmaxscale_pw
 
ERROR 1045 (28000): Access denied for user 'maxscale'@'10.2.14.189' (using password: YES)


[root@karma172 ~]# maxctrl list servers
 
┌──────────────────┬──────────┬──────┬─────────────┬───────┬──────┬────────────────────┐
 
│ Server           │ Address  │ Port │ Connections │ State │ GTID │ Monitor            │
 
├──────────────────┼──────────┼──────┼─────────────┼───────┼──────┼────────────────────┤
 
│ vqc007c-karma050 │ karma050 │ 3306 │ 0           │ Down  │      │ Gluttonmon-vqc007c │
 
├──────────────────┼──────────┼──────┼─────────────┼───────┼──────┼────────────────────┤
 
│ vqc007c-karma058 │ karma058 │ 3306 │ 0           │ Down  │      │ Gluttonmon-vqc007c │
 
├──────────────────┼──────────┼──────┼─────────────┼───────┼──────┼────────────────────┤
 
│ vqc007c-karma053 │ karma053 │ 3306 │ 0           │ Down  │      │ Gluttonmon-vqc007c │
 
└──────────────────┴──────────┴──────┴─────────────┴───────┴──────┴────────────────────┘

When I uncomment IP's and comment hostnames in server entries in the maxscale cnofig file, same user can access through maxscale:

[root@vqc007c ~]# mysql -A -h karma172 -P 4010 -u maxscale -pmaxscale_pw
 
Welcome to the MariaDB monitor.  Commands end with ; or \g.
 
Your MySQL connection id is 19808
 
Server version: 5.0.45-Xpand-23.09.1_beta2
 
 
 
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
 
 
 
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
 
 
MySQL [(none)]> select @@hostname;
 
+-------------+
 
| @@hostname  |
 
+-------------+
 
| 10.2.15.127 |
 
+-------------+
 
1 row in set (0.00 sec)
 
 
 
MySQL [(none)]> show grants for maxscale@"%";
 
+-----------------------------------------------------------------+
 
| Grants for maxscale@%                                           |
 
+-----------------------------------------------------------------+
 
| GRANT SELECT ON `system`.`user_acl` TO 'maxscale'@'%'           |
 
| GRANT SELECT ON `system`.`users` TO 'maxscale'@'%'              |
 
| GRANT SELECT ON `system`.`softfailed_nodes` TO 'maxscale'@'%'   |
 
| GRANT SELECT ON `system`.`nodeinfo` TO 'maxscale'@'%'           |
 
| GRANT SELECT ON `system`.`membership` TO 'maxscale'@'%'         |
 
| GRANT ALL PRIVILEGES ON *.* TO 'maxscale'@'%' WITH GRANT OPTION |
 
+-----------------------------------------------------------------+
 
6 rows in set (0.01 sec)
 



Cluster Name:    gluttonvqc007c
 
Cluster Version: 23.09.1_beta2
 
Cluster Status:   OK
 
Cluster Size:    13 nodes - 16 CPUs per Node
 
Current Node:    karma158 - nid 6
 
 
 
nid | Hostname | Status | Substate |  IP Address |  Zone  |            Region            | TPS |      Used      | Total
 
----+----------+--------+----------+-------------+--------+------------------------------+-----+----------------+-------
 
  1 | karma050 |    OK  |   normal | 10.2.15.127 | zone-1 |   primary_region (PRIMARY)   |   8 |  28.0G (3.47%) | 807.9G
 
  2 | karma062 |    OK  |   normal | 10.2.14.113 | zone-1 |   primary_region (PRIMARY)   |   0 |  27.5G (3.41%) | 807.9G
 
  3 | karma181 |    OK  |   normal | 10.2.15.181 | zone-2 |   primary_region (PRIMARY)   |   0 |  27.8G (3.44%) | 807.9G
 
  4 | karma128 |    OK  |   normal | 10.2.12.238 | zone-2 |   primary_region (PRIMARY)   |   0 |  26.3G (3.25%) | 807.9G
 
  6 | karma158 |    OK  |   normal | 10.2.13.177 | zone-3 |   primary_region (PRIMARY)   |   0 |  26.4G (3.26%) | 807.9G
 
  7 | karma067 |    OK  |   normal | 10.2.14.123 | zone-4 | secondary_region (SECONDARY) |   0 |  22.3G (2.76%) | 807.9G
 
  8 | karma058 |    OK  |   normal | 10.2.15.147 | zone-4 | secondary_region (SECONDARY) |   1 |  22.7G (2.81%) | 807.9G
 
  9 | karma061 |    OK  |   normal | 10.2.14.111 | zone-5 | secondary_region (SECONDARY) |   0 |  22.4G (2.77%) | 807.9G
 
 10 | karma053 |    OK  |   normal | 10.2.15.142 | zone-5 | secondary_region (SECONDARY) |   0 |  22.1G (2.74%) | 807.9G
 
 11 | karma082 |    OK  |   normal | 10.2.12.226 | zone-6 | secondary_region (SECONDARY) |   0 |  22.4G (2.78%) | 807.9G
 
 12 | karma066 |    OK  |   normal | 10.2.14.121 | zone-6 | secondary_region (SECONDARY) |   0 |  23.1G (2.86%) | 807.9G
 
 14 | karma118 |    OK  |   normal |  10.2.15.67 | zone-7 |  observer_region (OBSERVER)  |   0 |  21.3M (0.00%) | 807.9G
 
 15 | karma164 |    OK  |   normal | 10.2.13.230 | zone-7 |  observer_region (OBSERVER)  |   0 |  22.8M (0.00%) | 807.9G
 
----+----------+--------+----------+-------------+--------+------------------------------+-----+----------------+-------
 
                                                                                             9 | 271.0G (2.58%) |  10.3T


[root@karma172 ~]# maxscale --version-full
 
MaxScale 6.4.10 - e21972b152407ccc41a5c835c91774578c64c472
 
CMake flags:   -DBUILD_TESTS=N  -DPACKAGE=Y -DDISTRIB_SUFFIX=rhel.7
 
 
 
and
 
 
 
[root@karma173 ~]# maxscale --version-full
 
MaxScale 23.02.3 - 2752b88a7d0037698da500c5f008464ffa704b15
 
CMake flags:   -DBUILD_TESTS=N  -DPACKAGE=Y -DDISTRIB_SUFFIX=rhel.7
 
 
 
and
 
maxctrl show maxscale | head -5 ;
 
┌──────────────┬──────────────────────────────────────────────────────────────┐
 
│ Version      │ 23.08.0                                                      │
 
├──────────────┼──────────────────────────────────────────────────────────────┤
 
│ Commit       │ 7c4f5b1525585dfe0abd4d30723774c110fa436d                     │
 
├──────────────┼──────────────────────────────────────────────────────────────┤



 Comments   
Comment by Johan Wikman [ 2023-09-27 ]

Could reproduce and the reason for the access denied, is that all servers were down. In the log there is probably an entry like

 2023-09-27 13:35:39   error  : No valid servers from which to query MariaDB user accounts found.

It seems that it is the health-check ping that does not work whith dynamic_node_detection=false and hostnames.

Comment by Johan Wikman [ 2023-09-28 ]

The health-check ping does work, but the information was not properly propagated when hostnames were used.

Generated at Thu Feb 08 04:30:53 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.