[MXS-1120] Could not read password prompt from MaxScale. Created: 2017-02-02  Updated: 2017-03-14  Resolved: 2017-03-14

Status: Closed
Project: MariaDB MaxScale
Component/s: maxadmin
Affects Version/s: 2.0.3, 2.0.4
Fix Version/s: 2.0.5, 2.1.1

Type: Bug Priority: Major
Reporter: Wesley Schaft Assignee: Timofey Turenko
Resolution: Fixed Votes: 0
Labels: None
Environment:

CentOS 6.8
2.6.32-642.13.1.el6.x86_64


 Description   

We use maxadmin to monitor the status of our database servers.
For example, if server A is indeed being used as Master and all others are Slave and are Synced.

In MaxScale 1.4.3 this went well. But since we've upgraded to 2.0.3 (and 2.0.4 today), every now and then the output of maxadmin is: "Could not read password prompt from MaxScale."

And we get an alert from our monitoring system.

Our command (password removed):

maxadmin -p******** 'list servers'
 
Servers.
 
-------------------+-----------------+-------+-------------+--------------------
 
Server             | Address         | Port  | Connections | Status
 
-------------------+-----------------+-------+-------------+--------------------
 
db-03              | 192.168.120.74  |  3306 |          17 | Master, Synced, Running
 
db-04              | 192.168.120.95  |  3306 |           3 | Slave, Synced, Running
 
db-05              | 192.168.120.96  |  3306 |           3 | Slave, Synced, Running
 
db-06              | 192.168.120.97  |  3306 |           3 | Slave, Synced, Running
 
db-07              | 192.168.120.98  |  3306 |           3 | Slave, Synced, Running
 
db-08              | 192.168.120.99  |  3306 |           2 | Slave, Synced, Running
 
db-09              | 192.168.120.100 |  3306 |           3 | Slave, Synced, Running
 
-------------------+-----------------+-------+-------------+--------------------

It can be reproduced by running this command in a for loop:

for i in {1..100}; do maxadmin -p******** 'list servers' | egrep -v 'db|Server|^-'; done
 
Could not read password prompt from MaxScale.



 Comments   
Comment by Johan Wikman [ 2017-02-06 ]

Are you using maxadmin over the network or on the same host where MaxScale is running?

If on the same host, then could you please try using a domain socket instead of an inet socket in the communication between maxadmin and MaxScale.

https://github.com/mariadb-corporation/MaxScale/blob/2.0.4/Documentation/Reference/MaxAdmin.md#configuring-mariadb-maxscale-for-maxadmin

Comment by Wesley Schaft [ 2017-02-07 ]

We're using it on the same host where MaxScale is running.

I just changed the configuration to use a domain socket instead of an inet socket.
Running the same command (but without the -p********) now runs without any issues.

Comment by Wesley Schaft [ 2017-02-07 ]

Running the maxadmin command as root works like charm.

However, our monitoring user (zabbix) isn't authorized:

As user 'zabbix':

# maxadmin list servers
 
Could connect to MaxScale, but was not authorized.

As root:

# maxadmin
 
MaxScale> show users
 
# echo $?
 
141
 
# maxadmin
 
MaxScale> add user zabbix
 
# echo $?
 
141

Comment by Johan Wikman [ 2017-02-07 ]

Invoke maxadmin as root. Then:

MaxAdmin> enable account zabbix

After that, you should be able to use maxadmin as zabbix.

Comment by Wesley Schaft [ 2017-02-08 ]

The thing is, every command in maxadmin seems to fail and I'm getting my prompt back. Return code is 141, as you can see in my previous comment.

Comment by Wesley Schaft [ 2017-02-08 ]

This seems to work however:

# maxadmin enable account zabbix
 
The Linux user zabbix has successfully been enabled.

Comment by Wesley Schaft [ 2017-02-08 ]

I did some more testing and apparently maxadmin sometimes doesn't give any output.

Doing a for loop and expecting 70 as output (because of 7 running servers):

# for i in {1..10}; do maxadmin list servers | grep Running; done | wc -l
 
70
 
# for i in {1..10}; do maxadmin list servers | grep Running; done | wc -l
 
70
 
# for i in {1..10}; do maxadmin list servers | grep Running; done | wc -l
 
70
 
# for i in {1..10}; do maxadmin list servers | grep Running; done | wc -l
 
63

Comment by markus makela [ 2017-02-15 ]

This might be related to MXS-1123 if connection_timeout is used.

Comment by Wesley Schaft [ 2017-02-15 ]

I've changed connection_timeout=3600 to connection_timeout=8640000, but it didn't help.

Comment by markus makela [ 2017-02-15 ]

Have you added the connection_timeout=8640000 to all services?

Comment by Wesley Schaft [ 2017-02-15 ]

I've only changed it for the Splitter Service. That didn't help.
Now I've added connection_timeout=8640000 to the CLI service and that seems to work.

Running the for loop over and over again, the result is always the same.

Thank you markus.

Comment by Wesley Schaft [ 2017-02-15 ]

The maxadmin interface itself is also working now.
It's showing output instead of dropping me back to the Linux prompt:

# maxadmin
 
MaxScale> show users
 
Enabled Linux accounts (secure)    : zabbix
 
Created network accounts (insecure):
 
MaxScale>

Generated at Thu Feb 08 04:04:23 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.