[MXS-1854] monitor backend database' mysql connect is closed by mxs once in a while Created: 2018-05-04  Updated: 2018-06-21  Resolved: 2018-06-21

Status: Closed
Project: MariaDB MaxScale
Component/s: readwritesplit
Affects Version/s: 2.2
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: xiangzhong Assignee: Unassigned
Resolution: Not a Bug Votes: 0
Labels: None
Environment:

linux + make debug + mysql 5.6.16


Attachments: PNG File image-2018-05-04-14-34-34-426.png    

 Description   

monitor config like:

[maxscale]
 
threads=1
 
log_info=1
 
log_debug=1
 
 
 
[RWMySQLMonitor]
 
type=monitor
 
module=mysqlmon
 
servers=server_m,server_s
 
monitor_interval=10000
 
 
 
[server_m]
 
type=server
 
address=10.125.198.61
 
port=3009
 
protocol=MySQLBackend
 
[server_s]
 
type=server
 
address=10.125.194.99
 
port=3009
 
protocol=MySQLBackend

we add log find closed connect always server_s connect

My title

warning: [mariadbmon] (do_show_slave_status): exec query failed:SHOW SLAVE STATUS, mysql:0x1fc4260
error : (mon_report_query_error): Failed to execute query on server 'server_s' ([10.125.194.99]:3009): Lost connection to MySQL server during query
notice : (mon_log_state_change): Server changed state: server_s[10.125.194.99:3009]: lost_slave. [Slave, Running] -> [Running]

we add the first line log in do_show_slave_status

    else
 
    {
 
        MXS_WARNING("exec query failed:%s, mysql:%p", query, database->con);
 
        mon_report_query_error(database);
 
    }

bt

My title

(gdb) bt
#0 mon_report_query_error (db=0x1fc23c0) at /home/xiangzhong.wxd/gitlab/MaxScale/server/core/monitor.cc:1680
#1 0x00007f038a953934 in do_show_slave_status (mon=0x1fc3280, serv_info=0x1fc49e0, database=0x1fc23c0)
at /home/xiangzhong.wxd/gitlab/MaxScale/server/modules/monitor/mariadbmon/mariadbmon.cc:1530
#2 0x00007f038a953b3f in monitor_mysql_db (mon=0x1fc3280, database=0x1fc23c0, serv_info=0x1fc49e0)
at /home/xiangzhong.wxd/gitlab/MaxScale/server/modules/monitor/mariadbmon/mariadbmon.cc:1578
#3 0x00007f038a95433c in monitorDatabase (mon=0x1fc0e60, database=0x1fc23c0) at /home/xiangzhong.wxd/gitlab/MaxScale/server/modules/monitor/mariadbmon/mariadbmon.cc:1798
#4 0x00007f038a95524e in monitorMain (arg=0x1fc3280) at /home/xiangzhong.wxd/gitlab/MaxScale/server/modules/monitor/mariadbmon/mariadbmon.cc:2216
#5 0x00007f038f2c1aa1 in start_thread () from /lib64/libpthread.so.0
#6 0x00007f038d48c93d in clone () from /lib64/libc.so.6

we cap packet find, the connect is closed by mxs:

maxscale IP: 10.101.161.10.40112
server_s IP:10.125.194.99.3009

 Comments   
Comment by markus makela [ 2018-05-04 ]

Can you try if adding query_retries=2 into the [maxscale] section helps?

Here is an example configuration:

[maxscale]
 
query_retries=2

Comment by xiangzhong [ 2018-05-04 ]

my question is who close this connect, and no any info.

Comment by markus makela [ 2018-05-04 ]

The query is done via the MariaDB Connector/C so the connector would have closed it.

Comment by dapeng huang [ 2018-05-07 ]

It is driver's read timeout

Comment by xiangzhong [ 2018-05-08 ]

yes, @depeng have confirmed: exec "show slave status" excess 1s lead to this problem. but i think this log:
"error : (mon_report_query_error): Failed to execute query on server 'server_s' ([10.125.194.99]:3009): Lost connection to MySQL server during query" is very unreasonable and did not explain the real reason。

Comment by markus makela [ 2018-06-21 ]

Closing as Not a Bug since this is the expected behavior for a network error. Increasing network timeouts or retry amounts should fix it.

Generated at Thu Feb 08 04:09:53 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.