[MXS-487] lost connection to backend server Created: 2015-11-24 Updated: 2016-02-10 Resolved: 2016-02-10 |
|
| Status: | Closed |
| Project: | MariaDB MaxScale |
| Component/s: | readwritesplit |
| Affects Version/s: | 1.2.1 |
| Fix Version/s: | 1.3.0 |
| Type: | Bug | Priority: | Minor |
| Reporter: | Stephane Q. | Assignee: | markus makela |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Environment: |
centos 6.7 2.6.32-573.el6.x86_64, mariadb 5.5.41-1.el6 (yum repo) / maxscale 1.2.1 |
||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Description |
|
Hello, I'm using MaxScale 1.21 with MariaDB 5.5 in a readwritesplit environement, and I notice that every 5 or 10 min (it's random), I received errors from our php application: it is very similar to what is described by someone else here: the maxadmin show session is showing several 'invalid state'. Strange thing, is that if I activate the log_trace, the number of error jumps a lot: instead of 3 or 4 each hour, I have one error every minute. We have maxscale configured in front of 4 differents group of mariadb servers, [maxscale] example for 1 of the 4 clusters [fetch] [hint] [VHDLISTENER] [RWVHD] (etc...) not sure if it is implied, but here is some of the sysctl parameter used on the servers: Could it be linked with some sort of timeouts with the keep alive connections ? As I said, the errors rate is increasing a lot if we activate the log_trace, so we tried to reduce the amount of request handled by the mascale server , we managed to divide it by 2, but the errors are still there. we have around 280 req/s on each nodes, expect one with 2500req/s. last point, we are using an old application, mysql/myisam , php and the old mysql extension. not sure if it is linked, maybe mysqli could help ? |
| Comments |
| Comment by Stephane Q. [ 2015-11-26 ] |
|
we made some modifications on our php application, here is what we have found: when we connect to a server, the mysql_connect works, then immediately after, we have a mysql_select_db and this is where we get the 'lost connection to backend server'. If we receive that error, we wait 0.5s and we try again the mysql_connect, mysql_select_db, and since 12h: no more errors. now, we also have the same error on a mysql_query. so we only tried this: if the mysql_query fails with a 'lost connection to backend server', we wait 0.5s and try again the same query, with the same db handle: no more errors since 12h . so it looks like the connection between the php application and maxscale works well, but for some reason, maxscale is losing the connection to the backend servers and manage to reopen it alone, but it sends an error to the clients. |
| Comment by markus makela [ 2015-11-26 ] |
|
A test with the attached test script and data doesn't seem to yield any results. I'll continue testing with different environment. |
| Comment by markus makela [ 2015-12-31 ] |
|
Can you retest this with the 1.3.0-beta version of MaxScale? The binaries can be found here: http://maxscale-jenkins.mariadb.com/ci-repository/1.3.0-beta-debug/mariadb-maxscale/ |
| Comment by Stephane Q. [ 2016-01-06 ] |
|
ok I'll give it a try in the coming days, I will let you know |
| Comment by markus makela [ 2016-02-04 ] |
|
Stephane Any update on this issue? |
| Comment by Stephane Q. [ 2016-02-04 ] |
|
very sorry for the delay, got extra work to do , I couldn't try it earlier. |
| Comment by Stephane Q. [ 2016-02-10 ] |
|
it has been running since 5 days now, and everything looks much more stable. I don't see the lost connections problem anymore... |
| Comment by markus makela [ 2016-02-10 ] |
|
I'm closing this as fixed in 1.3.0. If it happens again, please reopen this issue. |