[MXS-419] Socket creation failed due 24, Too many open files. Created: 2015-10-21  Updated: 2016-05-09  Resolved: 2016-05-09

Status: Closed
Project: MariaDB MaxScale
Component/s: readconnroute
Affects Version/s: 1.2.1
Fix Version/s: 2.0.0

Type: Bug Priority: Minor
Reporter: Alex Vladulescu Assignee: markus makela
Resolution: Fixed Votes: 1
Labels: None
Environment:

Debian 7.9, VM under KVM, 8 vCPUs/8GB RAM (designed sql LB type) - deb package involved : maxscale-1.2.1-1.deb_wheezy.x86_64.deb



 Description   

Hi,

This might be a stupid thread but I'll take a shoot on it.

I have the max scale production environment described into MXS-415 ticket.
Under high load over database connections (many frontend servers connecting to the database Loadbalancer) - regularity is 1 per day - I get the maxscale error log filled with :

2015-10-21 16:53:03 Error : Failed to create Read Connection Router session.
2015-10-21 16:53:03 Error : Failed to create Read Connection Router session.
2015-10-21 16:53:03 Error: Establishing connection to backend server 10.200.100.153:3306 failed.
Socket creation failed due 24, Too many open files.
2015-10-21 16:53:03 Error: Establishing connection to backend server 10.200.100.152:3306 failed.
Socket creation failed due 24, Too many open files.
2015-10-21 16:53:03 Error : Failed to create Read Connection Router session.
2015-10-21 16:53:03 Error: Establishing connection to backend server 10.200.100.151:3306 failed.
Socket creation failed due 24, Too many open files.
2015-10-21 16:53:03 Error : Failed to create Read Connection Router session.

Now, I have set before and let it run for one day into /etc/security/limits.conf:

  • soft nofile 65535
  • hard nofile 65535

after which I have restarted the maxscale daemon to pick-up the change.

Question is (didn't do it but I am going to test now as a last resort) should there be included into the init script before daemon start-up the ulimit -HSn 65535 value by default in the standard rpm/deb build the ulimit setting ?

I suppose other folks are using maxsclae in deployments where the load is quite big.

Thanks,
Alex

 Comments   
Comment by Henning Granz [ 2015-10-26 ]

Same Problem on Ubuntu 14.04 LTS with 4 CPUs and 16GB RAM on VMWARE.

nlimit -n shows 100000 but maxscale stalls at 1087 open file handles.

Comment by Alex Vladulescu [ 2015-10-26 ]

Hi,

Put ulimit -HSn 65535 into the init script of maxscale, just before the env variables initialization and adapt your limits.conf for soft and hard limits.

To check if new settings have been taken after a maxscale restart, just get the pid of the new process from PID path location and check out cat /proc/PID_OF_MAXSCALE/limits
You'll see if daemon took the new imposed vars

Comment by markus makela [ 2016-05-09 ]

Added ulimit calls to init scripts.

Generated at Thu Feb 08 03:59:08 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.