[MDEV-18410] Can't set mysqld_safe's open-files-limit to unlimited Created: 2019-01-29  Updated: 2023-04-27

Status: Open
Project: MariaDB Server
Component/s: Scripts & Clients
Affects Version/s: 10.1.37, 10.2.21, 10.3.12
Fix Version/s: 10.4

Type: Bug Priority: Minor
Reporter: Geoff Montee (Inactive) Assignee: Anel Husakovic
Resolution: Unresolved Votes: 0
Labels: foundation

Issue Links:
Relates
relates to MDEV-18420 Increase OS_FILE_LIMIT to ULLONG_MAX Open

 Description   

The value provided for mysqld_safe's open-files-limit option gets passed right to "ulimit" here:

  if test -n "$open_files"
  then
    ulimit -n $open_files
  fi

https://github.com/MariaDB/server/blob/c2318291be7458238729ed80233ea71f1e6a62b8/scripts/mysqld_safe.sh#L709

The "ulimit" command accepts the special value of "unlimited" to represent no limit:

https://ss64.com/bash/ulimit.html

So it would make sense to be able to do the following:

[mysqld_safe]
open_files_limit=unlimited

However, this doesn't currently work because mysqld_safe appends its own value of open-files-limit to the list of arguments for mysqld here:

if test -n "$open_files"
then
  append_arg_to_args "--open-files-limit=$open_files"
fi

https://github.com/MariaDB/server/blob/c2318291be7458238729ed80233ea71f1e6a62b8/scripts/mysqld_safe.sh#L715

And since mysqld expects a numeric value for this option, it causes an error:

190128 23:27:41 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
Unknown suffix 'u' used for variable 'open_files_limit' (value 'unlimited')
2019-01-28 23:27:41 139683855239360 [ERROR] //sbin/mysqld: Error while setting value 'unlimited' to 'open_files_limit'

https://mariadb.com/kb/en/library/server-system-variables/#open_files_limit

Maybe this should be changed, so that:

  • If open-files-limit=unlimited, mysqld_safe does not pass the value to mysqld.
    or
  • If open-files-limit=unlimited, mysqld_safe passes open_files_limit=4294967295 to mysqld instead.


 Comments   
Comment by Elena Stepanova [ 2019-01-29 ]

Does it make sense, realistically?
To make it actually work, the user should have the hard limit set to unlimited. I don't know if it happens, I've never seen it so far. But even if it does, would someone really want to allow such access for mysqld?

Comment by Geoff Montee (Inactive) [ 2019-01-29 ]

To make it actually work, the user should have the hard limit set to unlimited. I don't know if it happens, I've never seen it so far.

mysqld_safe executes "ulimit -n $open_files". The manual says that this does set both the hard and soft limit.

When setting new limits, if neither '-H' nor '-S' is supplied, both the hard and soft limits are set.

https://ss64.com/bash/ulimit.html

But even if it does, would someone really want to allow such access for mysqld?

I think that it's reasonable to assume that MariaDB can manage file descriptors well enough, and that it's reasonable to set the open file limit to unlimited to guarantee that MariaDB can't run out of file descriptors.

Anyway, it's probably pointless because it looks like the server itself won't allow an infinite limit right now. It sets the its own limit with setrlimit:

https://github.com/MariaDB/server/blob/5abc79dd7ab2fccb4b05ca38a512ec816d2f8e52/mysys/my_file.c#L42

And it sets a maximum limit of OS_FILE_LIMIT on that limit:

https://github.com/MariaDB/server/blob/5abc79dd7ab2fccb4b05ca38a512ec816d2f8e52/mysys/my_file.c#L102

And OS_FILE_LIMIT is defined as UINT_MAX (4294967295):

https://github.com/MariaDB/server/blob/8dc460b844dcb8a8ef70396bfaf932010076b9a3/include/my_global.h#L695

Whereas RLIMIT_INFINITY seems to be defined as ULLONG_MAX on my system:

$ grep "RLIM_INFINITY" /usr/include/bits/resource.h
# define RLIM_INFINITY ((__rlim_t) -1)
# define RLIM_INFINITY 0xffffffffffffffffuLL
#define RLIM_SAVED_MAX  RLIM_INFINITY
#define RLIM_SAVED_CUR  RLIM_INFINITY

So OS_FILE_LIMIT and the maximum value of the server's open_files_limit variable would probably have to be increased to ULLONG_MAX to really set the limit to infinite:

https://github.com/MariaDB/server/blob/78829a5780d3e48da376c1ca4e62731da450c551/sql/sys_vars.cc#L2434

But it probably doesn't matter. open_files_limit=4294967295 might be effectively "unlimited"

Comment by Elena Stepanova [ 2019-01-29 ]

mysqld_safe executes "ulimit -n $open_files". The manual says that this does set both the hard and soft limit.

Okay, sure, if we are only talking about running mysqld_safe as root, then it can increase the hard limit.
However, regardless what mysqld or MariaDB tools do with the value, I don't know on which systems open_files=unlimited is actually allowed and where it's not, at least my Debian Jessie doesn't allow ulimit -n unlimited even for root. Maximum it accepts is 1048576

root@ws:~# ulimit -n 1048576
root@ws:~# ulimit -n 1048577
-su: ulimit: open files: cannot modify limit: Operation not permitted
root@ws:~# ulimit -n unlimited
-su: ulimit: open files: cannot modify limit: Operation not permitted
root@ws:~# ulimit -n
1048576

For how MariaDB server handles the issue, cvicentiu just recently was fixing something in the area, maybe he can comment on the latest status.

Generated at Thu Feb 08 08:43:53 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.