Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-29847

MDEV-29847: Wrong warning on rlimit capping of max_open_files: was: open_files_limit can no longer be set explicitly through LimitNOFILE

    XMLWordPrintable

Details

    Description

      After upgrading from version 10.5.17 to version 10.6.10 the way open_files_limit is determined seems to have changed.
      Before the open_files_limit could be set as follows:

      1. joe /etc/sysctl.conf (set OS limit)

      fs.file-max=500000

      2. joe /etc/security/limits.conf (set mysql user limits)

      mysql soft nofile 300000
      mysql hard nofile 300000

      3. joe /etc/systemd/system/mariadb.service.d/limits.conf (effectively set open_files_limit)

      [Service]
      LimitNOFILE=250000

      Result:

      MariaDB [(none)]> show variables like '%open_files_limit%';
      ------------------------+

      Variable_name Value

      ------------------------+

      open_files_limit 250000

      ------------------------+

      After upgrading to version 10.6.10 all seemed well initially:

      MariaDB [(none)]> show variables like '%open_files_limit%';
      ------------------------+

      Variable_name Value

      ------------------------+

      open_files_limit 250000

      ------------------------+

      However, an error has now appeared in /var/log/messages on starting MariaDB:

      [Warning] Could not increase number of max_open_files to more than 250000 (request: 262327)

      Upon further investigation, the error disappears when changing the value of LimitNOFILE to a number larger than the requested 262327 mentioned in the warning:

      3. joe /etc/systemd/system/mariadb.service.d/limits.conf

      [Service]
      LimitNOFILE=300000

      However, the open_files_limit is now set to the aforementioned 262327 from the warning message in the log instead of the expected 250000:

      MariaDB [(none)]> show variables like '%open_files_limit%';
      ------------------------+

      Variable_name Value

      ------------------------+

      open_files_limit 262327

      ------------------------+

      This effective value for open_files_limit of 262327 seems to be based on the following settings in /etc/my.cnf:

      table_open_cache = 16384
      max_connections = 150

      Roughly following this equation:
      open_files_limit = table_open_cache * 16 + max_connections + (unclear remainder?)

      Or in this case:
      262327 = 16384 * 16 + 150 + (33?)

      When adding 1000 to table_open_cache in /etc/my.cnf as follows:

      table_open_cache = 17384
      max_connections = 150

      The result is:

      MariaDB [(none)]> show variables like '%open_files_limit%';
      ------------------------+

      Variable_name Value

      ------------------------+

      open_files_limit 278327

      ------------------------+

      Which is exactly 16000 more than the previous open_files_limit of 262327

      When subsequently adding 10 to max_connections in /etc/my.cnf as follows:

      table_open_cache = 17384
      max_connections = 160

      The result is:

      MariaDB [(none)]> show variables like '%open_files_limit%';
      ------------------------+

      Variable_name Value

      ------------------------+

      open_files_limit 278337

      ------------------------+

      Which is exactly 10 more than the previous open_files_limit of 278327

      Conclusion:

      Previously it was possible to explicitly specify open_files_limit through LimitNOFILE.

      In the new situation:

      1. If you specify a LimitNOFILE value smaller than 'table_open_cache * 16 + max_connections + (unclear remainder?)' the effective open_files_limit does equal the value for LimitNOFILE, but a warning is given in /var/log/messages that it could not be set to 'table_open_cache * 16 + max_connections + (unclear remainder?)'

      2. If you specify a LimitNOFILE value larger than 'table_open_cache * 16 + max_connections + (unclear remainder?)' the effective open_files_limit does not equal the value for LimitNOFILE, but equals 'table_open_cache * 16 + max_connections + (unclear remainder?)'

      Attachments

        Activity

          People

            TheLinuxJedi Andrew Hutchings (Inactive)
            ejdejongh E.J. de Jongh
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.