Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-18027

Running out of file descriptors and eventual crash

Details

    Description

      Server generates errors and eventually crashes due to execeeding limit on number of open file descriptors.

      This occurs when additional open_table_caches_instances are created. The calculation for open_files_limit does not account for the fact that there may be multiple instances.

      I expect (but have not proven) problem could be avoided adjusting setting in config file to limit number of open_table_caches_instances or increasing open_files_limit. Currently neither of these are set in our config.

      Attachments

        Issue Links

          mentioned in

          Page Failed to load

          Page Failed to load

          Activity

            Ascending order - Click to sort in descending order
            View 7 older comments
            svoj Sergey Vojtovich added a comment -

            My concern if it really should go to 10.2 because next complain from support will be that user upgraded and now number of connections and table cache decreased because there is no enough file handlers...

            It wouldn't be the case if it were implemented as I suggested May 15: Don't let table cache instances number go up if soft limit cannot be raised.

            svoj Sergey Vojtovich added a comment - My concern if it really should go to 10.2 because next complain from support will be that user upgraded and now number of connections and table cache decreased because there is no enough file handlers... It wouldn't be the case if it were implemented as I suggested May 15: Don't let table cache instances number go up if soft limit cannot be raised.
            serg Sergei Golubchik added a comment -

            I think svoj suggested a better fix than fb27ed99a79f7e9b6c4e838d8a788a4685cfbee4.

            New cache instances are created, when the contention is too high, which normally means there is some hot table accessed by many connections concurrently.

            There are many common workloads wihout a hot table, in these cases there will be only one table cache instance. Your fix in fb27ed99a79f7e9b6c4e838d8a788a4685cfbee4 would unnecessary penalize these workloads — they'll have a smaller table cache for no good reason. I'd suggest to auto-reduce tc_instances instead.

            serg Sergei Golubchik added a comment - I think svoj suggested a better fix than fb27ed99a79f7e9b6c4e838d8a788a4685cfbee4. New cache instances are created, when the contention is too high, which normally means there is some hot table accessed by many connections concurrently. There are many common workloads wihout a hot table, in these cases there will be only one table cache instance. Your fix in fb27ed99a79f7e9b6c4e838d8a788a4685cfbee4 would unnecessary penalize these workloads — they'll have a smaller table cache for no good reason. I'd suggest to auto-reduce tc_instances instead.
            sanja Oleksandr Byelkin added a comment -

            I still do not understand how playing with soft limit can solve the problem described above?

            We promise big number of instances and big cash, in the middle of the game we say no we can not open more - how it is different from what we have now (inability to open files and crash)?

            sanja Oleksandr Byelkin added a comment - I still do not understand how playing with soft limit can solve the problem described above? We promise big number of instances and big cash, in the middle of the game we say no we can not open more - how it is different from what we have now (inability to open files and crash)?
            svoj Sergey Vojtovich added a comment -

            Don't increment number of instances if failed to increment soft limit. Then everything is under control, right?

            svoj Sergey Vojtovich added a comment - Don't increment number of instances if failed to increment soft limit. Then everything is under control, right?
            sanja Oleksandr Byelkin added a comment -

            ommit edc9059c31bddfaa5294423dafc6adfd5a3eabc0 (HEAD > bb-10.2MDEV-18027, origin/bb-10.2-MDEV-18027)
            Author: Oleksandr Byelkin <sanja@mariadb.com>
            Date: Wed Jul 10 13:40:54 2019 +0200

            MDEV-18027: Running out of file descriptors and eventual crash

            For automatic number of opened files limit take into account number of table instances for table cache

            sanja Oleksandr Byelkin added a comment - ommit edc9059c31bddfaa5294423dafc6adfd5a3eabc0 (HEAD > bb-10.2 MDEV-18027 , origin/bb-10.2- MDEV-18027 ) Author: Oleksandr Byelkin <sanja@mariadb.com> Date: Wed Jul 10 13:40:54 2019 +0200 MDEV-18027 : Running out of file descriptors and eventual crash For automatic number of opened files limit take into account number of table instances for table cache

            People

              sanja Oleksandr Byelkin
              David Crimmins David Crimmins
              Votes:
              2 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.