Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-24117

Memory management problem in statistics state for queries that have a large field IN (values) part

Details

    Description

      A query with a large IN () section will be stuck in statistics and use up all RAM. Multiple such queries will crash the server regardless of memory limitations specified in my.cnf

      I noticed this in 10.2.35. Downgrading to 10.2.34 fixes the problem

      An example is the WordPress "select post_id,meta_value from wp_postmeta where meta_id in (1,2,3 etc ...)" query.

      Attachments

        Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-27380 After minor upgrade to 10.5.13 queries with large IN clauses have performance issues

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          is caused by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-23811 With large number of indexes optimizer chooses an inefficient plan

          • Major - Major loss of function.
          • Closed
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24163 Mariadb is restarting after sending this message [Warning] Aborted connection 5889 to db: 'xxxx' user: 'xxxxx' host:

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24173 After minor upgrade to 10.2.35 queries with large IN clauses have performance issues

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24195 Huge memory usage after upgrade

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            View 9 older comments
            jaedcar74 Javier Carvajal added a comment - - edited

            I have same issue. Server was had to be upgraded to 32G RAM with 10.2.35 on Debian 9 (Stretch). RAM wildly fluctuates. The server was killed by the kernel OOM killer.

            Any quick fix?

            jaedcar74 Javier Carvajal added a comment - - edited I have same issue. Server was had to be upgraded to 32G RAM with 10.2.35 on Debian 9 (Stretch). RAM wildly fluctuates. The server was killed by the kernel OOM killer. Any quick fix?
            fuegas Ferdi van der Werf added a comment -

            I'd recommend to downgrade to 10.2.34 and apt-mark hold it until there is a fix available. We've seen servers downgraded to 10.2.34 to return to normal RAM usage.

            fuegas Ferdi van der Werf added a comment - I'd recommend to downgrade to 10.2.34 and apt-mark hold it until there is a fix available. We've seen servers downgraded to 10.2.34 to return to normal RAM usage.
            psergei Sergei Petrunia added a comment - - edited

            Hello,

            A set of release is about to be made (10.2.36, 10.3.27, 10.4.17, 10.5.8) which will contain the fix for this.
            (EDIT: fix version#)

            psergei Sergei Petrunia added a comment - - edited Hello, A set of release is about to be made (10.2.36, 10.3.27, 10.4.17, 10.5.8) which will contain the fix for this. (EDIT: fix version#)
            psergei Sergei Petrunia added a comment -

            The commits that fix this are (all 3 were made in a row):

            https://github.com/MariaDB/server/commit/f81eef62e7742806e5e74b5f37f35b7cd2f82291
            https://github.com/MariaDB/server/commit/1404f3bea796c8479cf401cb36d518658600ddca
            https://github.com/MariaDB/server/commit/bea84aefb0563a10a310ea81d46c372919345c10

            psergei Sergei Petrunia added a comment - The commits that fix this are (all 3 were made in a row): https://github.com/MariaDB/server/commit/f81eef62e7742806e5e74b5f37f35b7cd2f82291 https://github.com/MariaDB/server/commit/1404f3bea796c8479cf401cb36d518658600ddca https://github.com/MariaDB/server/commit/bea84aefb0563a10a310ea81d46c372919345c10
            danblack Daniel Black added a comment -

            For all those watching patiently, there have been packages released https://mariadb.org/mariadb-10-5-8-10-4-17-10-3-27-and-10-2-36-now-available/

            danblack Daniel Black added a comment - For all those watching patiently, there have been packages released https://mariadb.org/mariadb-10-5-8-10-4-17-10-3-27-and-10-2-36-now-available/

            People

              psergei Sergei Petrunia
              glucz Geza Lucz
              Votes:
              11 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.