Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-9531

GROUP_CONCAT with ORDER BY inside takes a lot of memory while it's executed

Details

    • 10.1.14, 10.1.18, 10.1.19

    Description

      A query like 

      SELECT GROUP_CONCAT( seq, seq, seq, seq, seq, seq, seq, seq ORDER BY 2,1,3,4,6,5,8,7 ) AS cfield1 FROM seq_1_to_50000000

      easily uses up to 4-5 Gb memory while it's running.

      <montywi> everything has limits
      <montywi> even for filesort; For bigger things it should always hit disk, never memory
      ...
      <montywi> the problem is that when doing a sort, we create a sorted tree in memory
      <montywi> however there is no memory check for this
      <montywi> so this can be any size
      <montywi> Normally it would make sense to limit this to group_concat_max_length, but I fear that would break things, as we sort records there, which can have a notable overhead
      <montywi> So I have limit it to max_heap_table_size

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-30282 Uncontrollable memory consumption upon GROUP_CONCAT with ORDER BY

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            elenst Elena Stepanova created issue -
            elenst Elena Stepanova made changes -
            Field Original Value New Value
            Priority Major [ 3 ] Minor [ 4 ]
            serg Sergei Golubchik made changes -
            Assignee Elena Stepanova [ elenst ] Oleksandr Byelkin [ sanja ]
            serg Sergei Golubchik made changes -
            Priority Minor [ 4 ] Major [ 3 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Sprint 10.1.14 [ 51 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Rank Ranked lower
            sanja Oleksandr Byelkin made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            sanja Oleksandr Byelkin made changes -
            Assignee Oleksandr Byelkin [ sanja ] Sergei Golubchik [ serg ]
            Status In Progress [ 3 ] In Review [ 10002 ]
            serg Sergei Golubchik made changes -
            Assignee Sergei Golubchik [ serg ] Oleksandr Byelkin [ sanja ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            sanja Oleksandr Byelkin made changes -
            Status Stalled [ 10000 ] In Progress [ 3 ]
            sanja Oleksandr Byelkin made changes -
            Status In Progress [ 3 ] Stalled [ 10000 ]
            sanja Oleksandr Byelkin made changes -
            Assignee Oleksandr Byelkin [ sanja ] Sergei Golubchik [ serg ]
            Status Stalled [ 10000 ] In Review [ 10002 ]
            serg Sergei Golubchik made changes -
            Assignee Sergei Golubchik [ serg ] Oleksandr Byelkin [ sanja ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            serg Sergei Golubchik made changes -
            Assignee Oleksandr Byelkin [ sanja ] Sergei Golubchik [ serg ]
            serg Sergei Golubchik made changes -
            Status Stalled [ 10000 ] In Review [ 10002 ]
            serg Sergei Golubchik made changes -
            Sprint 10.1.14 [ 51 ] 10.1.14, 10.1.18 [ 51, 98 ]
            serg Sergei Golubchik made changes -
            Assignee Sergei Golubchik [ serg ] Oleksandr Byelkin [ sanja ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            sanja Oleksandr Byelkin made changes -
            Status Stalled [ 10000 ] In Progress [ 3 ]
            sanja Oleksandr Byelkin made changes -
            Assignee Oleksandr Byelkin [ sanja ] Sergei Golubchik [ serg ]
            serg Sergei Golubchik made changes -
            Assignee Sergei Golubchik [ serg ] Oleksandr Byelkin [ sanja ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Sprint 10.1.14, 10.1.18 [ 51, 98 ] 10.1.14, 10.1.18, 10.1.19 [ 51, 98, 109 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Rank Ranked higher
            serg Sergei Golubchik made changes -
            Assignee Oleksandr Byelkin [ sanja ] Sergei Golubchik [ serg ]
            serg Sergei Golubchik made changes -
            Status In Progress [ 3 ] Stalled [ 10000 ]
            serg Sergei Golubchik made changes -
            Status Stalled [ 10000 ] In Progress [ 3 ]
            serg Sergei Golubchik made changes -
            Status In Progress [ 3 ] Stalled [ 10000 ]
            serg Sergei Golubchik made changes -
            Assignee Sergei Golubchik [ serg ] Oleksandr Byelkin [ sanja ]
            Status Stalled [ 10000 ] In Review [ 10002 ]
            sanja Oleksandr Byelkin made changes -
            Assignee Oleksandr Byelkin [ sanja ] Sergei Golubchik [ serg ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            serg Sergei Golubchik made changes -
            Component/s OTHER [ 10125 ]
            Fix Version/s 10.1.39 [ 23305 ]
            Fix Version/s 10.2.24 [ 23308 ]
            Fix Version/s 10.3.15 [ 23309 ]
            Fix Version/s 10.4.5 [ 23311 ]
            Fix Version/s 10.1 [ 16100 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 73902 ] MariaDB v4 [ 150087 ]
            elenst Elena Stepanova made changes -

            People

              serg Sergei Golubchik
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.