Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-4593

Multiple concurrent queries with aggregates are bottlenecked, result in lack of user scalability

    XMLWordPrintable

Details

    • Bug
    • Status: Stalled (View Workflow)
    • Trivial
    • Resolution: Unresolved
    • 1.2.0
    • 23.10
    • None
    • 2021-4, 2021-5, 2021-6, 2021-7, 2021-8, 2021-9, 2021-10, 2021-11, 2021-12, 2021-13, 2021-14, 2021-15, 2021-16, 2021-17, 2022-22, 2022-23, 2023-4, 2023-5, 2023-6, 2023-7, 2023-8, 2023-10, 2023-11

    Description

      It has been proven by multiple experiments that Columnstore does not scale with increasing concurrency, when queries with aggregates are involved. Every additional query stream adds average latency to all other sessions regardless of the available resources and secret columnstore settings.

      Total CPU consumed by PrimProcs is likewise not linear (2 concurrent sessions spend more than 2x total CPU in PrimProc as compared to just one), indicating some sort of spin lock.

      The issue was escalated from a prospect (Quinstreet) who discovered it.

      Attachments

        Issue Links

          causes

          New Feature - A new feature of the product, which has yet to be developed. MCOL-4679 Simplify configuration of EM to PM connections

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          includes

          Task - A task that needs to be done. MCOL-4692 Remove global mutex lock in OAMCache facility.

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MCOL-4694 Distribute primitive jobs messages b/w TCP connections provided by DEC

          • Major - Major loss of function.
          • Closed
          is blocked by

          New Feature - A new feature of the product, which has yet to be developed. MCOL-5043 Reduce a number of pre-spawned ExeMgr threads

          • Major - Major loss of function.
          • Stalled

          New Feature - A new feature of the product, which has yet to be developed. MCOL-5044 Improve PP thread pool with a fair scheduler

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4691 Major Regression: Selects with aggregates 2x slower in 5.x than in 1.2 (due to collation support)

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MCOL-4770 Distribute queries processing b/w ExeMgrs/primprocs in the cluster.

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. MCOL-4849 Optimize ExeMgr to reduce a number of context switches

          • Major - Major loss of function.
          • Closed

          Activity

            People

              drrtuy Roman
              gdorman Gregory Dorman (Inactive)
              Votes:
              3 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - 70d
                  70d
                  Remaining:
                  Remaining Estimate - 70d
                  70d
                  Logged:
                  Time Spent - Not Specified
                  Not Specified

                  Git Integration

                    Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.