Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-28073

Slow query performance in MariaDB when using many tables

Details

    Description

      Looking for root cause for the following problem:
      A query which comes from a generated report used to take seconds and now takes 21 min (tested in house on MariaDB ES 10.6.5, in MySQL 5.6 it took 9.611s Customer is having the issue in 10.2.27 .)

      The customer found that by setting the optimizer_search_depth=10 the amount of time is similar to original run times. They tested 11 through 15 as well. They noticed `optimizer_search_depth=10` performed identically to `optimizer_search_depth=0` and `optimizer_search_depth=7`. However, `optimizer_search_depth=11` started to show degraded performance. They saw that at `optimizer_search_depth=15`, there was no, discernible difference between that and the default `optimizer_search_depth=62`.

      Below are the steps to reproduce the issue.
      The files are included in CS0379791-March102022.tar.gz in the case.
      1. Provision MariaDB enterprise 10.6.5-2 (customer had issue with 10.2 and 10.4 DB) using server.cnf.txt (modified version of customers xpo_testcase_my.cnf).
      2. Connect to MariaDB instance.
      3. create database discard;
      4. use discard;
      5. source xpo_full_testcase.sql;
      6. source xpo_testcase_explain.sql.

      Optimizer trace with default optimzer_search_depth takes 12 hours and we were unable to capture the data.
      However, setting optimizer_search_depth to 3 allowed the capture data from an optimizer trace see optimizer_output_depth3 .

      Relevant files are attached to case CS0379791 as CS0379791-March102022.tar.gz
      1). optimizer_search_depth_scaling.png - graph of time is takes to run the query when setting optimizer_search_depth = [1-25]
      2). server.cnf.txt - config file used by support when reproducing the issue
      3). xpo_testcase_my.cnf - config file usde by customer to reproduce the issue
      4). xpo_testcase_explain.sql - explain of the query in question
      5). xpo_full_testcase.sql - data dump to reproduce the issue with
      6). optimizer_output_depth3 - optimzer trace with optimizer_search_depth = 3
      7). CS0379791_output/ - flamegraph output including perf output and more
      8). explain_json.txt - Additionally the customer has attached output from explain json from customer

      Attachments

        Issue Links

          Activity

            mpflaum Maria M Pflaum (Inactive) created issue -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            Field Original Value New Value
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            julien.fritsch Julien Fritsch made changes -
            Assignee Sergei Petrunia [ psergey ]
            julien.fritsch Julien Fritsch made changes -
            Key MDEV-28035 MENT-1442
            Project MariaDB Server [ 10000 ] MariaDB Enterprise [ 11500 ]
            julien.fritsch Julien Fritsch made changes -
            Affects Version/s 10.6.5-2 [ 26062 ]
            julien.fritsch Julien Fritsch made changes -
            Fix Version/s 10.6 [ 24027 ]
            julien.fritsch Julien Fritsch made changes -
            Fix Version/s 10.6 [ 24028 ]
            Fix Version/s 10.6 [ 24027 ]
            Key MENT-1442 MDEV-28073
            Affects Version/s 10.6.5-2 [ 26062 ]
            Project MariaDB Enterprise [ 11500 ] MariaDB Server [ 10000 ]
            julien.fritsch Julien Fritsch made changes -
            Affects Version/s 10.6.5 [ 26034 ]
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            Affects Version/s 10.2.27 [ 23717 ]
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            julien.fritsch Julien Fritsch made changes -
            Fix Version/s 10.2 [ 14601 ]
            julien.fritsch Julien Fritsch made changes -
            Comment [ A comment with security level 'Developers' was removed. ]
            julien.fritsch Julien Fritsch made changes -
            Status Open [ 1 ] Confirmed [ 10101 ]
            serg Sergei Golubchik made changes -
            Fix Version/s 10.2 [ 14601 ]
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            julien.fritsch Julien Fritsch made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            monty Michael Widenius made changes -
            Assignee Sergei Petrunia [ psergey ] Michael Widenius [ monty ]
            monty Michael Widenius made changes -
            Status Confirmed [ 10101 ] In Progress [ 3 ]
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            monty Michael Widenius made changes -
            Summary Query perfomance degradation in newer MariaDB versions Query perfromance degradation in newer MariaDB versions when using many tables
            julien.fritsch Julien Fritsch made changes -
            Summary Query perfromance degradation in newer MariaDB versions when using many tables Query performance degradation in newer MariaDB versions when using many tables
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            monty Michael Widenius made changes -
            Summary Query performance degradation in newer MariaDB versions when using many tables Slow query performance in MariaDB when using many tables
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            serg Sergei Golubchik made changes -
            Fix Version/s 10.6.8 [ 27506 ]
            Fix Version/s 10.6 [ 24028 ]
            serg Sergei Golubchik made changes -
            Fix Version/s 10.9.1 [ 27114 ]
            Fix Version/s 10.8.3 [ 27502 ]
            Fix Version/s 10.7.4 [ 27504 ]
            serg Sergei Golubchik made changes -
            Component/s Optimizer [ 10200 ]
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Closed [ 6 ]
            psergei Sergei Petrunia made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            Attachment global_variables_10-6-9.csv [ 64220 ]
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            mariadb-jira-automation Jira Automation (IT) made changes -
            Zendesk Related Tickets 201658 194725 156926
            Zendesk active tickets 201658

            People

              monty Michael Widenius
              mpflaum Maria M Pflaum (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.