Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-40509

Redundant DISTINCT in set-operation branches creates unnecessary temporary tables

    XMLWordPrintable

Details

    • Related to performance

    Description

       
      ## Description
       
      MariaDB retains `DISTINCT` inside both branches of `UNION`, `INTERSECT`, and
      `EXCEPT`, although the set operation already provides distinct set semantics.
       
      ### Expected behaviour
       
      No branch-level temporary table should be required. The branch `DISTINCT`
      operators should be removed or merged with the set operation.
       
      ### Actual behaviour
       
      `EXPLAIN FORMAT=JSON` reports a `temporary_table` inside each branch in
      addition to the `union_result`, `intersect_result`, or `except_result`.
       
      | Operation | Without branch DISTINCT | With branch DISTINCT | Slowdown |
      |---|---:|---:|---:|
      | UNION | 62.57 ms | 84.94 ms | 1.36x |
      | INTERSECT | 51.98 ms | 72.63 ms | 1.40x |
      | EXCEPT | 44.83 ms | 68.46 ms | 1.53x |
       
      ## How to repeat
       
      ```sql
      DROP DATABASE IF EXISTS mariadb_set_branch_distinct;
      CREATE DATABASE mariadb_set_branch_distinct;
      USE mariadb_set_branch_distinct;
      CREATE TABLE digits(d INT PRIMARY KEY);
      INSERT INTO digits VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9);
      CREATE TABLE lhs(id INT, v INT);
      CREATE TABLE rhs(id INT, v INT);
      INSERT INTO lhs SELECT n,MOD(n,1000) FROM
       (SELECT a.d+b.d*10+c.d*100+d.d*1000+e.d*10000+1 n
        FROM digits a,digits b,digits c,digits d,digits e) numbers;
      INSERT INTO rhs SELECT n,MOD(n,1000) FROM
       (SELECT a.d+b.d*10+c.d*100+d.d*1000+e.d*10000+50001 n
        FROM digits a,digits b,digits c,digits d,digits e) numbers;
      ANALYZE TABLE lhs,rhs;
       
      EXPLAIN FORMAT=JSON SELECT COUNT(*) FROM
       ((SELECT DISTINCT id FROM lhs) UNION (SELECT DISTINCT id FROM rhs)) s;
      EXPLAIN FORMAT=JSON SELECT COUNT(*) FROM
       ((SELECT DISTINCT id FROM lhs) INTERSECT (SELECT DISTINCT id FROM rhs)) s;
      EXPLAIN FORMAT=JSON SELECT COUNT(*) FROM
       ((SELECT DISTINCT id FROM lhs) EXCEPT (SELECT DISTINCT id FROM rhs)) s;
       
      SELECT COUNT(*) FROM ((SELECT id FROM lhs) UNION (SELECT id FROM rhs)) s;
      SELECT COUNT(*) FROM
       ((SELECT DISTINCT id FROM lhs) UNION (SELECT DISTINCT id FROM rhs)) s;
      SELECT COUNT(*) FROM ((SELECT id FROM lhs) INTERSECT (SELECT id FROM rhs)) s;
      SELECT COUNT(*) FROM
       ((SELECT DISTINCT id FROM lhs) INTERSECT (SELECT DISTINCT id FROM rhs)) s;
      SELECT COUNT(*) FROM ((SELECT id FROM lhs) EXCEPT (SELECT id FROM rhs)) s;
      SELECT COUNT(*) FROM
       ((SELECT DISTINCT id FROM lhs) EXCEPT (SELECT DISTINCT id FROM rhs)) s;
      ```
      
      

      Attachments

        Issue Links

          Activity

            People

              psergei Sergei Petrunia
              chen7897 cl hl
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.