Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-3828

UNION: complains about table not in query

Details

    Description

      Consider the following sequence from working_tpch/union/union.sql

      CREATE TABLE t1 (
      cid smallint(5) ,
      cv varchar(250)
      ) engine=columnstore;

      INSERT INTO t1 VALUES (8,'dummy');
      CREATE TABLE t2 (
      cid bigint(20),
      cap varchar(255)
      ) engine=columnstore;
      CREATE TABLE t3 (
      gid bigint(20) ,
      gn varchar(255),
      must tinyint(4)
      ) engine=columnstore;
      INSERT INTO t3 VALUES (1,'V1',NULL);
      CREATE TABLE t4 (
      uid bigint(20),
      gid bigint(20),
      rid bigint(20),
      cid bigint(20)
      ) engine=columnstore;

      INSERT INTO t4 VALUES (1,1,NULL,NULL);
      CREATE TABLE t5 (
      rid bigint(20),
      rl varchar(255)
      ) engine=columnstore;
      CREATE TABLE t6 (
      uid bigint(20),
      un varchar(250),
      uc smallint(5)
      ) engine=columnstore;
      INSERT INTO t6 VALUES (1,'test',8);

      SELECT t4.uid, t5.rl, t3.gn as g1, t4.cid, t4.gid as gg FROM t3, t6, t1, t4 left join t5 on t5.rid = t4.rid left join t2 on t2.cid = t4.cid WHERE t3.gid=t4.gid AND t6.uid = t4.uid AND t6.uc = t1.cid AND t1.cv = "dummy" AND t6.un = "test";
      SELECT t4.uid, t5.rl, t3.gn as g1, t4.cid, t4.gid as gg FROM t3, t6, t1, t4 left join t5 on t5.rid = t4.rid left join t2 on t2.cid = t4.cid WHERE t3.gid=t4.gid AND t6.uid = t4.uid AND t3.must IS NOT NULL AND t6.uc = t1.cid AND t1.cv = "dummy" AND t6.un = "test";
      (SELECT t4.uid, t5.rl, t3.gn as g1, t4.cid, t4.gid as gg FROM t3, t6, t1, t4 left join t5 on t5.rid = t4.rid left join t2 on t2.cid = t4.cid WHERE t3.gid=t4.gid AND t6.uid = t4.uid AND t3.must IS NOT NULL AND t6.uc = t1.cid AND t1.cv = "dummy" AND t6.un = "test") UNION (SELECT t4.uid, t5.rl, t3.gn as g1, t4.cid, t4.gid as gg FROM t3, t6, t1, t4 left join t5 on t5.rid = t4.rid left join t2 on t2.cid = t4.cid WHERE t3.gid=t4.gid AND t6.uid = t4.uid AND t6.uc = t1.cid AND t1.cv = "dummy" AND t6.un = "test");
      drop table if exists t1;
      drop table if exists t2;
      drop table if exists t3;
      drop table if exists t4;
      drop table if exists t5;
      drop table if exists t6;

      create table t1 (a int) engine=columnstore;
      insert into t1 values (1),(2),(3);
      create table t2 (a int) engine=columnstore;
      insert into t2 values (3),(4),(5);

      SELECT * FROM t1 UNION SELECT * FROM t2 ORDER BY a desc;
      ERROR 1815 (HY000): Internal error: IDB-2006: 'tpch1.t6' does not exist in Columnstore.

      This only happens if you run the full sequence. Otherwise, there's no complaint about a table not existing – one that isn't in the query at all.

      Attachments

        Issue Links

          Activity

            This appears to be a similar problem that sometimes occurs in working_tpch1_comparelogonly.view/view.sql
            CREATE TABLE t1 (lid int, name char(10)) engine=columnstore;
            INSERT INTO t1 (lid, name) VALUES (1, 'YES'), (2, 'NO');

            CREATE TABLE t2 ( id int, gid int, lid int, dt date) engine=columnstore;
            INSERT INTO t2 (id, gid, lid, dt) VALUES
            (1, 1, 1, '2007-01-01'),(2, 1, 2, '2007-01-02'),
            (3, 2, 2, '2007-02-01'),(4, 2, 1, '2007-02-02');

            SELECT DISTINCT t2.gid AS lgid,(SELECT t1.name FROM t1, t2 WHERE t1.lid = t2.lid AND t2.gid = t1.lid and t1.lid =1 ) as clid FROM t2 order by 1, 2;

            CREATE VIEW v1 AS SELECT DISTINCT t2.gid AS lgid,(SELECT t1.name FROM t1, t2 WHERE t1.lid = t2.lid AND t2.gid = t1.lid and t1.lid =1 ) as clid FROM t2;
            SELECT * FROM v1 order by 1, 2;

            DROP VIEW v1;
            DROP table t1;
            drop table t2;

            CREATE TABLE t1 (a INT) engine=columnstore;
            INSERT INTO t1 VALUES (1),(2),(3);
            CREATE VIEW v1 AS SELECT a FROM t1 ORDER BY a;

            SELECT * FROM t1 UNION SELECT * FROM v1;
            ERROR 1815 (HY000) at line 896: Internal error: IDB-2006: 'tpch1.t2' does not exist in Columnstore.

            David.Hall David Hall (Inactive) added a comment - This appears to be a similar problem that sometimes occurs in working_tpch1_comparelogonly.view/view.sql CREATE TABLE t1 (lid int, name char(10)) engine=columnstore; INSERT INTO t1 (lid, name) VALUES (1, 'YES'), (2, 'NO'); CREATE TABLE t2 ( id int, gid int, lid int, dt date) engine=columnstore; INSERT INTO t2 (id, gid, lid, dt) VALUES (1, 1, 1, '2007-01-01'),(2, 1, 2, '2007-01-02'), (3, 2, 2, '2007-02-01'),(4, 2, 1, '2007-02-02'); SELECT DISTINCT t2.gid AS lgid,(SELECT t1.name FROM t1, t2 WHERE t1.lid = t2.lid AND t2.gid = t1.lid and t1.lid =1 ) as clid FROM t2 order by 1, 2; CREATE VIEW v1 AS SELECT DISTINCT t2.gid AS lgid,(SELECT t1.name FROM t1, t2 WHERE t1.lid = t2.lid AND t2.gid = t1.lid and t1.lid =1 ) as clid FROM t2; SELECT * FROM v1 order by 1, 2; DROP VIEW v1; DROP table t1; drop table t2; CREATE TABLE t1 (a INT) engine=columnstore; INSERT INTO t1 VALUES (1),(2),(3); CREATE VIEW v1 AS SELECT a FROM t1 ORDER BY a; SELECT * FROM t1 UNION SELECT * FROM v1; ERROR 1815 (HY000) at line 896: Internal error: IDB-2006: 'tpch1.t2' does not exist in Columnstore.

            Got a different error this time:
            create table t1 (a int) engine=columnstore;
            insert into t1 values (1),(2),(3);
            create table t2 (a int) engine=columnstore;
            insert into t2 values (3),(4),(5);
            SELECT * FROM t1 UNION SELECT * FROM t2 ORDER BY a desc;

            ERROR 1815 (HY000) at line 115: Internal error: value is not numerical.

            David.Hall David Hall (Inactive) added a comment - Got a different error this time: create table t1 (a int) engine=columnstore; insert into t1 values (1),(2),(3); create table t2 (a int) engine=columnstore; insert into t2 values (3),(4),(5); SELECT * FROM t1 UNION SELECT * FROM t2 ORDER BY a desc; ERROR 1815 (HY000) at line 115: Internal error: value is not numerical.

            Found a similar problem in working_tpch1_compareLogOnly/view/mts_view.sql lines 896 and 897

            David.Hall David Hall (Inactive) added a comment - Found a similar problem in working_tpch1_compareLogOnly/view/mts_view.sql lines 896 and 897
            drrtuy Roman added a comment -

            So, there were two bugs that caused the issues:
            "Internal error: value is not numerical" and "'tpch1.t6' does not exist in Columnstore" were caused by our conditions pushdown mechanism that doesn't clean after query execution with table API. I fixed the issue but the whole mechanism needs to be rewritten given this is an easy thing to do.

            tables are not joined was caused by join order and conditions replacement introduced by MDB's optimizer that was called by optimize_unflattened_subqueries_mcs() function under the hood. I replaced the f() to address equi-JOIN conditions injection for IN + subquery predicates.

            drrtuy Roman added a comment - So, there were two bugs that caused the issues: "Internal error: value is not numerical" and "'tpch1.t6' does not exist in Columnstore" were caused by our conditions pushdown mechanism that doesn't clean after query execution with table API. I fixed the issue but the whole mechanism needs to be rewritten given this is an easy thing to do. tables are not joined was caused by join order and conditions replacement introduced by MDB's optimizer that was called by optimize_unflattened_subqueries_mcs() function under the hood. I replaced the f() to address equi-JOIN conditions injection for IN + subquery predicates.
            drrtuy Roman added a comment -

            Plz review.

            drrtuy Roman added a comment - Plz review.

            Builds verified: 1.4.4-1 source, 1.5.0-1 BB

            1.4.4-1

            /root/ColumnStore/buildColumnstoreFromGithubSource/server
            commit ec0071afa50b78930860a14802b8cfc9791f7d13
            Author: Sergei Petrunia <psergey@askmonty.org>
            Date: Thu Mar 26 01:26:39 2020 +0300

            MDEV-21887: federatedx crashes on SELECT ... INTO query in select_handler code

            /root/ColumnStore/buildColumnstoreFromGithubSource/server/engine
            commit 6e953d01d268161ac50dabb6aa9a122ed1d43e0f
            Merge: 4468c93 ba06727
            Author: Gagan Goel <gagan.nith@gmail.com>
            Date: Thu Mar 26 11:41:35 2020 -0400

            Merge pull request #1117 from drrtuy/MCOL-3828_1_4

            MCOL-3828 This commit replaces the method that calls JOIN::optimise()

            1.5.0-1

            engine commit:
            a35b208

            Verified tests cases in both bug description and in comments.

            dleeyh Daniel Lee (Inactive) added a comment - Builds verified: 1.4.4-1 source, 1.5.0-1 BB 1.4.4-1 /root/ColumnStore/buildColumnstoreFromGithubSource/server commit ec0071afa50b78930860a14802b8cfc9791f7d13 Author: Sergei Petrunia <psergey@askmonty.org> Date: Thu Mar 26 01:26:39 2020 +0300 MDEV-21887 : federatedx crashes on SELECT ... INTO query in select_handler code /root/ColumnStore/buildColumnstoreFromGithubSource/server/engine commit 6e953d01d268161ac50dabb6aa9a122ed1d43e0f Merge: 4468c93 ba06727 Author: Gagan Goel <gagan.nith@gmail.com> Date: Thu Mar 26 11:41:35 2020 -0400 Merge pull request #1117 from drrtuy/ MCOL-3828 _1_4 MCOL-3828 This commit replaces the method that calls JOIN::optimise() 1.5.0-1 engine commit: a35b208 Verified tests cases in both bug description and in comments.

            Build verified: 1.4.3-5 hot fix, Azure 20200401.6

            MariaDB [mytest]> CREATE TABLE t1 (
            -> cid smallint(5) ,
            -> cv varchar(250)
            -> ) engine=columnstore;
            Query OK, 0 rows affected (0.161 sec)

            MariaDB [mytest]> INSERT INTO t1 VALUES (8,'dummy');
            Query OK, 1 row affected (0.222 sec)

            MariaDB [mytest]> CREATE TABLE t2 (
            -> cid bigint(20),
            -> cap varchar(255)
            -> ) engine=columnstore;
            Query OK, 0 rows affected (0.164 sec)

            MariaDB [mytest]> CREATE TABLE t3 (
            -> gid bigint(20) ,
            -> gn varchar(255),
            -> must tinyint(4)
            -> ) engine=columnstore;
            Query OK, 0 rows affected (0.160 sec)

            MariaDB [mytest]> INSERT INTO t3 VALUES (1,'V1',NULL);
            Query OK, 1 row affected (0.343 sec)

            MariaDB [mytest]> CREATE TABLE t4 (
            -> uid bigint(20),
            -> gid bigint(20),
            -> rid bigint(20),
            -> cid bigint(20)
            -> ) engine=columnstore;
            Query OK, 0 rows affected (0.184 sec)

            MariaDB [mytest]> INSERT INTO t4 VALUES (1,1,NULL,NULL);
            t5.rl, t3.gn as g1, t4.cid, t4.gid as gg FROM t3, t6, t1, t4 left join t5 on t5.rid = t4.rid left join t2 on t2.cid = t4.cid WHERE t3.gid=t4.gid AND t6.uid = t4.uid AND t6.uc = t1.cid AND t1.cv = "dummy" AND t6.un = "test");
            drop table if exists t1;
            drop table if exists t2;
            drop table if exists t3;
            drop table if exists t4;
            drop table if exists t5;
            drop table if exists t6;
            Query OK, 1 row affected (0.369 sec)

            MariaDB [mytest]> CREATE TABLE t5 (
            -> rid bigint(20),
            -> rl varchar(255)
            -> ) engine=columnstore;
            Query OK, 0 rows affected (0.170 sec)

            MariaDB [mytest]> CREATE TABLE t6 (
            -> uid bigint(20),
            -> un varchar(250),
            -> uc smallint(5)
            -> ) engine=columnstore;
            Query OK, 0 rows affected (0.177 sec)

            MariaDB [mytest]> INSERT INTO t6 VALUES (1,'test',8);
            Query OK, 1 row affected (0.295 sec)

            MariaDB [mytest]> SELECT t4.uid, t5.rl, t3.gn as g1, t4.cid, t4.gid as gg FROM t3, t6, t1, t4 left join t5 on t5.rid = t4.rid left join t2 on t2.cid = t4.cid WHERE t3.gid=t4.gid AND t6.uid = t4.uid AND t6.uc = t1.cid AND t1.cv = "dummy" AND t6.un = "test";
            ----------------------

            uid rl g1 cid gg

            ----------------------

            1 NULL V1 NULL 1

            ----------------------
            1 row in set (0.236 sec)

            MariaDB [mytest]> SELECT t4.uid, t5.rl, t3.gn as g1, t4.cid, t4.gid as gg FROM t3, t6, t1, t4 left join t5 on t5.rid = t4.rid left join t2 on t2.cid = t4.cid WHERE t3.gid=t4.gid AND t6.uid = t4.uid AND t3.must IS NOT NULL AND t6.uc = t1.cid AND t1.cv = "dummy" AND t6.un = "test";
            Empty set (0.064 sec)

            MariaDB [mytest]> (SELECT t4.uid, t5.rl, t3.gn as g1, t4.cid, t4.gid as gg FROM t3, t6, t1, t4 left join t5 on t5.rid = t4.rid left join t2 on t2.cid = t4.cid WHERE t3.gid=t4.gid AND t6.uid = t4.uid AND t3.must IS NOT NULL AND t6.uc = t1.cid AND t1.cv = "dummy" AND t6.un = "test") UNION (SELECT t4.uid, t5.rl, t3.gn as g1, t4.cid, t4.gid as gg FROM t3, t6, t1, t4 left join t5 on t5.rid = t4.rid left join t2 on t2.cid = t4.cid WHERE t3.gid=t4.gid AND t6.uid = t4.uid AND t6.uc = t1.cid AND t1.cv = "dummy" AND t6.un = "test");
            ----------------------

            uid rl g1 cid gg

            ----------------------

            1 NULL V1 NULL 1

            ----------------------
            1 row in set (0.138 sec)

            MariaDB [mytest]> drop table if exists t1;
            Query OK, 0 rows affected (0.293 sec)

            MariaDB [mytest]> drop table if exists t2;
            Query OK, 0 rows affected (0.199 sec)

            MariaDB [mytest]> drop table if exists t3;
            Query OK, 0 rows affected (0.193 sec)

            MariaDB [mytest]> drop table if exists t4;
            Query OK, 0 rows affected (0.209 sec)

            MariaDB [mytest]> drop table if exists t5;
            Query OK, 0 rows affected (0.395 sec)

            MariaDB [mytest]> drop table if exists t6;
            Query OK, 0 rows affected (0.211 sec)

            MariaDB [mytest]> create table t1 (a int) engine=columnstore;
            Query OK, 0 rows affected (0.156 sec)

            MariaDB [mytest]> insert into t1 values (1),(2),(3);
            Query OK, 3 rows affected (0.327 sec)
            Records: 3 Duplicates: 0 Warnings: 0

            MariaDB [mytest]> create table t2 (a int) engine=columnstore;
            Query OK, 0 rows affected (0.142 sec)

            MariaDB [mytest]> insert into t2 values (3),(4),(5);
            Query OK, 3 rows affected (0.327 sec)
            Records: 3 Duplicates: 0 Warnings: 0

            MariaDB [mytest]> SELECT * FROM t1 UNION SELECT * FROM t2 ORDER BY a desc;
            ------

            a

            ------

            5
            4
            3
            2
            1

            ------
            5 rows in set (0.177 sec)

            dleeyh Daniel Lee (Inactive) added a comment - Build verified: 1.4.3-5 hot fix, Azure 20200401.6 MariaDB [mytest] > CREATE TABLE t1 ( -> cid smallint(5) , -> cv varchar(250) -> ) engine=columnstore; Query OK, 0 rows affected (0.161 sec) MariaDB [mytest] > INSERT INTO t1 VALUES (8,'dummy'); Query OK, 1 row affected (0.222 sec) MariaDB [mytest] > CREATE TABLE t2 ( -> cid bigint(20), -> cap varchar(255) -> ) engine=columnstore; Query OK, 0 rows affected (0.164 sec) MariaDB [mytest] > CREATE TABLE t3 ( -> gid bigint(20) , -> gn varchar(255), -> must tinyint(4) -> ) engine=columnstore; Query OK, 0 rows affected (0.160 sec) MariaDB [mytest] > INSERT INTO t3 VALUES (1,'V1',NULL); Query OK, 1 row affected (0.343 sec) MariaDB [mytest] > CREATE TABLE t4 ( -> uid bigint(20), -> gid bigint(20), -> rid bigint(20), -> cid bigint(20) -> ) engine=columnstore; Query OK, 0 rows affected (0.184 sec) MariaDB [mytest] > INSERT INTO t4 VALUES (1,1,NULL,NULL); t5.rl, t3.gn as g1, t4.cid, t4.gid as gg FROM t3, t6, t1, t4 left join t5 on t5.rid = t4.rid left join t2 on t2.cid = t4.cid WHERE t3.gid=t4.gid AND t6.uid = t4.uid AND t6.uc = t1.cid AND t1.cv = "dummy" AND t6.un = "test"); drop table if exists t1; drop table if exists t2; drop table if exists t3; drop table if exists t4; drop table if exists t5; drop table if exists t6; Query OK, 1 row affected (0.369 sec) MariaDB [mytest] > CREATE TABLE t5 ( -> rid bigint(20), -> rl varchar(255) -> ) engine=columnstore; Query OK, 0 rows affected (0.170 sec) MariaDB [mytest] > CREATE TABLE t6 ( -> uid bigint(20), -> un varchar(250), -> uc smallint(5) -> ) engine=columnstore; Query OK, 0 rows affected (0.177 sec) MariaDB [mytest] > INSERT INTO t6 VALUES (1,'test',8); Query OK, 1 row affected (0.295 sec) MariaDB [mytest] > SELECT t4.uid, t5.rl, t3.gn as g1, t4.cid, t4.gid as gg FROM t3, t6, t1, t4 left join t5 on t5.rid = t4.rid left join t2 on t2.cid = t4.cid WHERE t3.gid=t4.gid AND t6.uid = t4.uid AND t6.uc = t1.cid AND t1.cv = "dummy" AND t6.un = "test"; ----- ---- ---- ---- ----- uid rl g1 cid gg ----- ---- ---- ---- ----- 1 NULL V1 NULL 1 ----- ---- ---- ---- ----- 1 row in set (0.236 sec) MariaDB [mytest] > SELECT t4.uid, t5.rl, t3.gn as g1, t4.cid, t4.gid as gg FROM t3, t6, t1, t4 left join t5 on t5.rid = t4.rid left join t2 on t2.cid = t4.cid WHERE t3.gid=t4.gid AND t6.uid = t4.uid AND t3.must IS NOT NULL AND t6.uc = t1.cid AND t1.cv = "dummy" AND t6.un = "test"; Empty set (0.064 sec) MariaDB [mytest] > (SELECT t4.uid, t5.rl, t3.gn as g1, t4.cid, t4.gid as gg FROM t3, t6, t1, t4 left join t5 on t5.rid = t4.rid left join t2 on t2.cid = t4.cid WHERE t3.gid=t4.gid AND t6.uid = t4.uid AND t3.must IS NOT NULL AND t6.uc = t1.cid AND t1.cv = "dummy" AND t6.un = "test") UNION (SELECT t4.uid, t5.rl, t3.gn as g1, t4.cid, t4.gid as gg FROM t3, t6, t1, t4 left join t5 on t5.rid = t4.rid left join t2 on t2.cid = t4.cid WHERE t3.gid=t4.gid AND t6.uid = t4.uid AND t6.uc = t1.cid AND t1.cv = "dummy" AND t6.un = "test"); ----- ---- ---- ---- ----- uid rl g1 cid gg ----- ---- ---- ---- ----- 1 NULL V1 NULL 1 ----- ---- ---- ---- ----- 1 row in set (0.138 sec) MariaDB [mytest] > drop table if exists t1; Query OK, 0 rows affected (0.293 sec) MariaDB [mytest] > drop table if exists t2; Query OK, 0 rows affected (0.199 sec) MariaDB [mytest] > drop table if exists t3; Query OK, 0 rows affected (0.193 sec) MariaDB [mytest] > drop table if exists t4; Query OK, 0 rows affected (0.209 sec) MariaDB [mytest] > drop table if exists t5; Query OK, 0 rows affected (0.395 sec) MariaDB [mytest] > drop table if exists t6; Query OK, 0 rows affected (0.211 sec) MariaDB [mytest] > create table t1 (a int) engine=columnstore; Query OK, 0 rows affected (0.156 sec) MariaDB [mytest] > insert into t1 values (1),(2),(3); Query OK, 3 rows affected (0.327 sec) Records: 3 Duplicates: 0 Warnings: 0 MariaDB [mytest] > create table t2 (a int) engine=columnstore; Query OK, 0 rows affected (0.142 sec) MariaDB [mytest] > insert into t2 values (3),(4),(5); Query OK, 3 rows affected (0.327 sec) Records: 3 Duplicates: 0 Warnings: 0 MariaDB [mytest] > SELECT * FROM t1 UNION SELECT * FROM t2 ORDER BY a desc; ------ a ------ 5 4 3 2 1 ------ 5 rows in set (0.177 sec)

            People

              dleeyh Daniel Lee (Inactive)
              David.Hall David Hall (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.