Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-26825

Bogus error for query with two usage of CTE referring another CTE

Details

    Description

      When a double encapsulated CTE query calls a function which reads a table which has been aliased in the CTE query, the optimizer cannot find the table anymore and ends with an error 1146 Table db.t1 doesn't exist.

      Script to reproduce:

      CREATE TABLE t1 (id INT(10) NOT NULL AUTO_INCREMENT PRIMARY KEY);
      /* Generate 10'000 entries */
      INSERT INTO t1(id) SELECT @row := @row + 1 AS id FROM 
      (SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t,
      (SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t2, 
      (SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t3, 
      (SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t4, 
      (SELECT @row:=0) numbers;
       
      CREATE OR REPLACE FUNCTION check_my_id(in_id INT) RETURNS INTEGER
      READS SQL DATA
      RETURN (SELECT id FROM t1 WHERE t1.id = in_id);
       
       
      WITH c1 AS (SELECT id	FROM t1 a WHERE check_my_id(id) = id GROUP BY id),
      	   c2 AS (SELECT pt.id FROM c1 pt GROUP BY pt.id)
      SELECT id FROM c2 GROUP BY id
      UNION ALL
      SELECT id FROM c2;
      

      If the table alias (a) is removed in the CTE c1, the query works.

      When I try on my real query (and not in the abbreviated example above), I sometimes get the error message and sometimes mysqld crashes with the error log attached.

      Attachments

        1. error.log
          11 kB
          Walter van der Geest

        Issue Links

          Activity

            Yet if we try to execute the query

            with c1 as (select id from t1 where f(id)=id group by id), 
                    c2 as (select id from c1 as pt group by id)
            select id from c2 as s1 union select id from c2 as s2;
            

            that uses the stored function

            create  function f(in_id int) returns integer
            return (select id from t1 where t1.id = in_id);
            

            the above inconsistency brings us to an assertion failure.
            Let's figure out how it happens.
            So after parsing LEX::query_tables starts the chain of TABLE_LIST objects in the following order:
            first the object for s1, then for t1(1), pt(1), s2, pt(2) and lastly the object for t1(2). Yet LEX::query_tables_last points to
            TABLE_LIST::next_global in the object for pt(2) rather than for in the last of object of the chain. Bare in mind that at this moment all names to CTE (s1, pt(1), s2, pt(2) has been already resolved.
            1. when executing open_tables()
            1.1. for each elements from the chain LEX::query_tables open_and_process_table() is called
            1.1.1. open_table() is called for t1(1)
            1.1.2. open_table() is called for t1(2)
            1.2 . open_and_process_routine() is called for each element from the list thd->lex->sroutines_list (with only one element)
            1.2.1 open_table() is called for t1(SF) used in the stored function.
            1..2.2. add_used_tables_to_table_list inserts the TABLE_LIST object for t1(SF) after the TABLE_LIST for pt(2) as the last element in the chain LEX::query_tables. The chain looks like this now s1, then for t1(1), pt(1), s2, pt(2), t1(SF).
            2. lock_tables() is called for the chain s1, t1(1), pt(1), s2, pt(2), t1(SF). It locks TABLE_LIST::table for t1(1), but does not lock TABLE_LIST::table for t1(2) as the latter is out of the chain.
            3. st_select_lex_unit::optimize() is called for the query. It sets TABLE_LIST::table->file->keyread for t1(1) and TABLE_LIST::table->file->keyread for t1(2) to 0 (the number of the index to be used when scanning the tables t1(1) and t1(2)).
            4. st_select_lex_unit::exec() is called for the query. The execution comes to the evaluation of the condition f(id)=id.
            4.1. open_table() is called for t1(SF) It sees that TABLE_LIST::table for t1(2) is not locked for usage and takes it to be used for t1(SF). When calling TABLE::init() it is noticed that TABLE_LIST::table->file->keyread for t1(2) is set to 0 while it should have been set to 64.

            igor Igor Babaev (Inactive) added a comment - Yet if we try to execute the query with c1 as ( select id from t1 where f(id)=id group by id), c2 as ( select id from c1 as pt group by id) select id from c2 as s1 union select id from c2 as s2; that uses the stored function create function f(in_id int ) returns integer return ( select id from t1 where t1.id = in_id); the above inconsistency brings us to an assertion failure. Let's figure out how it happens. So after parsing LEX::query_tables starts the chain of TABLE_LIST objects in the following order: first the object for s1, then for t1(1), pt(1), s2, pt(2) and lastly the object for t1(2). Yet LEX::query_tables_last points to TABLE_LIST::next_global in the object for pt(2) rather than for in the last of object of the chain. Bare in mind that at this moment all names to CTE (s1, pt(1), s2, pt(2) has been already resolved. 1. when executing open_tables() 1.1. for each elements from the chain LEX::query_tables open_and_process_table() is called 1.1.1. open_table() is called for t1(1) 1.1.2. open_table() is called for t1(2) 1.2 . open_and_process_routine() is called for each element from the list thd->lex->sroutines_list (with only one element) 1.2.1 open_table() is called for t1(SF) used in the stored function. 1..2.2. add_used_tables_to_table_list inserts the TABLE_LIST object for t1(SF) after the TABLE_LIST for pt(2) as the last element in the chain LEX::query_tables. The chain looks like this now s1, then for t1(1), pt(1), s2, pt(2), t1(SF). 2. lock_tables() is called for the chain s1, t1(1), pt(1), s2, pt(2), t1(SF). It locks TABLE_LIST::table for t1(1), but does not lock TABLE_LIST::table for t1(2) as the latter is out of the chain. 3. st_select_lex_unit::optimize() is called for the query. It sets TABLE_LIST::table->file->keyread for t1(1) and TABLE_LIST::table->file->keyread for t1(2) to 0 (the number of the index to be used when scanning the tables t1(1) and t1(2)). 4. st_select_lex_unit::exec() is called for the query. The execution comes to the evaluation of the condition f(id)=id. 4.1. open_table() is called for t1(SF) It sees that TABLE_LIST::table for t1(2) is not locked for usage and takes it to be used for t1(SF). When calling TABLE::init() it is noticed that TABLE_LIST::table->file->keyread for t1(2) is set to 0 while it should have been set to 64.

            Let's consider a slightly different query when the specification of c1 uses an alias for the table reference t1.

            with c1 as (select id from t1 as r where f(id)=id group by id), 
                    c2 as (select id from c1 as pt group by id)
            select id from c2 as s1 union select id from c2 as s2;
            

            After parsing we have the same inconsistency as for the previous query. LEX::query_tables points to the chain of TABLE_LIST objects for s1,r(1),pt(1),s2,pt(2),r(2) while LEX::query_tables_last points to TABLE_LIST::next_global for pt(2).
            When opening table for t1(SF) we come to the lines:

            1686     /*
            1687       No table in the locked tables list. In case of explicit LOCK TABLES
            1688       this can happen if a user did not include the table into the list.
            1689       In case of pre-locked mode locked tables list is generated automatically,
            1690       so we may only end up here if the table did not exist when
            1691       locked tables list was created.
            1692     */
            1693     if (thd->locked_tables_mode == LTM_PRELOCKED)
            1694       my_error(ER_NO_SUCH_TABLE, MYF(0), table_list->db, table_list->alias);
            1695     else
            1696       my_error(ER_TABLE_NOT_LOCKED, MYF(0), alias);
            

            because alias for t1(SF) differs from 'r'.
            Here the server reports the error message:

            Table 'test.t1' doesn't exist
            

            igor Igor Babaev (Inactive) added a comment - Let's consider a slightly different query when the specification of c1 uses an alias for the table reference t1. with c1 as ( select id from t1 as r where f(id)=id group by id), c2 as ( select id from c1 as pt group by id) select id from c2 as s1 union select id from c2 as s2; After parsing we have the same inconsistency as for the previous query. LEX::query_tables points to the chain of TABLE_LIST objects for s1,r(1),pt(1),s2,pt(2),r(2) while LEX::query_tables_last points to TABLE_LIST::next_global for pt(2). When opening table for t1(SF) we come to the lines: 1686 /* 1687 No table in the locked tables list. In case of explicit LOCK TABLES 1688 this can happen if a user did not include the table into the list. 1689 In case of pre-locked mode locked tables list is generated automatically, 1690 so we may only end up here if the table did not exist when 1691 locked tables list was created. 1692 */ 1693 if (thd->locked_tables_mode == LTM_PRELOCKED) 1694 my_error(ER_NO_SUCH_TABLE, MYF(0), table_list->db, table_list->alias); 1695 else 1696 my_error(ER_TABLE_NOT_LOCKED, MYF(0), alias); because alias for t1(SF) differs from 'r'. Here the server reports the error message: Table 'test.t1' doesn't exist

            Let's figure out how we get the above mentioned inconsistency for the query

            with c1 as (select id from t1 where id = id * id group by id), 
                    c2 as (select id from c1 as pt group by id)
            select id from c2 as s1 union select id from c2 as s2;
            

            At the moment when we have just finished parsing the query remaining still within the parser we call LEX::resolve_references_to_cte() with the chain LEX::query_tables containing TABLE_LIST objects for t1(1), pt(1), s1, s2. pt() and s1 are resolved against the selects specifying c1 and c2 correspondingly without any problem. Yet to resolve s2 we have to build a new copy of the select specifying c2. So we call With_element::clone_parsed_spec() with LEX for the main query as the first parameter old_lex and the TABLE_LIST object for s2 as the second parameter. At this moment s2 is already resolved against CTE c2.The function takes the specification. When parsing is finished LEX::query_tables contains only the TABLE_LIST object for pt(2). LEX::resolve_references_to_cte() is called for this chain. Yet it fails to resolve pt(2). Then we see that the chain pointing to the TABLE_LIST object for pt(2) is appended to the chain t1(1), pt(1), s1, s2. pt(2). After this LEX::resolve_references_to_cte() is called for the chain starting from pt(2). The call resolves pt(2) against c1, sees that this is the second usage of c1 and calls With_element::clone_parsed_spec to create a copy of the specification of c1 for pt(2). This call parses the specification of c1, gets the chain containing the TABLE_LIST object for t1(2) and calls LEX::resolve_references_to_cte() that does not resolve anything. After this the chain with the TABLE_LIST object for t1(2) is inserted after the TABLE_LIST object for pt(1) and in this way it appears in the chain LEX->query_tables for the main query. The function also updates LEX::query_tables_last, but it does it in the LEX structure created for the copy of the specification of c2. So LEX::query_tables_last used for the main query is not updated and remains the same as it was after the last update.

            igor Igor Babaev (Inactive) added a comment - Let's figure out how we get the above mentioned inconsistency for the query with c1 as ( select id from t1 where id = id * id group by id), c2 as ( select id from c1 as pt group by id) select id from c2 as s1 union select id from c2 as s2; At the moment when we have just finished parsing the query remaining still within the parser we call LEX::resolve_references_to_cte() with the chain LEX::query_tables containing TABLE_LIST objects for t1(1), pt(1), s1, s2. pt() and s1 are resolved against the selects specifying c1 and c2 correspondingly without any problem. Yet to resolve s2 we have to build a new copy of the select specifying c2. So we call With_element::clone_parsed_spec() with LEX for the main query as the first parameter old_lex and the TABLE_LIST object for s2 as the second parameter. At this moment s2 is already resolved against CTE c2.The function takes the specification. When parsing is finished LEX::query_tables contains only the TABLE_LIST object for pt(2). LEX::resolve_references_to_cte() is called for this chain. Yet it fails to resolve pt(2). Then we see that the chain pointing to the TABLE_LIST object for pt(2) is appended to the chain t1(1), pt(1), s1, s2. pt(2). After this LEX::resolve_references_to_cte() is called for the chain starting from pt(2). The call resolves pt(2) against c1, sees that this is the second usage of c1 and calls With_element::clone_parsed_spec to create a copy of the specification of c1 for pt(2). This call parses the specification of c1, gets the chain containing the TABLE_LIST object for t1(2) and calls LEX::resolve_references_to_cte() that does not resolve anything. After this the chain with the TABLE_LIST object for t1(2) is inserted after the TABLE_LIST object for pt(1) and in this way it appears in the chain LEX->query_tables for the main query. The function also updates LEX::query_tables_last, but it does it in the LEX structure created for the copy of the specification of c2. So LEX::query_tables_last used for the main query is not updated and remains the same as it was after the last update.

            OK to push

            sanja Oleksandr Byelkin added a comment - OK to push

            A fix for this bug was pushed into 10.2. It should be merged upstream as it is.

            igor Igor Babaev (Inactive) added a comment - A fix for this bug was pushed into 10.2. It should be merged upstream as it is.

            People

              igor Igor Babaev (Inactive)
              vdg@plan-bee.ch Walter van der Geest
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.