Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-33010

Crash when pushing condition with CHARSET()/COERCIBILITY() into derived table

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Fixed
    • 11.1.2, 11.2.2, 10.4(EOL), 10.5, 10.6, 10.11, 11.0(EOL), 11.1(EOL), 11.2(EOL), 11.3(EOL)
    • 10.5.26, 10.6.19, 10.11.9, 11.1.6, 11.2.5, 11.4.3
    • Optimizer, Server
    • None
    • Server version: 11.1.2-MariaDB-1:11.1.2+maria~ubu2204 source revision: 9bc25d98209df6810f7a7d5e7dd3ae677a313ab5
      key_buffer_size=134217728
      read_buffer_size=131072
      max_used_connections=1
      max_threads=153
      thread_count=1

    Description

      I first run the following statements to create tables.

      create table t1 (`c1` bigint, key (`c1`));
      insert into t1 values (-2);
      create table t2 ( `c2` double unsigned );
      insert into t2 values (12.991);
      

      After creating tables, I repeat executing the following statement through the MariaDB CLI.

      SELECT f6 FROM (SELECT DISTINCT `c1` AS `f6` FROM `t1`) AS `t1`
               JOIN (SELECT CHARSET(`c2`) AS `f5` FROM `t2`) AS `t3`
                    ON (EXISTS (SELECT `c2` FROM `t2`)) AND
                       (NOT ((CHARSET(`f5`)) NOT BETWEEN `f6` AND `f6`));
      

      After repeating several times (maybe 20 times), I lost the connection to the server.

      Or try to run the following EXPLAIN query twice, I can get the same error.

      EXPLAIN SELECT f6 FROM (SELECT DISTINCT `c1` AS `f6` FROM `t1`) AS `t1`
               JOIN (SELECT CHARSET(`c2`) AS `f5` FROM `t2`) AS `t3`
                    ON (EXISTS (SELECT `c2` FROM `t2`)) AND
                       (NOT ((CHARSET(`f5`)) NOT BETWEEN `f6` AND `f6`));
      

      The following trace is gotten from the log.

      stack_bottom = 0x7f4c90082c38 thread_stack 0x49000
      Printing to addr2line failed
      mariadbd(my_print_stacktrace+0x32)[0x563dd110a7c2]
      mariadbd(handle_fatal_signal+0x488)[0x563dd0be3cf8]
      /lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f4ca74bd520]
      mariadbd(_Z15optimize_keyuseP4JOINP16st_dynamic_array+0x148)[0x563dd09b5088]
      mariadbd(+0x8ad551)[0x563dd09eb551]
      mariadbd(_ZN4JOIN14optimize_innerEv+0x1322)[0x563dd09f07e2]
      mariadbd(_ZN4JOIN8optimizeEv+0xda)[0x563dd09f0e2a]
      mariadbd(+0x7ec60c)[0x563dd092a60c]
      mariadbd(_Z27mysql_handle_single_derivedP3LEXP10TABLE_LISTj+0x95)[0x563dd0929e35]
      mariadbd(_ZN4JOIN14optimize_innerEv+0xb27)[0x563dd09effe7]
      mariadbd(_ZN4JOIN8optimizeEv+0xda)[0x563dd09f0e2a]
      mariadbd(_Z12mysql_selectP3THDP10TABLE_LISTR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xd1)[0x563dd09f0f21]
      mariadbd(_Z13handle_selectP3THDP3LEXP13select_resulty+0x154)[0x563dd09f1774]
      mariadbd(+0x826f55)[0x563dd0964f55]
      mariadbd(_Z21mysql_execute_commandP3THDb+0x419e)[0x563dd0973f0e]
      mariadbd(_Z11mysql_parseP3THDPcjP12Parser_state+0x1e7)[0x563dd0975237]
      mariadbd(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x14bd)[0x563dd0977a1d]
      mariadbd(_Z10do_commandP3THDb+0x138)[0x563dd0979818]
      mariadbd(_Z24do_handle_one_connectionP7CONNECTb+0x3bf)[0x563dd0aa13af]
      mariadbd(handle_one_connection+0x5d)[0x563dd0aa16fd]
      mariadbd(+0xcd1906)[0x563dd0e0f906]
      /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3)[0x7f4ca750fac3]
      /lib/x86_64-linux-gnu/libc.so.6(+0x126a40)[0x7f4ca75a1a40]
      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      

      Attachments

        Issue Links

          Activity

            John Jove John Jove created issue -
            John Jove John Jove added a comment -

            When removing the key build in column c1 in table t1, the error is gone.

            John Jove John Jove added a comment - When removing the key build in column c1 in table t1, the error is gone.
            John Jove John Jove made changes -
            Field Original Value New Value
            Affects Version/s 11.2.2 [ 29035 ]
            John Jove John Jove made changes -
            Description I first run the following statements to create tables.
            {code:sql}
            create table t1 (`c1` bigint, key (`c1`));
            insert into t1 values (-2);
            create table t2 ( `c2` double unsigned );
            insert into t2 values (12.991);
            {code}
            After creating tables, I repeat executing the following statement through the MariaDB CLI.
            {code:sql}
            SELECT f6 FROM (SELECT DISTINCT `c1` AS `f6` FROM `t1`) AS `t1`
                     JOIN (SELECT CHARSET(`c2`) AS `f5` FROM `t2`) AS `t3`
                          ON (EXISTS (SELECT `c2` FROM `t2`)) AND
                             (NOT ((CHARSET(`f5`)) NOT BETWEEN `f6` AND `f6`));
            {code}
            After repeating several times (maybe 20 times), I lost the connection to the server.

            The following trace is gotten from the log.
            {code}
            stack_bottom = 0x7f4c90082c38 thread_stack 0x49000
            Printing to addr2line failed
            mariadbd(my_print_stacktrace+0x32)[0x563dd110a7c2]
            mariadbd(handle_fatal_signal+0x488)[0x563dd0be3cf8]
            /lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f4ca74bd520]
            mariadbd(_Z15optimize_keyuseP4JOINP16st_dynamic_array+0x148)[0x563dd09b5088]
            mariadbd(+0x8ad551)[0x563dd09eb551]
            mariadbd(_ZN4JOIN14optimize_innerEv+0x1322)[0x563dd09f07e2]
            mariadbd(_ZN4JOIN8optimizeEv+0xda)[0x563dd09f0e2a]
            mariadbd(+0x7ec60c)[0x563dd092a60c]
            mariadbd(_Z27mysql_handle_single_derivedP3LEXP10TABLE_LISTj+0x95)[0x563dd0929e35]
            mariadbd(_ZN4JOIN14optimize_innerEv+0xb27)[0x563dd09effe7]
            mariadbd(_ZN4JOIN8optimizeEv+0xda)[0x563dd09f0e2a]
            mariadbd(_Z12mysql_selectP3THDP10TABLE_LISTR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xd1)[0x563dd09f0f21]
            mariadbd(_Z13handle_selectP3THDP3LEXP13select_resulty+0x154)[0x563dd09f1774]
            mariadbd(+0x826f55)[0x563dd0964f55]
            mariadbd(_Z21mysql_execute_commandP3THDb+0x419e)[0x563dd0973f0e]
            mariadbd(_Z11mysql_parseP3THDPcjP12Parser_state+0x1e7)[0x563dd0975237]
            mariadbd(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x14bd)[0x563dd0977a1d]
            mariadbd(_Z10do_commandP3THDb+0x138)[0x563dd0979818]
            mariadbd(_Z24do_handle_one_connectionP7CONNECTb+0x3bf)[0x563dd0aa13af]
            mariadbd(handle_one_connection+0x5d)[0x563dd0aa16fd]
            mariadbd(+0xcd1906)[0x563dd0e0f906]
            /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3)[0x7f4ca750fac3]
            /lib/x86_64-linux-gnu/libc.so.6(+0x126a40)[0x7f4ca75a1a40]
            Trying to get some variables.
            Some pointers may be invalid and cause the dump to abort.
            {code}
            I first run the following statements to create tables.
            {code:sql}
            create table t1 (`c1` bigint, key (`c1`));
            insert into t1 values (-2);
            create table t2 ( `c2` double unsigned );
            insert into t2 values (12.991);
            {code}
            After creating tables, I repeat executing the following statement through the MariaDB CLI.
            {code:sql}
            SELECT f6 FROM (SELECT DISTINCT `c1` AS `f6` FROM `t1`) AS `t1`
                     JOIN (SELECT CHARSET(`c2`) AS `f5` FROM `t2`) AS `t3`
                          ON (EXISTS (SELECT `c2` FROM `t2`)) AND
                             (NOT ((CHARSET(`f5`)) NOT BETWEEN `f6` AND `f6`));
            {code}
            After repeating several times (maybe 20 times), I lost the connection to the server.

            Or try to EXPLAIN the query twice, I can get the same error.
            {code:sql}
            EXPLAIN SELECT f6 FROM (SELECT DISTINCT `c1` AS `f6` FROM `t1`) AS `t1`
                     JOIN (SELECT CHARSET(`c2`) AS `f5` FROM `t2`) AS `t3`
                          ON (EXISTS (SELECT `c2` FROM `t2`)) AND
                             (NOT ((CHARSET(`f5`)) NOT BETWEEN `f6` AND `f6`));
            {code}

            The following trace is gotten from the log.
            {code}
            stack_bottom = 0x7f4c90082c38 thread_stack 0x49000
            Printing to addr2line failed
            mariadbd(my_print_stacktrace+0x32)[0x563dd110a7c2]
            mariadbd(handle_fatal_signal+0x488)[0x563dd0be3cf8]
            /lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f4ca74bd520]
            mariadbd(_Z15optimize_keyuseP4JOINP16st_dynamic_array+0x148)[0x563dd09b5088]
            mariadbd(+0x8ad551)[0x563dd09eb551]
            mariadbd(_ZN4JOIN14optimize_innerEv+0x1322)[0x563dd09f07e2]
            mariadbd(_ZN4JOIN8optimizeEv+0xda)[0x563dd09f0e2a]
            mariadbd(+0x7ec60c)[0x563dd092a60c]
            mariadbd(_Z27mysql_handle_single_derivedP3LEXP10TABLE_LISTj+0x95)[0x563dd0929e35]
            mariadbd(_ZN4JOIN14optimize_innerEv+0xb27)[0x563dd09effe7]
            mariadbd(_ZN4JOIN8optimizeEv+0xda)[0x563dd09f0e2a]
            mariadbd(_Z12mysql_selectP3THDP10TABLE_LISTR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xd1)[0x563dd09f0f21]
            mariadbd(_Z13handle_selectP3THDP3LEXP13select_resulty+0x154)[0x563dd09f1774]
            mariadbd(+0x826f55)[0x563dd0964f55]
            mariadbd(_Z21mysql_execute_commandP3THDb+0x419e)[0x563dd0973f0e]
            mariadbd(_Z11mysql_parseP3THDPcjP12Parser_state+0x1e7)[0x563dd0975237]
            mariadbd(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x14bd)[0x563dd0977a1d]
            mariadbd(_Z10do_commandP3THDb+0x138)[0x563dd0979818]
            mariadbd(_Z24do_handle_one_connectionP7CONNECTb+0x3bf)[0x563dd0aa13af]
            mariadbd(handle_one_connection+0x5d)[0x563dd0aa16fd]
            mariadbd(+0xcd1906)[0x563dd0e0f906]
            /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3)[0x7f4ca750fac3]
            /lib/x86_64-linux-gnu/libc.so.6(+0x126a40)[0x7f4ca75a1a40]
            Trying to get some variables.
            Some pointers may be invalid and cause the dump to abort.
            {code}
            John Jove John Jove made changes -
            Description I first run the following statements to create tables.
            {code:sql}
            create table t1 (`c1` bigint, key (`c1`));
            insert into t1 values (-2);
            create table t2 ( `c2` double unsigned );
            insert into t2 values (12.991);
            {code}
            After creating tables, I repeat executing the following statement through the MariaDB CLI.
            {code:sql}
            SELECT f6 FROM (SELECT DISTINCT `c1` AS `f6` FROM `t1`) AS `t1`
                     JOIN (SELECT CHARSET(`c2`) AS `f5` FROM `t2`) AS `t3`
                          ON (EXISTS (SELECT `c2` FROM `t2`)) AND
                             (NOT ((CHARSET(`f5`)) NOT BETWEEN `f6` AND `f6`));
            {code}
            After repeating several times (maybe 20 times), I lost the connection to the server.

            Or try to EXPLAIN the query twice, I can get the same error.
            {code:sql}
            EXPLAIN SELECT f6 FROM (SELECT DISTINCT `c1` AS `f6` FROM `t1`) AS `t1`
                     JOIN (SELECT CHARSET(`c2`) AS `f5` FROM `t2`) AS `t3`
                          ON (EXISTS (SELECT `c2` FROM `t2`)) AND
                             (NOT ((CHARSET(`f5`)) NOT BETWEEN `f6` AND `f6`));
            {code}

            The following trace is gotten from the log.
            {code}
            stack_bottom = 0x7f4c90082c38 thread_stack 0x49000
            Printing to addr2line failed
            mariadbd(my_print_stacktrace+0x32)[0x563dd110a7c2]
            mariadbd(handle_fatal_signal+0x488)[0x563dd0be3cf8]
            /lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f4ca74bd520]
            mariadbd(_Z15optimize_keyuseP4JOINP16st_dynamic_array+0x148)[0x563dd09b5088]
            mariadbd(+0x8ad551)[0x563dd09eb551]
            mariadbd(_ZN4JOIN14optimize_innerEv+0x1322)[0x563dd09f07e2]
            mariadbd(_ZN4JOIN8optimizeEv+0xda)[0x563dd09f0e2a]
            mariadbd(+0x7ec60c)[0x563dd092a60c]
            mariadbd(_Z27mysql_handle_single_derivedP3LEXP10TABLE_LISTj+0x95)[0x563dd0929e35]
            mariadbd(_ZN4JOIN14optimize_innerEv+0xb27)[0x563dd09effe7]
            mariadbd(_ZN4JOIN8optimizeEv+0xda)[0x563dd09f0e2a]
            mariadbd(_Z12mysql_selectP3THDP10TABLE_LISTR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xd1)[0x563dd09f0f21]
            mariadbd(_Z13handle_selectP3THDP3LEXP13select_resulty+0x154)[0x563dd09f1774]
            mariadbd(+0x826f55)[0x563dd0964f55]
            mariadbd(_Z21mysql_execute_commandP3THDb+0x419e)[0x563dd0973f0e]
            mariadbd(_Z11mysql_parseP3THDPcjP12Parser_state+0x1e7)[0x563dd0975237]
            mariadbd(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x14bd)[0x563dd0977a1d]
            mariadbd(_Z10do_commandP3THDb+0x138)[0x563dd0979818]
            mariadbd(_Z24do_handle_one_connectionP7CONNECTb+0x3bf)[0x563dd0aa13af]
            mariadbd(handle_one_connection+0x5d)[0x563dd0aa16fd]
            mariadbd(+0xcd1906)[0x563dd0e0f906]
            /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3)[0x7f4ca750fac3]
            /lib/x86_64-linux-gnu/libc.so.6(+0x126a40)[0x7f4ca75a1a40]
            Trying to get some variables.
            Some pointers may be invalid and cause the dump to abort.
            {code}
            I first run the following statements to create tables.
            {code:sql}
            create table t1 (`c1` bigint, key (`c1`));
            insert into t1 values (-2);
            create table t2 ( `c2` double unsigned );
            insert into t2 values (12.991);
            {code}
            After creating tables, I repeat executing the following statement through the MariaDB CLI.
            {code:sql}
            SELECT f6 FROM (SELECT DISTINCT `c1` AS `f6` FROM `t1`) AS `t1`
                     JOIN (SELECT CHARSET(`c2`) AS `f5` FROM `t2`) AS `t3`
                          ON (EXISTS (SELECT `c2` FROM `t2`)) AND
                             (NOT ((CHARSET(`f5`)) NOT BETWEEN `f6` AND `f6`));
            {code}
            After repeating several times (maybe 20 times), I lost the connection to the server.

            Or try to run the following EXPLAIN query twice, I can get the same error.
            {code:sql}
            EXPLAIN SELECT f6 FROM (SELECT DISTINCT `c1` AS `f6` FROM `t1`) AS `t1`
                     JOIN (SELECT CHARSET(`c2`) AS `f5` FROM `t2`) AS `t3`
                          ON (EXISTS (SELECT `c2` FROM `t2`)) AND
                             (NOT ((CHARSET(`f5`)) NOT BETWEEN `f6` AND `f6`));
            {code}

            The following trace is gotten from the log.
            {code}
            stack_bottom = 0x7f4c90082c38 thread_stack 0x49000
            Printing to addr2line failed
            mariadbd(my_print_stacktrace+0x32)[0x563dd110a7c2]
            mariadbd(handle_fatal_signal+0x488)[0x563dd0be3cf8]
            /lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f4ca74bd520]
            mariadbd(_Z15optimize_keyuseP4JOINP16st_dynamic_array+0x148)[0x563dd09b5088]
            mariadbd(+0x8ad551)[0x563dd09eb551]
            mariadbd(_ZN4JOIN14optimize_innerEv+0x1322)[0x563dd09f07e2]
            mariadbd(_ZN4JOIN8optimizeEv+0xda)[0x563dd09f0e2a]
            mariadbd(+0x7ec60c)[0x563dd092a60c]
            mariadbd(_Z27mysql_handle_single_derivedP3LEXP10TABLE_LISTj+0x95)[0x563dd0929e35]
            mariadbd(_ZN4JOIN14optimize_innerEv+0xb27)[0x563dd09effe7]
            mariadbd(_ZN4JOIN8optimizeEv+0xda)[0x563dd09f0e2a]
            mariadbd(_Z12mysql_selectP3THDP10TABLE_LISTR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xd1)[0x563dd09f0f21]
            mariadbd(_Z13handle_selectP3THDP3LEXP13select_resulty+0x154)[0x563dd09f1774]
            mariadbd(+0x826f55)[0x563dd0964f55]
            mariadbd(_Z21mysql_execute_commandP3THDb+0x419e)[0x563dd0973f0e]
            mariadbd(_Z11mysql_parseP3THDPcjP12Parser_state+0x1e7)[0x563dd0975237]
            mariadbd(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x14bd)[0x563dd0977a1d]
            mariadbd(_Z10do_commandP3THDb+0x138)[0x563dd0979818]
            mariadbd(_Z24do_handle_one_connectionP7CONNECTb+0x3bf)[0x563dd0aa13af]
            mariadbd(handle_one_connection+0x5d)[0x563dd0aa16fd]
            mariadbd(+0xcd1906)[0x563dd0e0f906]
            /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3)[0x7f4ca750fac3]
            /lib/x86_64-linux-gnu/libc.so.6(+0x126a40)[0x7f4ca75a1a40]
            Trying to get some variables.
            Some pointers may be invalid and cause the dump to abort.
            {code}
            alice Alice Sherepa made changes -
            alice Alice Sherepa made changes -
            Affects Version/s 10.4 [ 22408 ]
            Affects Version/s 10.5 [ 23123 ]
            Affects Version/s 10.6 [ 24028 ]
            Affects Version/s 10.11 [ 27614 ]
            Affects Version/s 11.0 [ 28320 ]
            Affects Version/s 11.1 [ 28549 ]
            Affects Version/s 11.2 [ 28603 ]
            Affects Version/s 11.3 [ 28565 ]
            alice Alice Sherepa made changes -
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.6 [ 24028 ]
            Fix Version/s 10.11 [ 27614 ]
            Fix Version/s 11.0 [ 28320 ]
            Fix Version/s 11.1 [ 28549 ]
            Fix Version/s 11.2 [ 28603 ]
            Fix Version/s 11.3 [ 28565 ]
            alice Alice Sherepa added a comment - - edited

            Thank you!
            I repeated as described on 10.4-11.3 with InnoDB, not Myisam.
            On the release build it crashes after 20-30 executions of the query, while on debug on my machine that was ~9000 executions:

             --source include/have_innodb.inc
             
            create table t1 (`c1` bigint, key (`c1`)) engine=innodb;
            insert into t1 values (-2);
            create table t2 ( `c2` double unsigned );
            insert into t2 values (12.991);
             
             
            let $1= 10000;
            while ($1)
            {
              SELECT f6 FROM (SELECT DISTINCT `c1` AS `f6` FROM `t1`) AS `t1`
                     JOIN (SELECT CHARSET(`c2`) AS `f5` FROM `t2`) AS `t3`
                          ON (EXISTS (SELECT `c2` FROM `t2`)) AND
                             (NOT ((CHARSET(`f5`)) NOT BETWEEN `f6` AND `f6`));
              dec $1;
            }
            drop table t1,t2;
            

            231214 11:22:49 [ERROR] mysqld got signal 11 ;
             
            Server version: 10.4.33-MariaDB-debug-log source revision: 9f5078a1d79031c4a781d378af18df9c8c9d2321
             
            sql/signal_handler.cc:235(handle_fatal_signal)[0x558d689effd3]
            sigaction.c:0(__restore_rt)[0x7f9a6f82c420]
            sql/sql_select.cc:7139(optimize_keyuse(JOIN*, st_dynamic_array*))[0x558d6831ceba]
            sql/sql_select.cc:5708(make_join_statistics(JOIN*, List<TABLE_LIST>&, st_dynamic_array*))[0x558d6830fb03]
            sql/sql_select.cc:2388(JOIN::optimize_inner())[0x558d682ed791]
            sql/sql_select.cc:1731(JOIN::optimize())[0x558d682e67a7]
            sql/sql_derived.cc:1029(mysql_derived_optimize(THD*, LEX*, TABLE_LIST*))[0x558d681529ef]
            sql/sql_derived.cc:200(mysql_handle_single_derived(LEX*, TABLE_LIST*, unsigned int))[0x558d6814d19a]
            sql/sql_select.cc:2221(JOIN::optimize_inner())[0x558d682ebd0c]
            sql/sql_select.cc:1731(JOIN::optimize())[0x558d682e67a7]
            sql/sql_select.cc:4832(mysql_select(THD*, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*))[0x558d68307791]
            sql/sql_select.cc:442(handle_select(THD*, LEX*, select_result*, unsigned long))[0x558d682d81f8]
            sql/sql_parse.cc:6523(execute_sqlcom_select(THD*, TABLE_LIST*))[0x558d6823f1fd]
            sql/sql_parse.cc:3980(mysql_execute_command(THD*))[0x558d6822c81d]
            sql/sql_parse.cc:8062(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x558d68248779]
            sql/sql_parse.cc:1860(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x558d6821e925]
            sql/sql_parse.cc:1378(do_command(THD*))[0x558d6821b450]
            sql/sql_connect.cc:1419(do_handle_one_connection(CONNECT*))[0x558d686304c6]
            sql/sql_connect.cc:1324(handle_one_connection)[0x558d6862fd6a]
            perfschema/pfs.cc:1871(pfs_spawn_thread)[0x558d692cde4c]
            nptl/pthread_create.c:478(start_thread)[0x7f9a6f820609]
             
            Query (0x62b0000a1290): SELECT f6 FROM (SELECT DISTINCT `c1` AS `f6` FROM `t1`) AS `t1`
                     JOIN (SELECT CHARSET(`c2`) AS `f5` FROM `t2`) AS `t3`
                          ON (EXISTS (SELECT `c2` FROM `t2`)) AND
            (NOT ((CHARSET(`f5`)) NOT BETWEEN `f6` AND `f6`))
            

            alice Alice Sherepa added a comment - - edited Thank you! I repeated as described on 10.4-11.3 with InnoDB, not Myisam. On the release build it crashes after 20-30 executions of the query, while on debug on my machine that was ~9000 executions: --source include/have_innodb.inc   create table t1 (`c1` bigint , key (`c1`)) engine=innodb; insert into t1 values (-2); create table t2 ( `c2` double unsigned ); insert into t2 values (12.991);     let $1= 10000; while ($1) { SELECT f6 FROM ( SELECT DISTINCT `c1` AS `f6` FROM `t1`) AS `t1` JOIN ( SELECT CHARSET(`c2`) AS `f5` FROM `t2`) AS `t3` ON (EXISTS ( SELECT `c2` FROM `t2`)) AND ( NOT ((CHARSET(`f5`)) NOT BETWEEN `f6` AND `f6`)); dec $1; } drop table t1,t2; 231214 11:22:49 [ERROR] mysqld got signal 11 ;   Server version: 10.4.33-MariaDB-debug-log source revision: 9f5078a1d79031c4a781d378af18df9c8c9d2321   sql/signal_handler.cc:235(handle_fatal_signal)[0x558d689effd3] sigaction.c:0(__restore_rt)[0x7f9a6f82c420] sql/sql_select.cc:7139(optimize_keyuse(JOIN*, st_dynamic_array*))[0x558d6831ceba] sql/sql_select.cc:5708(make_join_statistics(JOIN*, List<TABLE_LIST>&, st_dynamic_array*))[0x558d6830fb03] sql/sql_select.cc:2388(JOIN::optimize_inner())[0x558d682ed791] sql/sql_select.cc:1731(JOIN::optimize())[0x558d682e67a7] sql/sql_derived.cc:1029(mysql_derived_optimize(THD*, LEX*, TABLE_LIST*))[0x558d681529ef] sql/sql_derived.cc:200(mysql_handle_single_derived(LEX*, TABLE_LIST*, unsigned int))[0x558d6814d19a] sql/sql_select.cc:2221(JOIN::optimize_inner())[0x558d682ebd0c] sql/sql_select.cc:1731(JOIN::optimize())[0x558d682e67a7] sql/sql_select.cc:4832(mysql_select(THD*, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*))[0x558d68307791] sql/sql_select.cc:442(handle_select(THD*, LEX*, select_result*, unsigned long))[0x558d682d81f8] sql/sql_parse.cc:6523(execute_sqlcom_select(THD*, TABLE_LIST*))[0x558d6823f1fd] sql/sql_parse.cc:3980(mysql_execute_command(THD*))[0x558d6822c81d] sql/sql_parse.cc:8062(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x558d68248779] sql/sql_parse.cc:1860(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x558d6821e925] sql/sql_parse.cc:1378(do_command(THD*))[0x558d6821b450] sql/sql_connect.cc:1419(do_handle_one_connection(CONNECT*))[0x558d686304c6] sql/sql_connect.cc:1324(handle_one_connection)[0x558d6862fd6a] perfschema/pfs.cc:1871(pfs_spawn_thread)[0x558d692cde4c] nptl/pthread_create.c:478(start_thread)[0x7f9a6f820609]   Query (0x62b0000a1290): SELECT f6 FROM (SELECT DISTINCT `c1` AS `f6` FROM `t1`) AS `t1` JOIN (SELECT CHARSET(`c2`) AS `f5` FROM `t2`) AS `t3` ON (EXISTS (SELECT `c2` FROM `t2`)) AND (NOT ((CHARSET(`f5`)) NOT BETWEEN `f6` AND `f6`))
            alice Alice Sherepa made changes -
            Status Open [ 1 ] Confirmed [ 10101 ]
            alice Alice Sherepa made changes -
            Assignee Sergei Petrunia [ psergey ]
            alice Alice Sherepa made changes -
            Component/s Optimizer [ 10200 ]
            psergei Sergei Petrunia made changes -
            Assignee Sergei Petrunia [ psergey ] Oleg Smirnov [ JIRAUSER50405 ]
            oleg.smirnov Oleg Smirnov made changes -
            Status Confirmed [ 10101 ] In Progress [ 3 ]
            oleg.smirnov Oleg Smirnov made changes -
            Comment [ Simplified test case (crashes consistently, no need to run a cycle):
            {code}
            EXPLAIN EXTENDED
                 SELECT dt1_c1 FROM
                   (SELECT c1 AS dt1_c1 FROM t1) AS dt1
                   JOIN
                   (SELECT 1 AS dt2_c2 FROM t2) AS dt2
                      ON CHARSET(dt2_c2) BETWEEN dt1_c1 AND dt1_c1;

            Warnings:
            Note 1003
            /* select#1 */ select `dt1`.`dt1_c1` AS `dt1_c1`
               from
               (/* select#2 */ select `test`.`t1`.`c1` AS `dt1_c1` from `test`.`t1`
                 where <cache>(charset(1)) between `test`.`t1`.`c1` and `test`.`t1`.`c1`) `dt1`
               where <cache>(charset(1)) between `dt1`.`dt1_c1` and `dt1`.`dt1_c1`
            {code}
             * Note: rewritten query text in the warning is obtained by disabling the {{optimize_keyuse()}} call.

            {{<cache>(charset(1)) between `test`.`t1`.`c1` and `test`.`t1`.`c1`}} is shown twice in the output, doesn't it look strange?

            Observation: replacing {{CHARSET()}} with {{LENGTH()}} eliminates the crash, and the execution doesn't have duplicate call of the function:
            {code}
            EXPLAIN EXTENDED
            SELECT dt1_c1 FROM
            (SELECT c1 AS dt1_c1 FROM t1) AS dt1
            JOIN
            (SELECT 1 AS dt2_c2 FROM t2) AS dt2
            ON LENGTH(dt2_c2) BETWEEN dt1_c1 AND dt1_c1;
            id select_type table type possible_keys key key_len ref rows filtered Extra
            1 PRIMARY <derived3> system NULL NULL NULL NULL 1 100.00
            1 PRIMARY <derived2> ref key0 key0 9 const 0 0.00 Using where
            3 DERIVED t2 system NULL NULL NULL NULL 1 100.00
            2 DERIVED t1 index NULL c1 9 NULL 1 100.00 Using index
            Warnings:
            Note 1003
            /* select#1 */ select `dt1`.`dt1_c1` AS `dt1_c1`
              from
              (/* select#2 */ select `test`.`t1`.`c1` AS `dt1_c1` from `test`.`t1`) `dt1`
                 where <cache>(octet_length(1)) between `dt1`.`dt1_c1` and `dt1`.`dt1_c1`
            {code} ]
            oleg.smirnov Oleg Smirnov added a comment -

            Below is the explain plan obtained from version 10.5:

            EXPLAIN EXTENDED 
            SELECT f6 FROM (SELECT DISTINCT `c1` AS `f6` FROM `t1`) AS `dt1`
                     JOIN (SELECT CHARSET(`c2`) AS `f5` FROM `t2`) AS `dt3`
                          ON (EXISTS (SELECT `c2` FROM `t2`)) AND
            (NOT ((CHARSET(`f5`)) NOT BETWEEN `f6` AND `f6`));
            id	select_type	table	type	possible_keys	key	key_len	ref	rows	filtered	Extra
            1	PRIMARY	<derived3>	system	NULL	NULL	NULL	NULL	1	100.00	
            1	PRIMARY	<derived2>	ref	key0	key0	9	const	0	0.00	Using where
            4	SUBQUERY	t2	system	NULL	NULL	NULL	NULL	1	100.00	
            3	DERIVED	t2	system	NULL	NULL	NULL	NULL	1	100.00	
            2	DERIVED	t1	range	c1	c1	9	NULL	2	50.00	Using where; Using index for group-by
            Warnings:
            Note	1003	/* select#1 */ select `dt1`.`f6` AS `f6` from (/* select#2 */ select distinct `test`.`t1`.`c1` AS `f6` from `test`.`t1` where <cache>(charset('binary')) between `test`.`t1`.`c1` and `test`.`t1`.`c1`) `dt1` where <cache>(charset('binary')) between `dt1`.`f6` and `dt1`.`f6`
            

            Call stack of the crash:

            1   optimize_keyuse             sql_select.cc    7189  0x555555f24f48 
            2   make_join_statistics        sql_select.cc    5751  0x555555f2085e 
            3   JOIN::optimize_inner        sql_select.cc    2386  0x555555f142f1 
            4   JOIN::optimize              sql_select.cc    1740  0x555555f11c24 
            5   mysql_derived_optimize      sql_derived.cc   1029  0x555555e60377 
            6   mysql_handle_single_derived sql_derived.cc   200   0x555555e5dd99 
            7   JOIN::optimize_inner        sql_select.cc    2204  0x555555f1387d 
            8   JOIN::optimize              sql_select.cc    1740  0x555555f11c24 
            9   mysql_select                sql_select.cc    4868  0x555555f1d66f 
            10  mysql_explain_union         sql_select.cc    28125 0x555555f5efa7 
            11  execute_sqlcom_select       sql_parse.cc     6356  0x555555ecada9 
            12  mysql_execute_command       sql_parse.cc     4022  0x555555ec1f97 
            ... <More>                                                            
            

            The crash happens because keyuse->used_tables == 2 when JOIN::table_count == 1. keyuse->used_tables == 2 due to keyuse->val->used_tables() == 2 and keyuse->val is Item_func_charset which is dbug_print'ed as "charset(t3.f5)".

            As I can see, there are two JOIN objects involved here, the inner one is initialized at mysql_derived_optimize (frame 5 of the call stack) as

            JOIN *join= derived->get_unit()->first_select()->join;
            

            The outer join processed at frame 7 has table_count == 2, so keyuse->val->used_tables() == 2 makes sense for that join.

            oleg.smirnov Oleg Smirnov added a comment - Below is the explain plan obtained from version 10.5: EXPLAIN EXTENDED SELECT f6 FROM (SELECT DISTINCT `c1` AS `f6` FROM `t1`) AS `dt1` JOIN (SELECT CHARSET(`c2`) AS `f5` FROM `t2`) AS `dt3` ON (EXISTS (SELECT `c2` FROM `t2`)) AND (NOT ((CHARSET(`f5`)) NOT BETWEEN `f6` AND `f6`)); id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY <derived3> system NULL NULL NULL NULL 1 100.00 1 PRIMARY <derived2> ref key0 key0 9 const 0 0.00 Using where 4 SUBQUERY t2 system NULL NULL NULL NULL 1 100.00 3 DERIVED t2 system NULL NULL NULL NULL 1 100.00 2 DERIVED t1 range c1 c1 9 NULL 2 50.00 Using where; Using index for group-by Warnings: Note 1003 /* select#1 */ select `dt1`.`f6` AS `f6` from (/* select#2 */ select distinct `test`.`t1`.`c1` AS `f6` from `test`.`t1` where <cache>(charset('binary')) between `test`.`t1`.`c1` and `test`.`t1`.`c1`) `dt1` where <cache>(charset('binary')) between `dt1`.`f6` and `dt1`.`f6` Call stack of the crash: 1 optimize_keyuse sql_select.cc 7189 0x555555f24f48 2 make_join_statistics sql_select.cc 5751 0x555555f2085e 3 JOIN::optimize_inner sql_select.cc 2386 0x555555f142f1 4 JOIN::optimize sql_select.cc 1740 0x555555f11c24 5 mysql_derived_optimize sql_derived.cc 1029 0x555555e60377 6 mysql_handle_single_derived sql_derived.cc 200 0x555555e5dd99 7 JOIN::optimize_inner sql_select.cc 2204 0x555555f1387d 8 JOIN::optimize sql_select.cc 1740 0x555555f11c24 9 mysql_select sql_select.cc 4868 0x555555f1d66f 10 mysql_explain_union sql_select.cc 28125 0x555555f5efa7 11 execute_sqlcom_select sql_parse.cc 6356 0x555555ecada9 12 mysql_execute_command sql_parse.cc 4022 0x555555ec1f97 ... <More> The crash happens because keyuse->used_tables == 2 when JOIN::table_count == 1 . keyuse->used_tables == 2 due to keyuse->val->used_tables() == 2 and keyuse->val is Item_func_charset which is dbug_print'ed as "charset(t3.f5)". As I can see, there are two JOIN objects involved here, the inner one is initialized at mysql_derived_optimize (frame 5 of the call stack) as JOIN *join= derived->get_unit()->first_select()->join; The outer join processed at frame 7 has table_count == 2 , so keyuse->val->used_tables() == 2 makes sense for that join.
            oleg.smirnov Oleg Smirnov added a comment - - edited

            Simplified test case (crashes consistently, no need to run a cycle):

            EXPLAIN EXTENDED 
                 SELECT dt1_c1 FROM 
                   (SELECT c1 AS dt1_c1 FROM t1) AS dt1
                   JOIN 
                   (SELECT 1 AS dt2_c2 FROM t2) AS dt2
                      ON CHARSET(dt2_c2) BETWEEN dt1_c1 AND dt1_c1;
             
            Warnings:
            Note	1003	
            /* select#1 */ select `dt1`.`dt1_c1` AS `dt1_c1` 
               from 
               (/* select#2 */ select `test`.`t1`.`c1` AS `dt1_c1` from `test`.`t1` 
                 where <cache>(charset(1)) between `test`.`t1`.`c1` and `test`.`t1`.`c1`) `dt1` 
               where <cache>(charset(1)) between `dt1`.`dt1_c1` and `dt1`.`dt1_c1`
            

            • Note: rewritten query text in the warning is obtained by disabling the optimize_keyuse() call.

            <cache>(charset(1)) between `test`.`t1`.`c1` and `test`.`t1`.`c1` is shown twice in the output, doesn't it look strange?

            Observation: replacing CHARSET() with LENGTH() eliminates the crash, and the execution doesn't have duplicate call of the function octet_length():

            EXPLAIN EXTENDED 
            SELECT dt1_c1 FROM 
            (SELECT c1 AS dt1_c1 FROM t1) AS dt1
            JOIN 
            (SELECT 1 AS dt2_c2 FROM t2) AS dt2
            ON LENGTH(dt2_c2) BETWEEN dt1_c1 AND dt1_c1;
            id	select_type	table	type	possible_keys	key	key_len	ref	rows	filtered	Extra
            1	PRIMARY	<derived3>	system	NULL	NULL	NULL	NULL	1	100.00	
            1	PRIMARY	<derived2>	ref	key0	key0	9	const	0	0.00	Using where
            3	DERIVED	t2	system	NULL	NULL	NULL	NULL	1	100.00	
            2	DERIVED	t1	index	NULL	c1	9	NULL	1	100.00	Using index
            Warnings:
            Note	1003	
            /* select#1 */ select `dt1`.`dt1_c1` AS `dt1_c1` 
              from 
              (/* select#2 */ select `test`.`t1`.`c1` AS `dt1_c1` from `test`.`t1`) `dt1` 
                 where <cache>(octet_length(1)) between `dt1`.`dt1_c1` and `dt1`.`dt1_c1`
            

            oleg.smirnov Oleg Smirnov added a comment - - edited Simplified test case (crashes consistently, no need to run a cycle): EXPLAIN EXTENDED SELECT dt1_c1 FROM (SELECT c1 AS dt1_c1 FROM t1) AS dt1 JOIN (SELECT 1 AS dt2_c2 FROM t2) AS dt2 ON CHARSET(dt2_c2) BETWEEN dt1_c1 AND dt1_c1;   Warnings: Note 1003 /* select#1 */ select `dt1`.`dt1_c1` AS `dt1_c1` from (/* select#2 */ select `test`.`t1`.`c1` AS `dt1_c1` from `test`.`t1` where <cache>(charset(1)) between `test`.`t1`.`c1` and `test`.`t1`.`c1`) `dt1` where <cache>(charset(1)) between `dt1`.`dt1_c1` and `dt1`.`dt1_c1` Note: rewritten query text in the warning is obtained by disabling the optimize_keyuse() call. <cache>(charset(1)) between `test`.`t1`.`c1` and `test`.`t1`.`c1` is shown twice in the output, doesn't it look strange? Observation: replacing CHARSET() with LENGTH() eliminates the crash, and the execution doesn't have duplicate call of the function octet_length() : EXPLAIN EXTENDED SELECT dt1_c1 FROM (SELECT c1 AS dt1_c1 FROM t1) AS dt1 JOIN (SELECT 1 AS dt2_c2 FROM t2) AS dt2 ON LENGTH(dt2_c2) BETWEEN dt1_c1 AND dt1_c1; id select_type table type possible_keys key key_len ref rows filtered Extra 1 PRIMARY <derived3> system NULL NULL NULL NULL 1 100.00 1 PRIMARY <derived2> ref key0 key0 9 const 0 0.00 Using where 3 DERIVED t2 system NULL NULL NULL NULL 1 100.00 2 DERIVED t1 index NULL c1 9 NULL 1 100.00 Using index Warnings: Note 1003 /* select#1 */ select `dt1`.`dt1_c1` AS `dt1_c1` from (/* select#2 */ select `test`.`t1`.`c1` AS `dt1_c1` from `test`.`t1`) `dt1` where <cache>(octet_length(1)) between `dt1`.`dt1_c1` and `dt1`.`dt1_c1`
            oleg.smirnov Oleg Smirnov made changes -
            Summary Crash MariaDB Server after repeating executing queries. Condition with CHARSET() erroneously pushed into derived table
            oleg.smirnov Oleg Smirnov made changes -
            Summary Condition with CHARSET() erroneously pushed into derived table Condition with CHARSET()/COERCIBILITY() erroneously pushed into derived table
            oleg.smirnov Oleg Smirnov added a comment -

            Looking at the result of rewrite of the problematic query

            /* select#1 */ select `dt1`.`dt1_c1` AS `dt1_c1` 
               from 
               (/* select#2 */ select `test`.`t1`.`c1` AS `dt1_c1` from `test`.`t1` 
                 where <cache>(charset(1)) between `test`.`t1`.`c1` and `test`.`t1`.`c1`) `dt1` 
               where <cache>(charset(1)) between `dt1`.`dt1_c1` and `dt1`.`dt1_c1`
            

            I can see that condition CHARSET(dt2_c2) BETWEEN dt1_c1 AND dt1_c1 has been pushed down into the derived table `dt1`, which must not have happened. Only conditions depending solely on the derived table can be pushed into, but CHARSET(dt2_c2) is dependent on `dt2` and must not be pushed down.

            The pushdown logic checks table dependencies but skips the check for const items. Const items are always considered as eligible for pushdown, and in this case Item_func_charset turns out to be erroneously const.

            This happens because Item_func_charset inherits from Item_func_expr_str_metadata which overrides const_item() in the following way, forcing the const property:

            bool const_item() const { return true; }
            

            That was introduced with the commit

            commit aa1002a35cefcc1851cbfb6b6b60463bda6f9aa3
            Author: Alexander Barkov <bar@mariadb.org>, Wed Sep 2 08:20:49 2015 +0400 (9 years ago)
             
            MDEV-8723 Wrong result for SELECT..WHERE COLLATION(a)='binary' AND a='a'
            

            The same was made for Item_func_coercibility, so the crash is reproducible if CHARSET() is replaced with COERCIBILITY().

            bar, psergei, please review the pull request.

            oleg.smirnov Oleg Smirnov added a comment - Looking at the result of rewrite of the problematic query /* select#1 */ select `dt1`.`dt1_c1` AS `dt1_c1` from (/* select#2 */ select `test`.`t1`.`c1` AS `dt1_c1` from `test`.`t1` where <cache>(charset(1)) between `test`.`t1`.`c1` and `test`.`t1`.`c1`) `dt1` where <cache>(charset(1)) between `dt1`.`dt1_c1` and `dt1`.`dt1_c1` I can see that condition CHARSET(dt2_c2) BETWEEN dt1_c1 AND dt1_c1 has been pushed down into the derived table `dt1`, which must not have happened. Only conditions depending solely on the derived table can be pushed into, but CHARSET(dt2_c2) is dependent on `dt2` and must not be pushed down. The pushdown logic checks table dependencies but skips the check for const items. Const items are always considered as eligible for pushdown, and in this case Item_func_charset turns out to be erroneously const. This happens because Item_func_charset inherits from Item_func_expr_str_metadata which overrides const_item() in the following way, forcing the const property: bool const_item() const { return true; } That was introduced with the commit commit aa1002a35cefcc1851cbfb6b6b60463bda6f9aa3 Author: Alexander Barkov <bar@mariadb.org>, Wed Sep 2 08:20:49 2015 +0400 (9 years ago)   MDEV-8723 Wrong result for SELECT..WHERE COLLATION(a)='binary' AND a='a' The same was made for Item_func_coercibility , so the crash is reproducible if CHARSET() is replaced with COERCIBILITY() . bar , psergei , please review the pull request .
            oleg.smirnov Oleg Smirnov made changes -
            Assignee Oleg Smirnov [ JIRAUSER50405 ] Alexander Barkov [ bar ]
            Status In Progress [ 3 ] In Review [ 10002 ]

            Hi oleg.smirnov,

            I think functions CHARSET() and COERCIBILITY should return const_item()==true. They are definitely constants during the query. They don't change their value per row.

            They are very close to DEFAULT(field) in this sense. I checked this script:

            create or replace table t1(a int default 1);
            insert into t1 values (1),(2);
            select * from t1 where a=default(a);
            

            and Item_default_value::const_item() returns true during the select statement.

            Can you please check the reported problem with Item_default_value? Does it also crash?

            bar Alexander Barkov added a comment - Hi oleg.smirnov , I think functions CHARSET() and COERCIBILITY should return const_item()==true. They are definitely constants during the query. They don't change their value per row. They are very close to DEFAULT(field) in this sense. I checked this script: create or replace table t1(a int default 1); insert into t1 values (1),(2); select * from t1 where a= default (a); and Item_default_value::const_item() returns true during the select statement. Can you please check the reported problem with Item_default_value? Does it also crash?
            bar Alexander Barkov made changes -
            Assignee Alexander Barkov [ bar ] Oleg Smirnov [ JIRAUSER50405 ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            oleg.smirnov Oleg Smirnov made changes -
            oleg.smirnov Oleg Smirnov added a comment -

            I filed a separate issue for DEFAULT() 'cause there is another call stack and it's reproducible not only on InnoDB but on MyISAM as well.

            oleg.smirnov Oleg Smirnov added a comment - I filed a separate issue for DEFAULT() 'cause there is another call stack and it's reproducible not only on InnoDB but on MyISAM as well.
            oleg.smirnov Oleg Smirnov added a comment -

            I agree with bar that CHARSET() and COERCIBILITY() should return const_item()==true. So this should be something related to the "condition pushdown into derived" optimization.

            A little bit more simplified test case:

            --source include/have_innodb.inc
            CREATE TABLE t1 (c1 INT, KEY (c1)) ENGINE=InnoDB;
            INSERT INTO t1 VALUES (1);
            CREATE TABLE t2 (c2 INT);
            INSERT INTO t2 VALUES (1);
             
            SET optimizer_switch='derived_merge=off';
            set optimizer_trace=1;
            EXPLAIN EXTENDED
                 SELECT dt1_c1 FROM 
                   (SELECT c1 AS dt1_c1 FROM t1) AS dt1
                   JOIN 
                   (SELECT 1 AS dt2_c2 FROM t2) AS dt2
                      ON CHARSET(dt2_c2) = dt1_c1;
            

            To reiterate: CHARSET(dt2_c2) = dt1_c1 is pushed down into dt1: SELECT c1 AS dt1_c1 FROM t1 WHERE CHARSET(dt2_c2) = dt1_c1 and CHARSET(dt2_c2) is a const item. The optimizer considers using KEY(c1) on t1 for value CHARSET(dt2_c2) and performs some operations at optimize_keyuse(). Since CHARSET(dt2_c2)::used_tables() == 2 and the dt1's JOIN::table_count == 1, a crash occurs:

            optimize_keyuse()

            if (keyuse->used_tables &
            	(map= (keyuse->used_tables & ~join->const_table_map &
            	       ~OUTER_REF_TABLE_BIT)))
                {
                  uint n_tables= my_count_bits(map);
                  if (n_tables == 1)			// Only one table
                  {
                    DBUG_ASSERT(!(map & PSEUDO_TABLE_BITS)); // Must be a real table
                    Table_map_iterator it(map);
                    int tablenr= it.next_bit();
                    DBUG_ASSERT(tablenr != Table_map_iterator::BITMAP_END);
            	TABLE *tmp_table=join->table[tablenr];
                    if (tmp_table) // already created
                      keyuse->ref_table_rows= MY_MAX(tmp_table->file->stats.records, 100);
                  }
                }
            

            The first idea: condition with CHARSET(dt2_c2) must not be pushed into dt1 (but why not if it's const?)
            Second idea: shouldn't we have JOIN::const_table_map == 2 after the pushdown? That eliminates the crash at optimize_keyuse().

            oleg.smirnov Oleg Smirnov added a comment - I agree with bar that CHARSET() and COERCIBILITY() should return const_item()==true. So this should be something related to the "condition pushdown into derived" optimization. A little bit more simplified test case: --source include/have_innodb.inc CREATE TABLE t1 (c1 INT, KEY (c1)) ENGINE=InnoDB; INSERT INTO t1 VALUES (1); CREATE TABLE t2 (c2 INT); INSERT INTO t2 VALUES (1);   SET optimizer_switch='derived_merge=off'; set optimizer_trace=1; EXPLAIN EXTENDED SELECT dt1_c1 FROM (SELECT c1 AS dt1_c1 FROM t1) AS dt1 JOIN (SELECT 1 AS dt2_c2 FROM t2) AS dt2 ON CHARSET(dt2_c2) = dt1_c1; To reiterate: CHARSET(dt2_c2) = dt1_c1 is pushed down into dt1 : SELECT c1 AS dt1_c1 FROM t1 WHERE CHARSET(dt2_c2) = dt1_c1 and CHARSET(dt2_c2) is a const item. The optimizer considers using KEY(c1) on t1 for value CHARSET(dt2_c2) and performs some operations at optimize_keyuse() . Since CHARSET(dt2_c2)::used_tables() == 2 and the dt1 's JOIN::table_count == 1, a crash occurs: optimize_keyuse() if (keyuse->used_tables & (map= (keyuse->used_tables & ~join->const_table_map & ~OUTER_REF_TABLE_BIT))) { uint n_tables= my_count_bits(map); if (n_tables == 1) // Only one table { DBUG_ASSERT(!(map & PSEUDO_TABLE_BITS)); // Must be a real table Table_map_iterator it(map); int tablenr= it.next_bit(); DBUG_ASSERT(tablenr != Table_map_iterator::BITMAP_END); TABLE *tmp_table=join->table[tablenr]; if (tmp_table) // already created keyuse->ref_table_rows= MY_MAX(tmp_table->file->stats.records, 100); } } The first idea: condition with CHARSET(dt2_c2) must not be pushed into dt1 (but why not if it's const?) Second idea: shouldn't we have JOIN::const_table_map == 2 after the pushdown? That eliminates the crash at optimize_keyuse().

            The first idea: condition with CHARSET(dt2_c2) must not be pushed into dt1 (but why not if it's const?)

            This may or may not be the solution. Will elaborate below.
            (I'm wondering, why does pushdown code check const_tables and not just use used_tables ?)

            Second idea: shouldn't we have JOIN::const_table_map == 2 after the pushdown? That eliminates the crash at optimize_keyuse().

            Do you the child join object should have const_table_map=2? This doesn't look like a good idea, because the number 2 (table bit #1 counting from 0) comes from the parent select. The child join may not have two tables (like in this example) or it may have another table with table->map=2.

            psergei Sergei Petrunia added a comment - The first idea: condition with CHARSET(dt2_c2) must not be pushed into dt1 (but why not if it's const?) This may or may not be the solution. Will elaborate below. (I'm wondering, why does pushdown code check const_tables and not just use used_tables ?) Second idea: shouldn't we have JOIN::const_table_map == 2 after the pushdown? That eliminates the crash at optimize_keyuse(). Do you the child join object should have const_table_map=2 ? This doesn't look like a good idea, because the number 2 (table bit #1 counting from 0) comes from the parent select. The child join may not have two tables (like in this example) or it may have another table with table->map=2.
            psergei Sergei Petrunia added a comment - - edited

            The fact that CHARSET(dt2_c2)::used_tables() == 2 after this expression has been pushed into dt1 is wrong.
            I can see two options:

            • If CHARSET(...) continues to refer to dt2, then it should have used_tables()=OUTER_REF_TABLE_BIT now. This is a bad approach to take as the optimizer doesn't currently support outer references inside derived tables.
            • Can we assume that CHARSET(dt2.dt2_c2) does not depend on dt2 and so it should have used_tables()=0?
            psergei Sergei Petrunia added a comment - - edited The fact that CHARSET(dt2_c2)::used_tables() == 2 after this expression has been pushed into dt1 is wrong. I can see two options: If CHARSET(...) continues to refer to dt2, then it should have used_tables()=OUTER_REF_TABLE_BIT now. This is a bad approach to take as the optimizer doesn't currently support outer references inside derived tables. Can we assume that CHARSET(dt2.dt2_c2) does not depend on dt2 and so it should have used_tables()=0?

            bar, any objections to making CHARSET(table.column) have used_tables()=0 ? currently it seems to have arg[0]->used_tables() ?

            psergei Sergei Petrunia added a comment - bar , any objections to making CHARSET(table.column) have used_tables()=0 ? currently it seems to have arg [0] ->used_tables() ?

            psergei, Sorry, I know methods used_tables() and const_tables() very badly. Cannot suggest anything.

            bar Alexander Barkov added a comment - psergei , Sorry, I know methods used_tables() and const_tables() very badly. Cannot suggest anything.
            oleg.smirnov Oleg Smirnov added a comment -

            The idea discussed during a meeting: move args[0]->charset_for_protocol() call to the preparation phase (fix_fields()) and remember the charset returned:

            String *Item_func_charset::val_str(String *str)
            {
              DBUG_ASSERT(fixed());
              uint dummy_errors;
             
              CHARSET_INFO *cs= args[0]->charset_for_protocol();
              null_value= 0;
              str->copy(cs->cs_name.str, cs->cs_name.length,
                        &my_charset_latin1, collation.collation, &dummy_errors);
              return str;
            }
            

            So we can safely set used_tables() to return 0 and avoid referring to arguments (args[0]) during the execution phase (call to val_str() is made during the execution).

            oleg.smirnov Oleg Smirnov added a comment - The idea discussed during a meeting: move args [0] ->charset_for_protocol() call to the preparation phase ( fix_fields() ) and remember the charset returned: String *Item_func_charset::val_str(String *str) { DBUG_ASSERT(fixed()); uint dummy_errors;   CHARSET_INFO *cs= args[0]->charset_for_protocol(); null_value= 0; str->copy(cs->cs_name.str, cs->cs_name.length, &my_charset_latin1, collation.collation, &dummy_errors); return str; } So we can safely set used_tables() to return 0 and avoid referring to arguments (args [0] ) during the execution phase (call to val_str() is made during the execution).
            julien.fritsch Julien Fritsch made changes -
            Fix Version/s 11.0 [ 28320 ]
            Fix Version/s 11.3 [ 28565 ]
            oleg.smirnov Oleg Smirnov made changes -
            Status Stalled [ 10000 ] In Progress [ 3 ]
            oleg.smirnov Oleg Smirnov made changes -
            Summary Condition with CHARSET()/COERCIBILITY() erroneously pushed into derived table Crash when pushing condition with CHARSET()/COERCIBILITY() into derived table
            oleg.smirnov Oleg Smirnov added a comment -

            sanja, can you please review the PR?

            oleg.smirnov Oleg Smirnov added a comment - sanja , can you please review the PR ?
            oleg.smirnov Oleg Smirnov made changes -
            Assignee Oleg Smirnov [ JIRAUSER50405 ] Oleksandr Byelkin [ sanja ]
            Status In Progress [ 3 ] In Review [ 10002 ]
            bar Alexander Barkov made changes -
            Assignee Oleksandr Byelkin [ sanja ] Alexander Barkov [ bar ]

            Hi oleg.smirnov,

            the patch is OK to push.

            I have just one small suggestion: Please change the data type of the new member from:

            CHARSET_INFO *m_cached_charset_info;
            

            to

            LEX_CSTRING m_cached_charset_info;
            

            to avoid strlen() evaluation per row by evaluating it during fix_length_and_dec() instead.

            Note, starting from 10.6 CHARSET_INFO was modified:

              const char *csname;
            

            was changed to:

              LEX_CSTRING cs_name;
            

            So starting from 10.6 strlen() won't be needed even in fix_length_and_dec().

            bar Alexander Barkov added a comment - Hi oleg.smirnov , the patch is OK to push. I have just one small suggestion: Please change the data type of the new member from: CHARSET_INFO *m_cached_charset_info; to LEX_CSTRING m_cached_charset_info; to avoid strlen() evaluation per row by evaluating it during fix_length_and_dec() instead. Note, starting from 10.6 CHARSET_INFO was modified: const char *csname; was changed to: LEX_CSTRING cs_name; So starting from 10.6 strlen() won't be needed even in fix_length_and_dec().
            bar Alexander Barkov made changes -
            Status In Review [ 10002 ] Stalled [ 10000 ]
            bar Alexander Barkov made changes -
            Assignee Alexander Barkov [ bar ] Oleg Smirnov [ JIRAUSER50405 ]
            oleg.smirnov Oleg Smirnov added a comment - - edited

            Good point, bar! Implemented your suggestion and pushed to 10.5 (commit)

            oleg.smirnov Oleg Smirnov added a comment - - edited Good point, bar ! Implemented your suggestion and pushed to 10.5 ( commit )
            oleg.smirnov Oleg Smirnov made changes -
            Fix Version/s 10.5.26 [ 29832 ]
            Fix Version/s 10.4 [ 22408 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.6 [ 24028 ]
            Fix Version/s 10.11 [ 27614 ]
            Fix Version/s 11.1 [ 28549 ]
            Fix Version/s 11.2 [ 28603 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            JIraAutomate JiraAutomate made changes -
            Fix Version/s 10.6.19 [ 29833 ]
            Fix Version/s 10.11.9 [ 29834 ]
            Fix Version/s 11.1.6 [ 29835 ]
            Fix Version/s 11.2.5 [ 29836 ]
            Fix Version/s 11.4.3 [ 29837 ]

            Note for the changelog: if a query used a derived table DT (a CTE or a
            mergeable VIEW would work as well) and the WHERE clause compared columns
            of DT with the value of CHARSET() or COERCIBILITY() function, the query
            could produce wrong result or crash. The cause was incorrect processing
            of these functions by derived condition pushdown optimization (
            https://mariadb.com/kb/en/condition-pushdown-into-derived-table-optimization/)

            psergei Sergei Petrunia added a comment - Note for the changelog: if a query used a derived table DT (a CTE or a mergeable VIEW would work as well) and the WHERE clause compared columns of DT with the value of CHARSET() or COERCIBILITY() function, the query could produce wrong result or crash. The cause was incorrect processing of these functions by derived condition pushdown optimization ( https://mariadb.com/kb/en/condition-pushdown-into-derived-table-optimization/ )
            oleg.smirnov Oleg Smirnov made changes -

            People

              oleg.smirnov Oleg Smirnov
              John Jove John Jove
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.