Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-26350

Assertion failure: select_lex->ref_pointer_array.size() % 5 == 0

Details

    Description

      I used my fuzzing tool to test Mariadb , and found a bug that can result in an abortion.

      Mariadb installation:
      1) cd mariadb-10.5.9
      2) mkdir build; cd build
      3) cmake -DWITH_ASAN=ON -DWITH_ASAN_SCOPE=ON -DCMAKE_BUILD_TYPE=Debug ../
      4) make -j8 && sudo make install

      How to Repeat:
      export ASAN_OPTIONS=detect_leaks=0
      /usr/local/mysql/bin/mysqld_safe &
      /usr/local/mysql/bin/mysql -uroot -p123456(your password)
      MariaDB> drop database if exists test_db;
      MariaDB> create database test_db;
      MariaDB> use test_db;
      MariaDB> source fuzz.sql;

      I have simplified the content of fuzz.sql (this one is still very complicate), and I hope fuzz.sql can help you reproduce the bug and fix it. In addition, I attach the failure report (which has its stack trace).

      Attachments

        1. fuzz.sql
          3.90 MB
        2. gdb.txt
          180 kB
        3. report.txt
          70 kB

        Issue Links

          Activity

            danblack Daniel Black added a comment -

            Did reproduce in debug version:

            10.5.13-0268b871228

            CMakeCache.txt:WITH_ASAN:BOOL=ON
            CMakeCache.txt:WITH_ASAN_SCOPE:BOOL=ON
            CMAKE_CXX_COMPILER:STRING=/usr/lib64/ccache/clang++
            CMAKE_C_COMPILER:STRING=/usr/lib64/ccache/clang
            CMAKE_BUILD_TYPE:STRING=Debug
             
            $ /usr/lib64/ccache/clang++ --version
            clang version 12.0.0 (Fedora 12.0.0-2.fc34)
            Target: x86_64-unknown-linux-gnu
            Thread model: posix
            InstalledDir: /usr/bin
             
             
            (gdb) bt
            #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:49
            #1  0x00007ffff74ad8a4 in __GI_abort () at abort.c:79
            #2  0x00007ffff74ad789 in __assert_fail_base (fmt=<optimized out>, assertion=<optimized out>, file=<optimized out>, line=<optimized out>, function=<optimized out>) at assert.c:92
            #3  0x00007ffff74bca16 in __GI___assert_fail (assertion=0x3525ca0 <str> "select_lex->ref_pointer_array.size() % 5 == 0", file=0x3525c40 <str> "/home/dan/repos/mariadb-server-10.5/sql/sql_select.h", line=1663, function=0x3525d00 <__PRETTY_FUNCTION__._ZN4JOIN19ref_ptr_array_sliceEm> "Ref_ptr_array JOIN::ref_ptr_array_slice(size_t)") at assert.c:101
            #4  0x0000000000ec729e in JOIN::ref_ptr_array_slice (this=0x7fffc9ba36b0, slice_num=0) at /home/dan/repos/mariadb-server-10.5/sql/sql_select.h:1663
            #5  0x0000000000d9d0bb in JOIN::prepare (this=0x7fffc9ba36b0, tables_init=0x7fffc9b95138, conds_init=0x7fffc9b9ad68, og_num=0, order_init=0x0, skip_order_by=false, group_init=0x0, having_init=0x0, proc_param_init=0x0, select_lex_arg=0x62b0000a1a30, unit_arg=0x62b00009e430) at /home/dan/repos/mariadb-server-10.5/sql/sql_select.cc:1277
            #6  0x0000000000d922bc in mysql_select (thd=0x62b00009a288, tables=0x7fffc9b95138, fields=@0x62b0000a1b80: {<base_list> = {<Sql_alloc> = {<No data fields>}, first = 0x7fffc9b94c80, last = 0x7fffc9b950a8, elements = 4}, <No data fields>}, conds=0x7fffc9b9ad68, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2201171004160, result=0x7fffc9ba35a8, unit=0x62b00009e430, select_lex=0x62b0000a1a30) at /home/dan/repos/mariadb-server-10.5/sql/sql_select.cc:4748
            #7  0x0000000000d912f8 in handle_select (thd=0x62b00009a288, lex=0x62b00009e368, result=0x7fffc9ba35a8, setup_tables_done_option=0) at /home/dan/repos/mariadb-server-10.5/sql/sql_select.cc:444
            #8  0x0000000001001972 in Sql_cmd_create_table_like::execute (this=0x62b0000a12a8, thd=0x62b00009a288) at /home/dan/repos/mariadb-server-10.5/sql/sql_table.cc:12158
            #9  0x0000000000cc6e79 in mysql_execute_command (thd=0x62b00009a288) at /home/dan/repos/mariadb-server-10.5/sql/sql_parse.cc:6056
            #10 0x0000000000c9aa05 in mysql_parse (thd=0x62b00009a288, rawbuf=0x7fffcbced8a8 "create table t_lzzzzz as \nselect  \n    case when (select c_l2iidekuu from t_yqc order by c_l2iidekuu limit 1 offset 1)\n", ' ' <repeats 11 times>, "< ( \n        select  \n", ' ' <repeats 14 times>, "ref_1.c_l1awww as c0\n", ' ' <repeats 12 times>, "f"..., length=4067372, parser_state=0x7fffccf7da60, is_com_multi=false, is_next_command=false) at /home/dan/repos/mariadb-server-10.5/sql/sql_parse.cc:8100
            #11 0x0000000000c92fa2 in dispatch_command (command=COM_QUERY, thd=0x62b00009a288, packet=0x7fffcc0d1889 "create table t_lzzzzz as \nselect  \n    case when (select c_l2iidekuu from t_yqc order by c_l2iidekuu limit 1 offset 1)\n", ' ' <repeats 11 times>, "< ( \n        select  \n", ' ' <repeats 14 times>, "ref_1.c_l1awww as c0\n", ' ' <repeats 12 times>, "f"..., packet_length=4067372, is_com_multi=false, is_next_command=false) at /home/dan/repos/mariadb-server-10.5/sql/sql_parse.cc:1891
            #12 0x0000000000c9d080 in do_command (thd=0x62b00009a288) at /home/dan/repos/mariadb-server-10.5/sql/sql_parse.cc:1370
            #13 0x00000000011f33b1 in do_handle_one_connection (connect=0x61100004b208, put_in_cache=true) at /home/dan/repos/mariadb-server-10.5/sql/sql_connect.cc:1418
            #14 0x00000000011f299f in handle_one_connection (arg=0x61100004b208) at /home/dan/repos/mariadb-server-10.5/sql/sql_connect.cc:1312
            #15 0x00000000021e17b9 in pfs_spawn_thread (arg=0x616000362b08) at /home/dan/repos/mariadb-server-10.5/storage/perfschema/pfs.cc:2201
            #16 0x00007ffff78ad299 in start_thread (arg=0x7fffccf80640) at pthread_create.c:481
            #17 0x00007ffff7587353 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
             
            (gdb) up
            #4  0x0000000000ec729e in JOIN::ref_ptr_array_slice (this=0x7fffc9ba36b0, slice_num=0) at /home/dan/repos/mariadb-server-10.5/sql/sql_select.h:1663
            1663	    DBUG_ASSERT(select_lex->ref_pointer_array.size() % 5 == 0);
            (gdb) list
            1658	
            1659	  /// Initialzes a slice, see comments for ref_ptrs above.
            1660	  Ref_ptr_array ref_ptr_array_slice(size_t slice_num)
            1661	  {
            1662	    size_t slice_sz= select_lex->ref_pointer_array.size() / 5U;
            1663	    DBUG_ASSERT(select_lex->ref_pointer_array.size() % 5 == 0);
            1664	    DBUG_ASSERT(slice_num < 5U);
            1665	    return Ref_ptr_array(&select_lex->ref_pointer_array[slice_num * slice_sz],
            1666	                         slice_sz);
            1667	  }
            (gdb) p select_lex->ref_pointer_array.size()
            [Thread 0x7fffd03c5640 (LWP 790261) exited]
            [Thread 0x7fffd1c0a640 (LWP 790258) exited]
            [Thread 0x7fffd74ff640 (LWP 790244) exited]
            [Thread 0x7fffd5cc2640 (LWP 790247) exited]
            [Thread 0x7fffd2c38640 (LWP 790253) exited]
            [Thread 0x7fffd6cf0640 (LWP 790245) exited]
            [Thread 0x7fffd3c66640 (LWP 790251) exited]
            [Thread 0x7fffd13f3640 (LWP 790259) exited]
            [Thread 0x7fffd2419640 (LWP 790257) exited]
            [Thread 0x7fffd54ab640 (LWP 790248) exited]
            [Thread 0x7fffd447d640 (LWP 790250) exited]
            [Thread 0x7fffd4c94640 (LWP 790249) exited]
            [Thread 0x7fffd0bdc640 (LWP 790260) exited]
            [Thread 0x7fffd64d9640 (LWP 790246) exited]
            [Thread 0x7fffd344f640 (LWP 790252) exited]
            $1 = 469542303
            (gdb) p select_lex->ref_pointer_array
            $2 = {m_array = 0x7ffee9d328a8, m_size = 469542303}
            

            Failed to reproduce on release version:

            10.5.13-0268b871228

            CMakeCache.txt:WITH_ASAN:BOOL=ON
            CMakeCache.txt:WITH_ASAN_SCOPE:BOOL=ON
            CMAKE_CXX_COMPILER:STRING=/usr/lib64/ccache/clang++
            CMAKE_C_COMPILER:STRING=/usr/lib64/ccache/clang
            CMAKE_BUILD_TYPE:STRING=RelWithDebInfo
             
            $ /usr/lib64/ccache/clang++ --version
            clang version 12.0.0 (Fedora 12.0.0-2.fc34)
            Target: x86_64-unknown-linux-gnu
            Thread model: posix
            InstalledDir: /usr/bin
             
            MariaDB [test_db]> source ~/Downloads/fuzz-MDEV-26350.sql
            Query OK, 0 rows affected (0.003 sec)
             
            Query OK, 0 rows affected (0.005 sec)
            Records: 0  Duplicates: 0  Warnings: 0
             
             
            Query OK, 0 rows affected (16.628 sec)
            Records: 0  Duplicates: 0  Warnings: 0
            
            

            danblack Daniel Black added a comment - Did reproduce in debug version: 10.5.13-0268b871228 CMakeCache.txt:WITH_ASAN:BOOL=ON CMakeCache.txt:WITH_ASAN_SCOPE:BOOL=ON CMAKE_CXX_COMPILER:STRING=/usr/lib64/ccache/clang++ CMAKE_C_COMPILER:STRING=/usr/lib64/ccache/clang CMAKE_BUILD_TYPE:STRING=Debug   $ /usr/lib64/ccache/clang++ --version clang version 12.0.0 (Fedora 12.0.0-2.fc34) Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /usr/bin     (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:49 #1 0x00007ffff74ad8a4 in __GI_abort () at abort.c:79 #2 0x00007ffff74ad789 in __assert_fail_base (fmt=<optimized out>, assertion=<optimized out>, file=<optimized out>, line=<optimized out>, function=<optimized out>) at assert.c:92 #3 0x00007ffff74bca16 in __GI___assert_fail (assertion=0x3525ca0 <str> "select_lex->ref_pointer_array.size() % 5 == 0", file=0x3525c40 <str> "/home/dan/repos/mariadb-server-10.5/sql/sql_select.h", line=1663, function=0x3525d00 <__PRETTY_FUNCTION__._ZN4JOIN19ref_ptr_array_sliceEm> "Ref_ptr_array JOIN::ref_ptr_array_slice(size_t)") at assert.c:101 #4 0x0000000000ec729e in JOIN::ref_ptr_array_slice (this=0x7fffc9ba36b0, slice_num=0) at /home/dan/repos/mariadb-server-10.5/sql/sql_select.h:1663 #5 0x0000000000d9d0bb in JOIN::prepare (this=0x7fffc9ba36b0, tables_init=0x7fffc9b95138, conds_init=0x7fffc9b9ad68, og_num=0, order_init=0x0, skip_order_by=false, group_init=0x0, having_init=0x0, proc_param_init=0x0, select_lex_arg=0x62b0000a1a30, unit_arg=0x62b00009e430) at /home/dan/repos/mariadb-server-10.5/sql/sql_select.cc:1277 #6 0x0000000000d922bc in mysql_select (thd=0x62b00009a288, tables=0x7fffc9b95138, fields=@0x62b0000a1b80: {<base_list> = {<Sql_alloc> = {<No data fields>}, first = 0x7fffc9b94c80, last = 0x7fffc9b950a8, elements = 4}, <No data fields>}, conds=0x7fffc9b9ad68, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2201171004160, result=0x7fffc9ba35a8, unit=0x62b00009e430, select_lex=0x62b0000a1a30) at /home/dan/repos/mariadb-server-10.5/sql/sql_select.cc:4748 #7 0x0000000000d912f8 in handle_select (thd=0x62b00009a288, lex=0x62b00009e368, result=0x7fffc9ba35a8, setup_tables_done_option=0) at /home/dan/repos/mariadb-server-10.5/sql/sql_select.cc:444 #8 0x0000000001001972 in Sql_cmd_create_table_like::execute (this=0x62b0000a12a8, thd=0x62b00009a288) at /home/dan/repos/mariadb-server-10.5/sql/sql_table.cc:12158 #9 0x0000000000cc6e79 in mysql_execute_command (thd=0x62b00009a288) at /home/dan/repos/mariadb-server-10.5/sql/sql_parse.cc:6056 #10 0x0000000000c9aa05 in mysql_parse (thd=0x62b00009a288, rawbuf=0x7fffcbced8a8 "create table t_lzzzzz as \nselect \n case when (select c_l2iidekuu from t_yqc order by c_l2iidekuu limit 1 offset 1)\n", ' ' <repeats 11 times>, "< ( \n select \n", ' ' <repeats 14 times>, "ref_1.c_l1awww as c0\n", ' ' <repeats 12 times>, "f"..., length=4067372, parser_state=0x7fffccf7da60, is_com_multi=false, is_next_command=false) at /home/dan/repos/mariadb-server-10.5/sql/sql_parse.cc:8100 #11 0x0000000000c92fa2 in dispatch_command (command=COM_QUERY, thd=0x62b00009a288, packet=0x7fffcc0d1889 "create table t_lzzzzz as \nselect \n case when (select c_l2iidekuu from t_yqc order by c_l2iidekuu limit 1 offset 1)\n", ' ' <repeats 11 times>, "< ( \n select \n", ' ' <repeats 14 times>, "ref_1.c_l1awww as c0\n", ' ' <repeats 12 times>, "f"..., packet_length=4067372, is_com_multi=false, is_next_command=false) at /home/dan/repos/mariadb-server-10.5/sql/sql_parse.cc:1891 #12 0x0000000000c9d080 in do_command (thd=0x62b00009a288) at /home/dan/repos/mariadb-server-10.5/sql/sql_parse.cc:1370 #13 0x00000000011f33b1 in do_handle_one_connection (connect=0x61100004b208, put_in_cache=true) at /home/dan/repos/mariadb-server-10.5/sql/sql_connect.cc:1418 #14 0x00000000011f299f in handle_one_connection (arg=0x61100004b208) at /home/dan/repos/mariadb-server-10.5/sql/sql_connect.cc:1312 #15 0x00000000021e17b9 in pfs_spawn_thread (arg=0x616000362b08) at /home/dan/repos/mariadb-server-10.5/storage/perfschema/pfs.cc:2201 #16 0x00007ffff78ad299 in start_thread (arg=0x7fffccf80640) at pthread_create.c:481 #17 0x00007ffff7587353 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   (gdb) up #4 0x0000000000ec729e in JOIN::ref_ptr_array_slice (this=0x7fffc9ba36b0, slice_num=0) at /home/dan/repos/mariadb-server-10.5/sql/sql_select.h:1663 1663 DBUG_ASSERT(select_lex->ref_pointer_array.size() % 5 == 0); (gdb) list 1658 1659 /// Initialzes a slice, see comments for ref_ptrs above. 1660 Ref_ptr_array ref_ptr_array_slice(size_t slice_num) 1661 { 1662 size_t slice_sz= select_lex->ref_pointer_array.size() / 5U; 1663 DBUG_ASSERT(select_lex->ref_pointer_array.size() % 5 == 0); 1664 DBUG_ASSERT(slice_num < 5U); 1665 return Ref_ptr_array(&select_lex->ref_pointer_array[slice_num * slice_sz], 1666 slice_sz); 1667 } (gdb) p select_lex->ref_pointer_array.size() [Thread 0x7fffd03c5640 (LWP 790261) exited] [Thread 0x7fffd1c0a640 (LWP 790258) exited] [Thread 0x7fffd74ff640 (LWP 790244) exited] [Thread 0x7fffd5cc2640 (LWP 790247) exited] [Thread 0x7fffd2c38640 (LWP 790253) exited] [Thread 0x7fffd6cf0640 (LWP 790245) exited] [Thread 0x7fffd3c66640 (LWP 790251) exited] [Thread 0x7fffd13f3640 (LWP 790259) exited] [Thread 0x7fffd2419640 (LWP 790257) exited] [Thread 0x7fffd54ab640 (LWP 790248) exited] [Thread 0x7fffd447d640 (LWP 790250) exited] [Thread 0x7fffd4c94640 (LWP 790249) exited] [Thread 0x7fffd0bdc640 (LWP 790260) exited] [Thread 0x7fffd64d9640 (LWP 790246) exited] [Thread 0x7fffd344f640 (LWP 790252) exited] $1 = 469542303 (gdb) p select_lex->ref_pointer_array $2 = {m_array = 0x7ffee9d328a8, m_size = 469542303} Failed to reproduce on release version: 10.5.13-0268b871228 CMakeCache.txt:WITH_ASAN:BOOL=ON CMakeCache.txt:WITH_ASAN_SCOPE:BOOL=ON CMAKE_CXX_COMPILER:STRING=/usr/lib64/ccache/clang++ CMAKE_C_COMPILER:STRING=/usr/lib64/ccache/clang CMAKE_BUILD_TYPE:STRING=RelWithDebInfo   $ /usr/lib64/ccache/clang++ --version clang version 12.0.0 (Fedora 12.0.0-2.fc34) Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /usr/bin   MariaDB [test_db]> source ~/Downloads/fuzz-MDEV-26350.sql Query OK, 0 rows affected (0.003 sec)   Query OK, 0 rows affected (0.005 sec) Records: 0 Duplicates: 0 Warnings: 0     Query OK, 0 rows affected (16.628 sec) Records: 0 Duplicates: 0 Warnings: 0
            danblack Daniel Black added a comment -

            produced on 10.4-debug. not reproducible on <10.4-debug or any release version.

            danblack Daniel Black added a comment - produced on 10.4-debug. not reproducible on <10.4-debug or any release version.
            danblack Daniel Black added a comment -

            This is an integer overflow.

            The following is sufficient to reduce it to a memory allocation error:

            workaround

            diff --git a/sql/sql_lex.cc b/sql/sql_lex.cc
            index 1ec5d0b0550..83a832f5e4f 100644
            --- a/sql/sql_lex.cc
            +++ b/sql/sql_lex.cc
            @@ -2998,7 +2998,7 @@ bool st_select_lex::setup_ref_array(THD *thd, uint order_group_num)
                 prepared statement
               */
               Query_arena *arena= thd->stmt_arena;
            -  const uint n_elems= (n_sum_items +
            +  const size_t n_elems= (n_sum_items +
                                    n_child_sum_items +
                                    item_list.elements +
                                    select_n_reserved +
            @@ -3006,7 +3006,8 @@ bool st_select_lex::setup_ref_array(THD *thd, uint order_group_num)
                                    select_n_where_fields +
                                    order_group_num +
                                    hidden_bit_fields +
            -                       fields_in_window_functions) * 5;
            +                       fields_in_window_functions) * 5ULL;
            +  DBUG_ASSERT(n_elems % 5 == 0);
               if (!ref_pointer_array.is_null())
               {
                 /*
            

            MariaDB [test_db]> source /home/dan/Downloads/fuzz-MDEV-26350.sql;
            Query OK, 0 rows affected (0.023 sec)
             
            Query OK, 0 rows affected (0.015 sec)
            Records: 0  Duplicates: 0  Warnings: 0
             
            ERROR 5 (HY000) at line 27 in file: '/home/dan/Downloads/fuzz-MDEV-26350.sql': Out of memory (Needed 3756338448 bytes)
             
            MariaDB [test_db]> select 1;
            +---+
            | 1 |
            +---+
            | 1 |
            +---+
            1 row in set (0.001 sec)
            

            And continues to run

            danblack Daniel Black added a comment - This is an integer overflow. The following is sufficient to reduce it to a memory allocation error: workaround diff --git a/sql/sql_lex.cc b/sql/sql_lex.cc index 1ec5d0b0550..83a832f5e4f 100644 --- a/sql/sql_lex.cc +++ b/sql/sql_lex.cc @@ -2998,7 +2998,7 @@ bool st_select_lex::setup_ref_array(THD *thd, uint order_group_num) prepared statement */ Query_arena *arena= thd->stmt_arena; - const uint n_elems= (n_sum_items + + const size_t n_elems= (n_sum_items + n_child_sum_items + item_list.elements + select_n_reserved + @@ -3006,7 +3006,8 @@ bool st_select_lex::setup_ref_array(THD *thd, uint order_group_num) select_n_where_fields + order_group_num + hidden_bit_fields + - fields_in_window_functions) * 5; + fields_in_window_functions) * 5ULL; + DBUG_ASSERT(n_elems % 5 == 0); if (!ref_pointer_array.is_null()) { /* MariaDB [test_db]> source /home/dan/Downloads/fuzz-MDEV-26350.sql; Query OK, 0 rows affected (0.023 sec)   Query OK, 0 rows affected (0.015 sec) Records: 0 Duplicates: 0 Warnings: 0   ERROR 5 (HY000) at line 27 in file: '/home/dan/Downloads/fuzz-MDEV-26350.sql': Out of memory (Needed 3756338448 bytes)   MariaDB [test_db]> select 1; +---+ | 1 | +---+ | 1 | +---+ 1 row in set (0.001 sec) And continues to run
            danblack Daniel Black added a comment -

            While the exact test case provided doesn't trigger the error on 10.2, the code is still susceptable to an integer overflow and this assertion in debug builds.

            serg can you review bb-10.2-danielblack-MDEV-26350-int_overflow-select_lex-ref_pointer_array please.

            danblack Daniel Black added a comment - While the exact test case provided doesn't trigger the error on 10.2, the code is still susceptable to an integer overflow and this assertion in debug builds. serg can you review bb-10.2-danielblack- MDEV-26350 -int_overflow-select_lex-ref_pointer_array please.

            OK to push.

            (IMHO DBUG_ASSERT is not needed but it does not hart so let it be).

            sanja Oleksandr Byelkin added a comment - OK to push. (IMHO DBUG_ASSERT is not needed but it does not hart so let it be).

            People

              danblack Daniel Black
              Zuming Jiang Zuming Jiang
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.