Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-19361

Assertion `marked_for_read()' failed in Field_long::val_int

Details

    Description

      CREATE TABLE t1 (a INT) ENGINE=MyISAM;
      INSERT INTO t1 VALUES (1),(0);
       
      CREATE TABLE t2 (
        b INT,
        b1 INT GENERATED ALWAYS AS (b/2) STORED,
        b2 INT GENERATED ALWAYS AS (b*2) VIRTUAL,
        b3 INT GENERATED ALWAYS AS (b MOD 128) STORED,
        KEY (b1),
        KEY (b3),
        KEY (b1,b3,b2),
        KEY (b2,b3)
      ) ENGINE=MyISAM;
      INSERT INTO t2 (b) VALUES (6),(6),(8),(0),(4);
       
      CREATE TABLE t3 (c INT) ENGINE=MyISAM;
      INSERT INTO t3 VALUES (1),(2);
       
      CREATE ALGORITHM = MERGE VIEW v AS SELECT t2.b1, t2.b2 FROM t1 STRAIGHT_JOIN t2 ON t1.a = t2.b3;
       
      SELECT * FROM v STRAIGHT_JOIN t3 ON v.b2 = t3.c WHERE v.b1 <= 0;
       
      # Cleanup
      DROP VIEW v;
      DROP TABLE t1, t2, t3;
      

      10.4 81f6a3b0

      mysqld: /data/src/10.4/sql/field.cc:4257: virtual longlong Field_long::val_int(): Assertion `marked_for_read()' failed.
      190429 20:31:16 [ERROR] mysqld got signal 6 ;
       
      #7  0x00007ffa21e0aee2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
      #8  0x0000556612ba0fc4 in Field_long::val_int (this=0x7ffa001532f8) at /data/src/10.4/sql/field.cc:4257
      #9  0x0000556612bf28a2 in Item_field::val_int (this=0x7ffa001557b8) at /data/src/10.4/sql/item.cc:3153
      #10 0x0000556612c56f92 in Item_func_mul::int_op (this=0x7ffa00155630) at /data/src/10.4/sql/item_func.cc:1316
      #11 0x0000556612ab510b in Item_func_hybrid_field_type::val_int_from_int_op (this=0x7ffa00155630) at /data/src/10.4/sql/item_func.h:701
      #12 0x0000556612aa122c in Type_handler_int_result::Item_func_hybrid_field_type_val_int (this=0x556613f2f2f8 <type_handler_longlong>, item=0x7ffa00155630) at /data/src/10.4/sql/sql_type.cc:4679
      #13 0x000055661287dfee in Item_func_hybrid_field_type::val_int (this=0x7ffa00155630) at /data/src/10.4/sql/item_func.h:757
      #14 0x0000556612bfc09e in Item::save_int_in_field (this=0x7ffa00155630, field=0x7ffa001534b8, no_conversions=false) at /data/src/10.4/sql/item.cc:6482
      #15 0x0000556612a9ebc4 in Type_handler_int_result::Item_save_in_field (this=0x556613f2f2f8 <type_handler_longlong>, item=0x7ffa00155630, field=0x7ffa001534b8, no_conversions=false) at /data/src/10.4/sql/sql_type.cc:3587
      #16 0x0000556612bfc149 in Item::save_in_field (this=0x7ffa00155630, field=0x7ffa001534b8, no_conversions=false) at /data/src/10.4/sql/item.cc:6492
      #17 0x00005566129c59c4 in TABLE::update_virtual_fields (this=0x7ffa00152430, h=0x7ffa001536f8, update_mode=VCOL_UPDATE_FOR_READ) at /data/src/10.4/sql/table.cc:8198
      #18 0x0000556612bd6300 in handler::ha_rnd_next (this=0x7ffa001536f8, buf=0x7ffa001532a0 "\341\006") at /data/src/10.4/sql/handler.cc:2823
      #19 0x0000556612d69b63 in rr_sequential (info=0x7ffa0004a140) at /data/src/10.4/sql/records.cc:481
      #20 0x00005566127eed1b in READ_RECORD::read_record (this=0x7ffa0004a140) at /data/src/10.4/sql/records.h:73
      #21 0x000055661291544f in join_init_read_record (tab=0x7ffa0004a078) at /data/src/10.4/sql/sql_select.cc:20985
      #22 0x0000556612a580cc in JOIN_TAB_SCAN::open (this=0x7ffa0004bba8) at /data/src/10.4/sql/sql_join_cache.cc:3348
      #23 0x0000556612a5651c in JOIN_CACHE::join_matching_records (this=0x7ffa0004d428, skip_last=false) at /data/src/10.4/sql/sql_join_cache.cc:2251
      #24 0x0000556612a5607e in JOIN_CACHE::join_records (this=0x7ffa0004d428, skip_last=false) at /data/src/10.4/sql/sql_join_cache.cc:2088
      #25 0x0000556612912d1c in sub_select_cache (join=0x7ffa00045690, join_tab=0x7ffa0004a078, end_of_records=true) at /data/src/10.4/sql/sql_select.cc:19822
      #26 0x0000556612912f29 in sub_select (join=0x7ffa00045690, join_tab=0x7ffa00049cb0, end_of_records=true) at /data/src/10.4/sql/sql_select.cc:19993
      #27 0x00005566129126f1 in do_select (join=0x7ffa00045690, procedure=0x0) at /data/src/10.4/sql/sql_select.cc:19584
      #28 0x00005566128e9277 in JOIN::exec_inner (this=0x7ffa00045690) at /data/src/10.4/sql/sql_select.cc:4389
      #29 0x00005566128e841e in JOIN::exec (this=0x7ffa00045690) at /data/src/10.4/sql/sql_select.cc:4171
      #30 0x00005566128e9ad0 in mysql_select (thd=0x7ffa00000b00, tables=0x7ffa00015c78, wild_num=1, fields=..., conds=0x7ffa00017808, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7ffa00045668, unit=0x7ffa00004a28, select_lex=0x7ffa000156b8) at /data/src/10.4/sql/sql_select.cc:4603
      #31 0x00005566128da040 in handle_select (thd=0x7ffa00000b00, lex=0x7ffa00004960, result=0x7ffa00045668, setup_tables_done_option=0) at /data/src/10.4/sql/sql_select.cc:424
      #32 0x00005566128a38d9 in execute_sqlcom_select (thd=0x7ffa00000b00, all_tables=0x7ffa00015c78) at /data/src/10.4/sql/sql_parse.cc:6597
      #33 0x0000556612898eaf in mysql_execute_command (thd=0x7ffa00000b00) at /data/src/10.4/sql/sql_parse.cc:3886
      #34 0x00005566128a765f in mysql_parse (thd=0x7ffa00000b00, rawbuf=0x7ffa000155d8 "SELECT * FROM v STRAIGHT_JOIN t3 ON v.b2 = t3.c WHERE v.b1 <= 0", length=63, parser_state=0x7ffa0ffb4180, is_com_multi=false, is_next_command=false) at /data/src/10.4/sql/sql_parse.cc:8149
      #35 0x00005566128930b6 in dispatch_command (command=COM_QUERY, thd=0x7ffa00000b00, packet=0x7ffa001398f1 "", packet_length=63, is_com_multi=false, is_next_command=false) at /data/src/10.4/sql/sql_parse.cc:1828
      #36 0x000055661289189c in do_command (thd=0x7ffa00000b00) at /data/src/10.4/sql/sql_parse.cc:1361
      #37 0x0000556612a09cad in do_handle_one_connection (connect=0x5566151d5290) at /data/src/10.4/sql/sql_connect.cc:1398
      #38 0x0000556612a09a1e in handle_one_connection (arg=0x5566151d5290) at /data/src/10.4/sql/sql_connect.cc:1301
      #39 0x0000556612e2ed9d in pfs_spawn_thread (arg=0x5566152726d0) at /data/src/10.4/storage/perfschema/pfs.cc:1862
      #40 0x00007ffa23eff494 in start_thread (arg=0x7ffa0ffb5700) at pthread_create.c:333
      #41 0x00007ffa21ec793f in clone () from /lib/x86_64-linux-gnu/libc.so.6
      

      No obvious failure on a non-debug build.

      Not reproducible on 10.3.

      Attachments

        Issue Links

          Activity

            nikitamalyavin Nikita Malyavin added a comment - - edited

            The reason the initial test case does not fail is optimizer behavior change present by
            4d4b2867a MDEV-20056 Assertion `!prebuilt->index->is_primary()' failed
            in row_search_idx_cond_check

            --- a/sql/sql_select.cc
            +++ b/sql/sql_select.cc
            @@ -7701,7 +7707,8 @@ best_access_path(JOIN      *join,
                                                           found_ref);
                   } /* not ft_key */
             
            -      if (records < DBL_MAX)
            +      if (records < DBL_MAX &&
            +         (found_part & 1))   // start_key->key can be used for index access
                   {
                     double rows= record_count * records;
                     double access_cost_factor= MY_MIN(tmp / rows, 1.0);
            

            Reverting this code hunk makes the issue reproducible

            nikitamalyavin Nikita Malyavin added a comment - - edited The reason the initial test case does not fail is optimizer behavior change present by 4d4b2867a MDEV-20056 Assertion `!prebuilt->index->is_primary()' failed in row_search_idx_cond_check --- a/sql/sql_select.cc +++ b/sql/sql_select.cc @@ -7701,7 +7707,8 @@ best_access_path(JOIN *join, found_ref); } /* not ft_key */ - if (records < DBL_MAX) + if (records < DBL_MAX && + (found_part & 1)) // start_key->key can be used for index access { double rows= record_count * records; double access_cost_factor= MY_MIN(tmp / rows, 1.0); Reverting this code hunk makes the issue reproducible

            Night is spent on trying to expose the fact that table->prepare_for_keyread sets the bits for a virtual field, if it was used, but not for the fields inside the vcol expression.

            In JOIN::init_join_caches():

            if ((tab->read_first_record == join_read_first ||
                          tab->read_first_record == join_read_last) &&
                         !tab->filesort && table->covering_keys.is_set(tab->index) &&
                         !table->no_keyread)
                {
                  table->prepare_for_keyread(tab->index, table->read_set);
                }
            

            tab->read_first_record can be join_read_first when the key is a covering key, i.e. includes all the fields involved in select.
            In make_join_readinfo():

            if (!table->covering_keys.is_clear_all() &&
            		   !(tab->select && tab->select->quick))
            ...
              tab->read_first_record= join_read_first
            

            is the only place which seems to be possibly reachable.

            However, covering_keys are intersected with hinting keys.

            Using asterisk (SELECT *) is pointless, since it will then load the record anyway, and mark a read bit.

            Using USE/FORCE INDEX [FOR ORDER BY] did not provide any success.

            test_if_skip_sort_order is left unreachable even under ~50k rows test.

            nikitamalyavin Nikita Malyavin added a comment - Night is spent on trying to expose the fact that table->prepare_for_keyread sets the bits for a virtual field, if it was used, but not for the fields inside the vcol expression. In JOIN::init_join_caches() : if ((tab->read_first_record == join_read_first || tab->read_first_record == join_read_last) && !tab->filesort && table->covering_keys.is_set(tab->index) && !table->no_keyread) { table->prepare_for_keyread(tab->index, table->read_set); } tab->read_first_record can be join_read_first when the key is a covering key, i.e. includes all the fields involved in select. In make_join_readinfo() : if (!table->covering_keys.is_clear_all() && !(tab->select && tab->select->quick)) ... tab->read_first_record= join_read_first is the only place which seems to be possibly reachable. However, covering_keys are intersected with hinting keys. Using asterisk (SELECT *) is pointless, since it will then load the record anyway, and mark a read bit. Using USE/FORCE INDEX [FOR ORDER BY] did not provide any success. test_if_skip_sort_order is left unreachable even under ~50k rows test.

            nikitamalyavin,

            The reason the initial test case does not fail is optimizer behavior change present by 4d4b2867a MDEV-20056

            I don't understand the point you make by closing it as "Can't reproduce" based on the above.
            Are you saying that the patch for MDEV-20056 actually fixed the bug which was affecting virtual columns, or did it just make the test case irrelevant?
            If you can confirm that MDEV-20056 has fixed the problem described here, then it should be closed as a duplicate or as fixed, whichever you prefer.
            But if it just made the test irrelevant (hid the failure described here), closing it as "can't reproduce" is meaningless. Of course you can reproduce it, just take the revision before MDEV-20056 fix, when the test case was still relevant, and use it for debugging.

            elenst Elena Stepanova added a comment - nikitamalyavin , The reason the initial test case does not fail is optimizer behavior change present by 4d4b2867a MDEV-20056 I don't understand the point you make by closing it as "Can't reproduce" based on the above. Are you saying that the patch for MDEV-20056 actually fixed the bug which was affecting virtual columns, or did it just make the test case irrelevant? If you can confirm that MDEV-20056 has fixed the problem described here, then it should be closed as a duplicate or as fixed, whichever you prefer. But if it just made the test irrelevant (hid the failure described here), closing it as "can't reproduce" is meaningless. Of course you can reproduce it, just take the revision before MDEV-20056 fix, when the test case was still relevant, and use it for debugging.

            So the index was chosen correctly:

            (gdb) bt
            #0  0x000055c6327ac462 in bitmap_set_bit (map=0x7f1ee403bbf0, bit=2) at ../include/my_bitmap.h:116
            #1  0x000055c6327abfb7 in TABLE::mark_columns_used_by_index_no_reset (this=0x7f1ee403baa8, index=2, bitmap=0x7f1ee403bbf0) at ../sql/table.cc:6939
            #2  0x000055c6327ac23c in TABLE::mark_columns_used_by_index (this=0x7f1ee403baa8, index=2, bitmap=0x7f1ee403bbf0) at ../sql/table.cc:6905
            #3  0x000055c6327ac141 in TABLE::prepare_for_keyread (this=0x7f1ee403baa8, index=2, map=0x7f1ee403bbf0) at ../sql/table.cc:6889
            #4  0x000055c63269075d in JOIN::init_join_caches (this=0x7f1ee40516f8) at ../sql/sql_select.cc:1817
            #5  0x000055c63268d041 in JOIN::optimize_stage2 (this=0x7f1ee40516f8) at ../sql/sql_select.cc:3052
            #6  0x000055c63268f9dc in JOIN::optimize_inner (this=0x7f1ee40516f8) at ../sql/sql_select.cc:2335
            #7  0x000055c63268a038 in JOIN::optimize (this=0x7f1ee40516f8) at ../sql/sql_select.cc:1658
            #8  0x000055c632680ebf in mysql_select (thd=0x7f1ee4000d28, tables=0x7f1ee4013cb8, wild_num=1, fields=..., conds=0x7f1ee40158c0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7f1ee40516d0, unit=0x7f1ee4004ad0, select_lex=0x7f1ee40136d8) at ../sql/sql_select.cc:4725
            #9  0x000055c6326805ad in handle_select (thd=0x7f1ee4000d28, lex=0x7f1ee4004a10, result=0x7f1ee40516d0, setup_tables_done_option=0) at ../sql/sql_select.cc:436
            ....
            (gdb) f 4
            #4  0x000055c63269075d in JOIN::init_join_caches (this=0x7f1ee40516f8) at ../sql/sql_select.cc:1817
            1817	      table->prepare_for_keyread(tab->index, table->read_set);
            (gdb) p tab->index
            $1 = 2
            
            

            Keyread is enabled. However, we can see from the description stack trace that rr_sequential was used, which is incorrect due to keyread.

            First, in best_access_path rowid filter was assigned:

            filter=
              table->best_range_rowid_filter_for_partial_join(start_key->key, rows,
                                                              access_cost_factor);
            

            This is what was changed by the patch.
            Then keyread was enabled in JOIN::init_join_caches, like in the trace above . Then it was reenabled by rowid filter in fill():

            #0  handler::ha_start_keyread (this=0x7f1ee403c8a0, idx=0) at ../sql/handler.h:3299
            #1  0x000055c6328ef141 in Range_rowid_filter::fill (this=0x7f1ee40539c0) at ../sql/rowid_filter.cc:570
            #2  0x000055c6328ef8a5 in Range_rowid_filter::build (this=0x7f1ee40539c0) at ../sql/rowid_filter.h:267
            #3  0x000055c6326c0dee in st_join_table::build_range_rowid_filter_if_needed (this=0x7f1ee4057fc0) at ../sql/sql_select.cc:13380
            #4  0x000055c6328522ee in JOIN_CACHE::join_matching_records (this=0x7f1ee405c1f8, skip_last=false) at ../sql/sql_join_cache.cc:2320
            #5  0x000055c632851c7b in JOIN_CACHE::join_records (this=0x7f1ee405c1f8, skip_last=false) at ../sql/sql_join_cache.cc:2147
            

            Then it was disabled, also there and wasnt reenabled back.
            So after that, in init_read_record, if (table->file->keyread_enabled()) path is not chosen,
            and

             

            info->read_record_func= rr_sequential;

            {cpp}

            is evaluated

            psergei can you tell if there are other ways to create rowid filter together with keyread enabled? My point is this combination exposes the bug.

            After 4d4b2867a fix mentioned above, the latter code path is no longer accessible since && (found_part & 1) is added which means there should be a primary key. But then b will be in ext key parts and will be marked for read. (However I guess, rr_sequential will still be chosen, and keyread can be wrongly finished)

            So we can't at least add PK to the table. It is better to leave the test case as is. I wonder if the issue can be reproduced with carefully configured optimizer switch

            nikitamalyavin Nikita Malyavin added a comment - So the index was chosen correctly: (gdb) bt #0 0x000055c6327ac462 in bitmap_set_bit (map=0x7f1ee403bbf0, bit=2) at ../include/my_bitmap.h:116 #1 0x000055c6327abfb7 in TABLE::mark_columns_used_by_index_no_reset (this=0x7f1ee403baa8, index=2, bitmap=0x7f1ee403bbf0) at ../sql/table.cc:6939 #2 0x000055c6327ac23c in TABLE::mark_columns_used_by_index (this=0x7f1ee403baa8, index=2, bitmap=0x7f1ee403bbf0) at ../sql/table.cc:6905 #3 0x000055c6327ac141 in TABLE::prepare_for_keyread (this=0x7f1ee403baa8, index=2, map=0x7f1ee403bbf0) at ../sql/table.cc:6889 #4 0x000055c63269075d in JOIN::init_join_caches (this=0x7f1ee40516f8) at ../sql/sql_select.cc:1817 #5 0x000055c63268d041 in JOIN::optimize_stage2 (this=0x7f1ee40516f8) at ../sql/sql_select.cc:3052 #6 0x000055c63268f9dc in JOIN::optimize_inner (this=0x7f1ee40516f8) at ../sql/sql_select.cc:2335 #7 0x000055c63268a038 in JOIN::optimize (this=0x7f1ee40516f8) at ../sql/sql_select.cc:1658 #8 0x000055c632680ebf in mysql_select (thd=0x7f1ee4000d28, tables=0x7f1ee4013cb8, wild_num=1, fields=..., conds=0x7f1ee40158c0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7f1ee40516d0, unit=0x7f1ee4004ad0, select_lex=0x7f1ee40136d8) at ../sql/sql_select.cc:4725 #9 0x000055c6326805ad in handle_select (thd=0x7f1ee4000d28, lex=0x7f1ee4004a10, result=0x7f1ee40516d0, setup_tables_done_option=0) at ../sql/sql_select.cc:436 .... (gdb) f 4 #4 0x000055c63269075d in JOIN::init_join_caches (this=0x7f1ee40516f8) at ../sql/sql_select.cc:1817 1817 table->prepare_for_keyread(tab->index, table->read_set); (gdb) p tab->index $1 = 2 Keyread is enabled. However, we can see from the description stack trace that rr_sequential was used, which is incorrect due to keyread. First, in best_access_path rowid filter was assigned: filter= table->best_range_rowid_filter_for_partial_join(start_key->key, rows, access_cost_factor); This is what was changed by the patch. Then keyread was enabled in JOIN::init_join_caches , like in the trace above . Then it was reenabled by rowid filter in fill(): #0 handler::ha_start_keyread (this=0x7f1ee403c8a0, idx=0) at ../sql/handler.h:3299 #1 0x000055c6328ef141 in Range_rowid_filter::fill (this=0x7f1ee40539c0) at ../sql/rowid_filter.cc:570 #2 0x000055c6328ef8a5 in Range_rowid_filter::build (this=0x7f1ee40539c0) at ../sql/rowid_filter.h:267 #3 0x000055c6326c0dee in st_join_table::build_range_rowid_filter_if_needed (this=0x7f1ee4057fc0) at ../sql/sql_select.cc:13380 #4 0x000055c6328522ee in JOIN_CACHE::join_matching_records (this=0x7f1ee405c1f8, skip_last=false) at ../sql/sql_join_cache.cc:2320 #5 0x000055c632851c7b in JOIN_CACHE::join_records (this=0x7f1ee405c1f8, skip_last=false) at ../sql/sql_join_cache.cc:2147 Then it was disabled, also there and wasnt reenabled back. So after that, in init_read_record, if (table->file->keyread_enabled()) path is not chosen, and   info->read_record_func= rr_sequential; {cpp} is evaluated psergei can you tell if there are other ways to create rowid filter together with keyread enabled? My point is this combination exposes the bug. After 4d4b2867a fix mentioned above, the latter code path is no longer accessible since && (found_part & 1) is added which means there should be a primary key. But then b will be in ext key parts and will be marked for read. (However I guess, rr_sequential will still be chosen, and keyread can be wrongly finished) So we can't at least add PK to the table. It is better to leave the test case as is. I wonder if the issue can be reproduced with carefully configured optimizer switch

            The original test case in the description required rowid_filter, and that failure disappeared after this commit (released in 10.4.11):

            commit 4d4b2867a2526872a7b476a5c3577be695800a8e (HEAD)
            Author: Igor Babaev
            Date:   Mon Nov 25 17:40:47 2019 -0800
             
                MDEV-20056 Assertion `!prebuilt->index->is_primary()' failed
                           in row_search_idx_cond_check
            

            And the alternative test case in the comments was fixed by MDEV-18166.
            So, since we don't have remaining failing test cases here, I will close it.
            Unfortunately the failure itself still happens a lot, we'll just need another bug report for it.

            elenst Elena Stepanova added a comment - The original test case in the description required rowid_filter , and that failure disappeared after this commit (released in 10.4.11): commit 4d4b2867a2526872a7b476a5c3577be695800a8e (HEAD) Author: Igor Babaev Date: Mon Nov 25 17:40:47 2019 -0800   MDEV-20056 Assertion `!prebuilt->index->is_primary()' failed in row_search_idx_cond_check And the alternative test case in the comments was fixed by MDEV-18166 . So, since we don't have remaining failing test cases here, I will close it. Unfortunately the failure itself still happens a lot, we'll just need another bug report for it.

            People

              Unassigned Unassigned
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.