Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-12255

innodb_prefix_index_cluster_optimization hits debug build assert on UTF-8 columns

Details

    • 10.1.32

    Description

      ./mtr --do-test=ctype --mysqld=--innodb_prefix_index_cluster_optimization=ON
      ...
      2017-03-14 14:48:08 700001c51000  InnoDB: Assertion failure in thread 123145332002816 in file row0sel.cc line 2717
      InnoDB: Failing assertion: len * templ->mbmaxlen >= templ->mysql_col_len || (field_no == templ->icp_rec_field_no && field->prefix_len > 0)
      ...
      Failing test(s): main.ctype_utf8 main.ctype_utf8mb4 main.ctype_utf8mb4_innodb
      

      This is a parallel bug to https://github.com/facebook/mysql-5.6/issues/568, but appears to be less serious, as Facebook patch silently returns wrong query results too, whereas MariaDB only crashes the debug build. I don't know if these testcases return correct results in the release build.

      Attachments

        Issue Links

          Activity

            10.1 3bb32e8682f849413ce994277e9029ba5732baae

            2017-04-17 17:06:00 7f5c22fdbb00  InnoDB: Assertion failure in thread 140033700772608 in file row0sel.cc line 2709
            InnoDB: Failing assertion: len * templ->mbmaxlen >= templ->mysql_col_len || (field_no == templ->icp_rec_field_no && field->prefix_len > 0)
            InnoDB: We intentionally generate a memory trap.
             
            #5  0x00007f5c1f8d4448 in __GI_abort () at abort.c:89
            #6  0x00007f5c19030393 in row_sel_field_store_in_mysql_format_func (dest=0x7f5c0a0a1e89 "a", '\217' <repeats 29 times>, "\245\377", '\245' <repeats 31 times>, "\330\036\n\n\\\177", templ=0x7f5c0a28a378, index=0x7f5c0a1269f8, field_no=0, data=0x7f5c16b2407f "a", len=1) at /data/src/10.1/storage/innobase/row/row0sel.cc:2707
            #7  0x00007f5c190309ce in row_sel_store_mysql_field_func (mysql_rec=0x7f5c0a0a1e88 "\377a", '\217' <repeats 29 times>, "\245\377", '\245' <repeats 31 times>, "\330\036\n\n\\\177", prebuilt=0x7f5c0a084678, rec=0x7f5c16b2407f "a", index=0x7f5c0a1269f8, offsets=0x7f5c22fd85c0, field_no=0, templ=0x7f5c0a28a378) at /data/src/10.1/storage/innobase/row/row0sel.cc:2882
            #8  0x00007f5c19030b7b in row_sel_store_mysql_rec (mysql_rec=0x7f5c0a0a1e88 "\377a", '\217' <repeats 29 times>, "\245\377", '\245' <repeats 31 times>, "\330\036\n\n\\\177", prebuilt=0x7f5c0a084678, rec=0x7f5c16b2407f "a", rec_clust=0, index=0x7f5c0a1269f8, offsets=0x7f5c22fd85c0) at /data/src/10.1/storage/innobase/row/row0sel.cc:2946
            #9  0x00007f5c19034b61 in row_search_for_mysql (buf=0x7f5c0a0a1e88 "\377a", '\217' <repeats 29 times>, "\245\377", '\245' <repeats 31 times>, "\330\036\n\n\\\177", mode=2, prebuilt=0x7f5c0a084678, match_mode=1, direction=0) at /data/src/10.1/storage/innobase/row/row0sel.cc:5012
            #10 0x00007f5c18efe7c0 in ha_innodb::index_read (this=0x7f5c0a0b3888, buf=0x7f5c0a0a1e88 "\377a", '\217' <repeats 29 times>, "\245\377", '\245' <repeats 31 times>, "\330\036\n\n\\\177", key_ptr=0x7f5c0a0fc310 "", key_len=4, find_flag=HA_READ_KEY_EXACT) at /data/src/10.1/storage/innobase/handler/ha_innodb.cc:9310
            #11 0x00007f5c18f14b44 in handler::index_read_map (this=0x7f5c0a0b3888, buf=0x7f5c0a0a1e88 "\377a", '\217' <repeats 29 times>, "\245\377", '\245' <repeats 31 times>, "\330\036\n\n\\\177", key=0x7f5c0a0fc310 "", keypart_map=1, find_flag=HA_READ_KEY_EXACT) at /data/src/10.1/sql/handler.h:3052
            #12 0x00007f5c2231b2b0 in handler::index_read_idx_map (this=0x7f5c0a0b3888, buf=0x7f5c0a0a1e88 "\377a", '\217' <repeats 29 times>, "\245\377", '\245' <repeats 31 times>, "\330\036\n\n\\\177", index=0, key=0x7f5c0a0fc310 "", keypart_map=1, find_flag=HA_READ_KEY_EXACT) at /data/src/10.1/sql/handler.cc:5443
            #13 0x00007f5c22314dfb in handler::ha_index_read_idx_map (this=0x7f5c0a0b3888, buf=0x7f5c0a0a1e88 "\377a", '\217' <repeats 29 times>, "\245\377", '\245' <repeats 31 times>, "\330\036\n\n\\\177", index=0, key=0x7f5c0a0fc310 "", keypart_map=1, find_flag=HA_READ_KEY_EXACT) at /data/src/10.1/sql/handler.cc:2646
            #14 0x00007f5c22140faa in join_read_const (tab=0x7f5c0a0fb2b8) at /data/src/10.1/sql/sql_select.cc:18893
            #15 0x00007f5c22140aa9 in join_read_const_table (thd=0x7f5c163a6070, tab=0x7f5c0a0fb2b8, pos=0x7f5c0a0fb808) at /data/src/10.1/sql/sql_select.cc:18768
            #16 0x00007f5c2211c515 in make_join_statistics (join=0x7f5c0a0fac68, tables_list=..., keyuse_array=0x7f5c0a0fafa0) at /data/src/10.1/sql/sql_select.cc:3936
            #17 0x00007f5c2211345b in JOIN::optimize_inner (this=0x7f5c0a0fac68) at /data/src/10.1/sql/sql_select.cc:1366
            #18 0x00007f5c22112346 in JOIN::optimize (this=0x7f5c0a0fac68) at /data/src/10.1/sql/sql_select.cc:1045
            #19 0x00007f5c2211a9aa in mysql_select (thd=0x7f5c163a6070, rref_pointer_array=0x7f5c163aa430, tables=0x7f5c0a0fa298, wild_num=0, fields=..., conds=0x7f5c0a0faa28, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7f5c0a0fac48, unit=0x7f5c163a9ab8, select_lex=0x7f5c163aa1b8) at /data/src/10.1/sql/sql_select.cc:3431
            #20 0x00007f5c22110496 in handle_select (thd=0x7f5c163a6070, lex=0x7f5c163a99f0, result=0x7f5c0a0fac48, setup_tables_done_option=0) at /data/src/10.1/sql/sql_select.cc:384
            #21 0x00007f5c220e03fb in execute_sqlcom_select (thd=0x7f5c163a6070, all_tables=0x7f5c0a0fa298) at /data/src/10.1/sql/sql_parse.cc:5920
            #22 0x00007f5c220d68f7 in mysql_execute_command (thd=0x7f5c163a6070) at /data/src/10.1/sql/sql_parse.cc:2979
            #23 0x00007f5c220e3b19 in mysql_parse (thd=0x7f5c163a6070, rawbuf=0x7f5c0a0fa088 "select c as c_a from t1 where c='a'", length=35, parser_state=0x7f5c22fda5e0) at /data/src/10.1/sql/sql_parse.cc:7343
            #24 0x00007f5c220d2b06 in dispatch_command (command=COM_QUERY, thd=0x7f5c163a6070, packet=0x7f5c1829b071 "select c as c_a from t1 where c='a'", packet_length=35) at /data/src/10.1/sql/sql_parse.cc:1490
            #25 0x00007f5c220d1865 in do_command (thd=0x7f5c163a6070) at /data/src/10.1/sql/sql_parse.cc:1109
            #26 0x00007f5c222086aa in do_handle_one_connection (thd_arg=0x7f5c163a6070) at /data/src/10.1/sql/sql_connect.cc:1349
            #27 0x00007f5c2220840e in handle_one_connection (arg=0x7f5c163a6070) at /data/src/10.1/sql/sql_connect.cc:1261
            #28 0x00007f5c225be10c in pfs_spawn_thread (arg=0x7f5c182b8970) at /data/src/10.1/storage/perfschema/pfs.cc:1860
            #29 0x00007f5c217ce064 in start_thread (arg=0x7f5c22fdbb00) at pthread_create.c:309
            #30 0x00007f5c1f98662d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
            

            On a release build, the tests return wrong results, e.g.

            main.ctype_utf8 'innodb_plugin'          [ fail ]
                    Test ended at 2017-04-17 17:02:57
             
            CURRENT_TEST: main.ctype_utf8
            --- /data/bld/10.1-rel/mysql-test/r/ctype_utf8.result	2017-03-11 16:20:47.000000000 +0200
            +++ /data/bld/10.1-rel/mysql-test/r/ctype_utf8.reject	2017-04-17 17:02:57.068614168 +0300
            @@ -448,7 +448,6 @@
             aaaaaaaaaa
             select c cb20 from t1 where c=repeat('b',20);
             cb20
            -bbbbbbbbbbbbbbbbbbbb
             drop table t1;
             create table t1 (c char(3) character set utf8, unique (c(2)));
             insert into t1 values ('1'),('2'),('3'),('4'),('x'),('y'),('z');
            @@ -774,7 +773,6 @@
             INSERT INTO t1 VALUES ('str2');
             select * from t1 where str='str';
             str
            -str
             drop table t1;
             create table t1 (
             str varchar(255) character set utf8 not null,
            @@ -804,7 +802,6 @@
             INSERT INTO t1 VALUES ('str2');
             select * from t1 where str='str';
             str
            -str
             drop table t1;
             CREATE TABLE t1 (a varchar(32) BINARY) CHARACTER SET utf8;
             INSERT INTO t1 VALUES ('test');
            @@ -887,11 +884,9 @@
             insert into t1 values(1,'foo'),(2,'foobar');
             select * from t1 where b like 'foob%';
             a	b
            -2	foobar
             alter table t1 engine=innodb;
             select * from t1 where b like 'foob%';
             a	b
            -2	foobar
             drop table t1;
             create table t1 (
             a enum('петя','вася','анюта') character set utf8 not null default 'анюта',
            @@ -2067,7 +2062,6 @@
             WHERE t1.colA < 3;
             colA	colB	colA	colB
             1	foo	1	foo
            -2	foo bar	2	foo bar
             DROP TABLE t1, t2;
             SELECT 'н1234567890' UNION SELECT _binary '1';
             н1234567890
             
            mysqltest: Result length mismatch
            

            We also have another bug report about wrong results with this option, MDEV-12486.

            elenst Elena Stepanova added a comment - 10.1 3bb32e8682f849413ce994277e9029ba5732baae 2017-04-17 17:06:00 7f5c22fdbb00 InnoDB: Assertion failure in thread 140033700772608 in file row0sel.cc line 2709 InnoDB: Failing assertion: len * templ->mbmaxlen >= templ->mysql_col_len || (field_no == templ->icp_rec_field_no && field->prefix_len > 0) InnoDB: We intentionally generate a memory trap.   #5 0x00007f5c1f8d4448 in __GI_abort () at abort.c:89 #6 0x00007f5c19030393 in row_sel_field_store_in_mysql_format_func (dest=0x7f5c0a0a1e89 "a", '\217' <repeats 29 times>, "\245\377", '\245' <repeats 31 times>, "\330\036\n\n\\\177", templ=0x7f5c0a28a378, index=0x7f5c0a1269f8, field_no=0, data=0x7f5c16b2407f "a", len=1) at /data/src/10.1/storage/innobase/row/row0sel.cc:2707 #7 0x00007f5c190309ce in row_sel_store_mysql_field_func (mysql_rec=0x7f5c0a0a1e88 "\377a", '\217' <repeats 29 times>, "\245\377", '\245' <repeats 31 times>, "\330\036\n\n\\\177", prebuilt=0x7f5c0a084678, rec=0x7f5c16b2407f "a", index=0x7f5c0a1269f8, offsets=0x7f5c22fd85c0, field_no=0, templ=0x7f5c0a28a378) at /data/src/10.1/storage/innobase/row/row0sel.cc:2882 #8 0x00007f5c19030b7b in row_sel_store_mysql_rec (mysql_rec=0x7f5c0a0a1e88 "\377a", '\217' <repeats 29 times>, "\245\377", '\245' <repeats 31 times>, "\330\036\n\n\\\177", prebuilt=0x7f5c0a084678, rec=0x7f5c16b2407f "a", rec_clust=0, index=0x7f5c0a1269f8, offsets=0x7f5c22fd85c0) at /data/src/10.1/storage/innobase/row/row0sel.cc:2946 #9 0x00007f5c19034b61 in row_search_for_mysql (buf=0x7f5c0a0a1e88 "\377a", '\217' <repeats 29 times>, "\245\377", '\245' <repeats 31 times>, "\330\036\n\n\\\177", mode=2, prebuilt=0x7f5c0a084678, match_mode=1, direction=0) at /data/src/10.1/storage/innobase/row/row0sel.cc:5012 #10 0x00007f5c18efe7c0 in ha_innodb::index_read (this=0x7f5c0a0b3888, buf=0x7f5c0a0a1e88 "\377a", '\217' <repeats 29 times>, "\245\377", '\245' <repeats 31 times>, "\330\036\n\n\\\177", key_ptr=0x7f5c0a0fc310 "", key_len=4, find_flag=HA_READ_KEY_EXACT) at /data/src/10.1/storage/innobase/handler/ha_innodb.cc:9310 #11 0x00007f5c18f14b44 in handler::index_read_map (this=0x7f5c0a0b3888, buf=0x7f5c0a0a1e88 "\377a", '\217' <repeats 29 times>, "\245\377", '\245' <repeats 31 times>, "\330\036\n\n\\\177", key=0x7f5c0a0fc310 "", keypart_map=1, find_flag=HA_READ_KEY_EXACT) at /data/src/10.1/sql/handler.h:3052 #12 0x00007f5c2231b2b0 in handler::index_read_idx_map (this=0x7f5c0a0b3888, buf=0x7f5c0a0a1e88 "\377a", '\217' <repeats 29 times>, "\245\377", '\245' <repeats 31 times>, "\330\036\n\n\\\177", index=0, key=0x7f5c0a0fc310 "", keypart_map=1, find_flag=HA_READ_KEY_EXACT) at /data/src/10.1/sql/handler.cc:5443 #13 0x00007f5c22314dfb in handler::ha_index_read_idx_map (this=0x7f5c0a0b3888, buf=0x7f5c0a0a1e88 "\377a", '\217' <repeats 29 times>, "\245\377", '\245' <repeats 31 times>, "\330\036\n\n\\\177", index=0, key=0x7f5c0a0fc310 "", keypart_map=1, find_flag=HA_READ_KEY_EXACT) at /data/src/10.1/sql/handler.cc:2646 #14 0x00007f5c22140faa in join_read_const (tab=0x7f5c0a0fb2b8) at /data/src/10.1/sql/sql_select.cc:18893 #15 0x00007f5c22140aa9 in join_read_const_table (thd=0x7f5c163a6070, tab=0x7f5c0a0fb2b8, pos=0x7f5c0a0fb808) at /data/src/10.1/sql/sql_select.cc:18768 #16 0x00007f5c2211c515 in make_join_statistics (join=0x7f5c0a0fac68, tables_list=..., keyuse_array=0x7f5c0a0fafa0) at /data/src/10.1/sql/sql_select.cc:3936 #17 0x00007f5c2211345b in JOIN::optimize_inner (this=0x7f5c0a0fac68) at /data/src/10.1/sql/sql_select.cc:1366 #18 0x00007f5c22112346 in JOIN::optimize (this=0x7f5c0a0fac68) at /data/src/10.1/sql/sql_select.cc:1045 #19 0x00007f5c2211a9aa in mysql_select (thd=0x7f5c163a6070, rref_pointer_array=0x7f5c163aa430, tables=0x7f5c0a0fa298, wild_num=0, fields=..., conds=0x7f5c0a0faa28, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7f5c0a0fac48, unit=0x7f5c163a9ab8, select_lex=0x7f5c163aa1b8) at /data/src/10.1/sql/sql_select.cc:3431 #20 0x00007f5c22110496 in handle_select (thd=0x7f5c163a6070, lex=0x7f5c163a99f0, result=0x7f5c0a0fac48, setup_tables_done_option=0) at /data/src/10.1/sql/sql_select.cc:384 #21 0x00007f5c220e03fb in execute_sqlcom_select (thd=0x7f5c163a6070, all_tables=0x7f5c0a0fa298) at /data/src/10.1/sql/sql_parse.cc:5920 #22 0x00007f5c220d68f7 in mysql_execute_command (thd=0x7f5c163a6070) at /data/src/10.1/sql/sql_parse.cc:2979 #23 0x00007f5c220e3b19 in mysql_parse (thd=0x7f5c163a6070, rawbuf=0x7f5c0a0fa088 "select c as c_a from t1 where c='a'", length=35, parser_state=0x7f5c22fda5e0) at /data/src/10.1/sql/sql_parse.cc:7343 #24 0x00007f5c220d2b06 in dispatch_command (command=COM_QUERY, thd=0x7f5c163a6070, packet=0x7f5c1829b071 "select c as c_a from t1 where c='a'", packet_length=35) at /data/src/10.1/sql/sql_parse.cc:1490 #25 0x00007f5c220d1865 in do_command (thd=0x7f5c163a6070) at /data/src/10.1/sql/sql_parse.cc:1109 #26 0x00007f5c222086aa in do_handle_one_connection (thd_arg=0x7f5c163a6070) at /data/src/10.1/sql/sql_connect.cc:1349 #27 0x00007f5c2220840e in handle_one_connection (arg=0x7f5c163a6070) at /data/src/10.1/sql/sql_connect.cc:1261 #28 0x00007f5c225be10c in pfs_spawn_thread (arg=0x7f5c182b8970) at /data/src/10.1/storage/perfschema/pfs.cc:1860 #29 0x00007f5c217ce064 in start_thread (arg=0x7f5c22fdbb00) at pthread_create.c:309 #30 0x00007f5c1f98662d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 On a release build, the tests return wrong results, e.g. main.ctype_utf8 'innodb_plugin' [ fail ] Test ended at 2017-04-17 17:02:57   CURRENT_TEST: main.ctype_utf8 --- /data/bld/10.1-rel/mysql-test/r/ctype_utf8.result 2017-03-11 16:20:47.000000000 +0200 +++ /data/bld/10.1-rel/mysql-test/r/ctype_utf8.reject 2017-04-17 17:02:57.068614168 +0300 @@ -448,7 +448,6 @@ aaaaaaaaaa select c cb20 from t1 where c=repeat('b',20); cb20 -bbbbbbbbbbbbbbbbbbbb drop table t1; create table t1 (c char(3) character set utf8, unique (c(2))); insert into t1 values ('1'),('2'),('3'),('4'),('x'),('y'),('z'); @@ -774,7 +773,6 @@ INSERT INTO t1 VALUES ('str2'); select * from t1 where str='str'; str -str drop table t1; create table t1 ( str varchar(255) character set utf8 not null, @@ -804,7 +802,6 @@ INSERT INTO t1 VALUES ('str2'); select * from t1 where str='str'; str -str drop table t1; CREATE TABLE t1 (a varchar(32) BINARY) CHARACTER SET utf8; INSERT INTO t1 VALUES ('test'); @@ -887,11 +884,9 @@ insert into t1 values(1,'foo'),(2,'foobar'); select * from t1 where b like 'foob%'; a b -2 foobar alter table t1 engine=innodb; select * from t1 where b like 'foob%'; a b -2 foobar drop table t1; create table t1 ( a enum('петя','вася','анюта') character set utf8 not null default 'анюта', @@ -2067,7 +2062,6 @@ WHERE t1.colA < 3; colA colB colA colB 1 foo 1 foo -2 foo bar 2 foo bar DROP TABLE t1, t2; SELECT 'н1234567890' UNION SELECT _binary '1'; н1234567890   mysqltest: Result length mismatch We also have another bug report about wrong results with this option, MDEV-12486 .

            Assert in row_sel_field_store_in_mysql_format_func():

                            ut_ad(len * templ->mbmaxlen >= templ->mysql_col_len
                                  || (field_no == templ->icp_rec_field_no
                                      && field->prefix_len > 0));
            

            looks very strict and wrong for --innodb_prefix_index_cluster_optimization case.

            There is no index condition pushdown. so let's ignore the second condition.

            The problematic assert is (len * templ->mbmaxlen >= templ->mysql_col_len).

            --source include/have_innodb.inc
            set global innodb_prefix_index_cluster_optimization = 1;
            let collation=utf8_unicode_ci;
            --source include/have_collation.inc
             
            set names utf8;
            --source include/ctype_regex.inc
            --source include/ctype_regex_utf8.inc
             
            create table t1 (
            c char(10) character set utf8,
            unique key a (c(1)))engine=innodb;
            insert into t1 values ('a');
            select c as c_a from t1 where c='a';
            

            For the above test case, InnoDB fetches record from secondary index which is of size "1" and field->prefix_len is 3.
            So, it satisfies the condition to avoid lookup for clustered index.

            Using secondary index record, row_sel_store_mysql_rec() to convert the innodb record to mysql record.
            length will be 1 as expected. templ->mbmaxlen will be 3, but templ->mysql_col_len will be 30. Above condition fails.

             --source include/have_innodb.inc
            set global innodb_prefix_index_cluster_optimization = 1;
            create table t1 (
            c char(10) character set utf8,
            c1 char(10) character set utf8,
            index (c(4), c1(4))
            ) engine=innodb;
            insert into t1 values ('a', 't');
            select c, c1 as c_a from t1 where c='a';
            drop table t1;
            

            Above test case also fails with assert.

            thiru Thirunarayanan Balathandayuthapani added a comment - Assert in row_sel_field_store_in_mysql_format_func(): ut_ad(len * templ->mbmaxlen >= templ->mysql_col_len || (field_no == templ->icp_rec_field_no && field->prefix_len > 0)); looks very strict and wrong for --innodb_prefix_index_cluster_optimization case. There is no index condition pushdown. so let's ignore the second condition. The problematic assert is (len * templ->mbmaxlen >= templ->mysql_col_len). --source include/have_innodb.inc set global innodb_prefix_index_cluster_optimization = 1; let collation=utf8_unicode_ci; --source include/have_collation.inc   set names utf8; --source include/ctype_regex.inc --source include/ctype_regex_utf8.inc   create table t1 ( c char(10) character set utf8, unique key a (c(1)))engine=innodb; insert into t1 values ('a'); select c as c_a from t1 where c='a'; For the above test case, InnoDB fetches record from secondary index which is of size "1" and field->prefix_len is 3. So, it satisfies the condition to avoid lookup for clustered index. Using secondary index record, row_sel_store_mysql_rec() to convert the innodb record to mysql record. length will be 1 as expected. templ->mbmaxlen will be 3, but templ->mysql_col_len will be 30. Above condition fails. --source include/have_innodb.inc set global innodb_prefix_index_cluster_optimization = 1; create table t1 ( c char(10) character set utf8, c1 char(10) character set utf8, index (c(4), c1(4)) ) engine=innodb; insert into t1 values ('a', 't'); select c, c1 as c_a from t1 where c='a'; drop table t1; Above test case also fails with assert.

            People

              thiru Thirunarayanan Balathandayuthapani
              laurynas Laurynas Biveinis
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.