[MDEV-18899] Server crashes in Field::set_warning_truncated_wrong_value Created: 2019-03-12  Updated: 2020-08-25  Resolved: 2019-03-28

Status: Closed
Project: MariaDB Server
Component/s: Data types, Optimizer
Affects Version/s: 10.2, 10.3, 10.4
Fix Version/s: 10.2.24, 10.3.14, 10.4.4

Type: Bug Priority: Critical
Reporter: Alice Sherepa Assignee: Varun Gupta (Inactive)
Resolution: Fixed Votes: 1
Labels: None

Issue Links:
Relates
relates to MDEV-20589 Server still crashes in Field::set_wa... Closed
relates to MDEV-19061 table_share used for reading statisti... Closed

 Description   

Reproducible on 10.2-10.4, with MyIsam/Innodb

set names utf8;
 
set use_stat_tables=complementary;
 
 
 
create table t1 ( a varchar(255) character set utf8);
 
insert into t1 values ('ӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥӥ');
 
 
 
analyze table t1;
 
select 1 from t1;

10.2 bf71d263621c90cbddc7bde

 
    #0 0x5577ef4348a6 in Field::set_warning_truncated_wrong_value(char const*, char const*) /home/alice/git/10.2/sql/field.cc:10949
 
    #1 0x5577ef413a76 in Field_longstr::check_string_copy_error(String_copier const*, char const*, charset_info_st const*) /home/alice/git/10.2/sql/field.cc:7031
 
    #2 0x5577ef43746a in Field_longstr::check_conversion_status(String_copier const*, char const*, charset_info_st const*, bool) /home/alice/git/10.2/sql/field.h:1770
 
    #3 0x5577ef4192d4 in Field_varstring::store(char const*, unsigned int, charset_info_st const*) /home/alice/git/10.2/sql/field.cc:7635
 
    #4 0x5577ef05a995 in Column_stat::get_stat_values() (/home/alice/git/10.2/sql/mysqld+0xf7b995)
 
    #5 0x5577ef04fd7d in read_statistics_for_table /home/alice/git/10.2/sql/sql_statistics.cc:2998
 
    #6 0x5577ef051b09 in read_statistics_for_tables_if_needed(THD*, TABLE_LIST*) /home/alice/git/10.2/sql/sql_statistics.cc:3289
 
    #7 0x5577eed987dc in open_and_lock_tables(THD*, DDL_options_st const&, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*) /home/alice/git/10.2/sql/sql_base.cc:4779
 
    #8 0x5577eed7aa77 in open_and_lock_tables(THD*, TABLE_LIST*, bool, unsigned int) /home/alice/git/10.2/sql/sql_base.h:506
 
    #9 0x5577eeeab591 in execute_sqlcom_select /home/alice/git/10.2/sql/sql_parse.cc:6446
 
    #10 0x5577eee98fac in mysql_execute_command(THD*) /home/alice/git/10.2/sql/sql_parse.cc:3537
 
    #11 0x5577eeeb466c in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /home/alice/git/10.2/sql/sql_parse.cc:8059
 
    #12 0x5577eee8f8d4 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /home/alice/git/10.2/sql/sql_parse.cc:1829
 
    #13 0x5577eee8ca53 in do_command(THD*) /home/alice/git/10.2/sql/sql_parse.cc:1379
 
    #14 0x5577ef1b52f2 in do_handle_one_connection(CONNECT*) /home/alice/git/10.2/sql/sql_connect.cc:1336
 
    #15 0x5577ef1b4cfa in handle_one_connection /home/alice/git/10.2/sql/sql_connect.cc:1242
 
    #16 0x5577f0365327 in pfs_spawn_thread /home/alice/git/10.2/storage/perfschema/pfs.cc:1862
 
    #17 0x7fb000b9d6b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
 
    #18 0x7fb00003241c in clone (/lib/x86_64-linux-gnu/libc.so.6+0x10741c)

Reproducible on non-debug build

10.3.13

 
190312 11:40:40 [ERROR] mysqld got signal 11 ;
 
 
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fc2c4b2a390]
 
sql/field.cc:11253(Field::set_warning_truncated_wrong_value(char const*, char const*))[0x557c68f3767b]
 
sql/field.cc:6963(Field_longstr::check_string_copy_error(String_copier const*, char const*, charset_info_st const*))[0x557c68f37777]
 
sql/field.h:1790(Field_longstr::check_conversion_status(String_copier const*, char const*, charset_info_st const*, bool))[0x557c68f3e6f1]
 
sql/field.h:3357(Field_varstring::store_length(unsigned int))[0x557c68f38056]
 
sql/sql_statistics.cc:1157(Column_stat::get_stat_values())[0x557c68dfaeef]
 
sql/sql_statistics.cc:4088(read_statistics_for_table)[0x557c68df96a4]
 
sql/sql_base.cc:5019(open_and_lock_tables(THD*, DDL_options_st const&, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*))[0x557c68d33d72]
 
sql/sql_base.h:503(open_and_lock_tables(THD*, TABLE_LIST*, bool, unsigned int))[0x557c68d06977]
 
sql/sql_parse.cc:6472(execute_sqlcom_select(THD*, TABLE_LIST*))[0x557c68ca10cd]
 
sql/sql_parse.cc:3772(mysql_execute_command(THD*))[0x557c68d8a78c]
 
sql/sql_class.h:1810(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x557c68d8cdf9]
 
sql/sql_parse.cc:1854(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x557c68d8ef95]
 
sql/sql_parse.cc:1398(do_command(THD*))[0x557c68d8f73e]
 
sql/sql_connect.cc:1403(do_handle_one_connection(CONNECT*))[0x557c68e573ff]
 
sql/sql_connect.cc:1311(handle_one_connection)[0x557c68e57524]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7fc2c4b206ba]
 
x86_64/clone.S:111(clone)[0x7fc2c398541d]
 
 
 
Query (0x7fc2500115d0): select 1 from t1



 Comments   
Comment by Alexander Barkov [ 2019-03-13 ]

The crash happens in Field_varstring::store() which is called from here:

          case COLUMN_STAT_MIN_VALUE:
 
            table_field->read_stats->min_value->set_notnull();
 
            stat_field->val_str(&val);
 
            table_field->read_stats->min_value->store(val.ptr(), val.length(),
 
                                                      &my_charset_bin);
 
            break;

Field_varstring::store() tries to generate a warning, because the value in the column column_statistics.min_value is not well-formed: it has a hard limit of 255 bytes, so in this bug report on the position 255 it contains the first byte (of a two-byte character) without the second byte.

The warning is generated here:

void Field::set_warning_truncated_wrong_value(const char *type_arg,
 
                                              const char *value)
 
{
 
  THD *thd= get_thd();
 
  const char *db_name= table->s->db.str;
 
  const char *table_name= table->s->table_name.str;
 
  ...
 
}

Notice, it tries to get the db and the table from from "table", however "table" is a NULL pointer here.

table_field->read_stats->min_value is created here:

static
 
void create_min_max_statistical_fields_for_table_share(THD *thd,
 
                                                       TABLE_SHARE *table_share)
 
{
 
  ...
 
      for (Field **field_ptr= table_share->field; *field_ptr; field_ptr++)
 
      {
 
        Field *fld;
 
        Field *table_field= *field_ptr;
 
        my_ptrdiff_t diff= record - table_share->default_values;
 
        if (!(fld= table_field->clone(&stats_cb->mem_root, diff)))
 
          continue;
 
        if (i == 0)
 
          table_field->read_stats->min_value= fld;
 
        else
 
          table_field->read_stats->max_value= fld;
 
      }
 
 ...
 
}

The above code clones a Field instance but does not set the "table" pointer for the cloned Field.
Note, the original instance, table_field, does have a non-NULL pointer.

So a solution to fix the crash would probably to copy table_field->table to min_value and max_value.

However, it will fix only a part of the problem.

Well-formedness problem

It is generally wrong that any warnings are generated while copying from column_statistics.min_value.
1. either column_statistics.min_value should store well-formed values (e.g. don't break characters in the middle)
2. or the copying code should suppress warnings

Note, column_statistics.min_value can be modified manually. So probably both #1 and #2 are needed.

Contraction problem

Also, the underlying code should be checked for contraction compatibility. The code copying to column_statistics.min_value should make sure not to break contractions in the middle, otherwise max_value can be very far from the actual maximum value.

For example, consider this data in combination with Czech collation:

CONCAT(REPEAT('x',254), 'ch'))

'ch' is a separate letter which is sorted between 'h' and 'i':
http://collation-charts.org/mysql60/mysql604.utf8_czech_ci.html

'ch' should not be broken into parts when copying to column_statistics.min_value:
'c' cannot be the last 255-th byte in column_statistics.min_value, because it was followed by 'h' in the original full-length data. The copying code should store only the REPEAT('x',254) part.

For column_statistics.max_value, the copying code will be even harder: it should replace 'ch' to the character which immediately follows 'ch' in the collation, which is 'i'.

Comment by Alexander Barkov [ 2019-03-13 ]

igor, please take over this bug. The problem resides in the statistical code. See the analysis above.

Comment by Varun Gupta (Inactive) [ 2019-03-19 ]

Some imput from bar

To fix the crash
1. Fix create_min_max_statistical_fields_for_table_share() to make sure to set table_field->table to a non-NULL pointer
2. Fix writing code to write only full multi-byte sequences, so from the desciption we have letter 'ӥ' (0xD3A5 in utf8), if we want to write a multi-bytes sequence then we must be sure that there is space to write the entire multi-byte sequence, else we should reject the multi-byte character

Another task would be to make sure that min and max values are set correctly, the suggestion is to fix this in 10.4

Comment by Varun Gupta (Inactive) [ 2019-03-23 ]

Patch
http://lists.askmonty.org/pipermail/commits/2019-March/013580.html

Comment by Varun Gupta (Inactive) [ 2019-03-23 ]

Filed MDEV-19028 as a separate issue to address the contraction problem mentioned above

Generated at Thu Feb 08 08:47:34 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.