[MDEV-23150] Assertion `(*endptr - (const char *) s) % 2 == 0' failed in my_strtoll10_mb2 Created: 2020-07-12  Updated: 2023-11-28

Status: Confirmed
Project: MariaDB Server
Component/s: Character Sets
Affects Version/s: 10.1, 10.2, 10.3, 10.4, 10.5, 10.6, 10.7, 10.8, 10.9, 10.10, 10.11, 11.0, 11.1, 11.2
Fix Version/s: 10.4, 10.5, 10.6, 10.11, 11.0, 11.1, 11.2

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Alexander Barkov
Resolution: Unresolved Votes: 0
Labels: upstream-8.0


 Description   

SET CHARACTER_SET_CONNECTION= utf16;
PREPARE stmt FROM "SELECT SUBSTR('foo', 1, IFNULL(10000000000000000,'1'))";

10.2 737c3025

mysqld: /data/src/10.2/strings/ctype-ucs2.c:848: my_strtoll10_mb2: Assertion `(*endptr - (const char *) s) % 2 == 0' failed.
200712 14:27:24 [ERROR] mysqld got signal 6 ;
 
#7  0x00007f6f2568bf12 in __GI___assert_fail (assertion=0x555ca29f1a68 "(*endptr - (const char *) s) % 2 == 0", file=0x555ca29f1a40 "/data/src/10.2/strings/ctype-ucs2.c", line=848, function=0x555ca29f2090 <__PRETTY_FUNCTION__.11298> "my_strtoll10_mb2") at assert.c:101
#8  0x0000555ca256a589 in my_strtoll10_mb2 (cs=0x555ca303e3c0 <my_charset_utf16_general_ci>, nptr=0x7f6ec015f8f0 "1", '0' <repeats 16 times>, "\245\245\245\245\245\245\245h4z\025\217\217\217\217\240", endptr=0x7f6f2453fda0, error=0x7f6f2453fda8) at /data/src/10.2/strings/ctype-ucs2.c:848
#9  0x0000555ca1a9efe4 in Value_source::Converter_strtoll10::Converter_strtoll10 (this=0x7f6f2453fda0, cs=0x555ca303e3c0 <my_charset_utf16_general_ci>, str=0x7f6ec015f8f0 "1", '0' <repeats 16 times>, "\245\245\245\245\245\245\245h4z\025\217\217\217\217\240", length=17) at /data/src/10.2/sql/field.h:196
#10 0x0000555ca1a9f0ae in Value_source::Converter_strtoll10_with_warn::Converter_strtoll10_with_warn (this=0x7f6f2453fda0, thd=0x7f6ec0000af0, filter=..., cs=0x555ca303e3c0 <my_charset_utf16_general_ci>, str=0x7f6ec015f8f0 "1", '0' <repeats 16 times>, "\245\245\245\245\245\245\245h4z\025\217\217\217\217\240", length=17) at /data/src/10.2/sql/field.h:253
#11 0x0000555ca1a9f14e in Value_source::longlong_from_string_with_check (this=0x7f6ec0082050, cs=0x555ca303e3c0 <my_charset_utf16_general_ci>, cptr=0x7f6ec015f8f0 "1", '0' <repeats 16 times>, "\245\245\245\245\245\245\245h4z\025\217\217\217\217\240", end=0x7f6ec015f901 "\245\245\245\245\245\245\245h4z\025\217\217\217\217\240") at /data/src/10.2/sql/field.h:291
#12 0x0000555ca1e17c78 in Value_source::longlong_from_string_with_check (this=0x7f6ec0082050, str=0x7f6ec0082080) at /data/src/10.2/sql/field.h:325
#13 0x0000555ca1e5ce5d in Item_func_hybrid_field_type::val_int (this=0x7f6ec0082050) at /data/src/10.2/sql/item_func.cc:959
#14 0x0000555ca1e9095c in Item_func_substr::fix_length_and_dec (this=0x7f6ec0082120) at /data/src/10.2/sql/item_strfunc.cc:1710
#15 0x0000555ca1e5a6b4 in Item_func::fix_fields (this=0x7f6ec0082120, thd=0x7f6ec0000af0, ref=0x7f6ec0082220) at /data/src/10.2/sql/item_func.cc:230
#16 0x0000555ca1e8ab5d in Item_str_func::fix_fields (this=0x7f6ec0082120, thd=0x7f6ec0000af0, ref=0x7f6ec0082220) at /data/src/10.2/sql/item_strfunc.cc:106
#17 0x0000555ca1b041fb in setup_fields (thd=0x7f6ec0000af0, ref_pointer_array=..., fields=..., mark_used_columns=MARK_COLUMNS_READ, sum_func_list=0x7f6ec0012980, pre_fix=0x7f6ec0080f50, allow_sum_func=true) at /data/src/10.2/sql/sql_base.cc:7238
#18 0x0000555ca1ba98ed in JOIN::prepare (this=0x7f6ec0012660, tables_init=0x0, wild_num=0, conds_init=0x0, og_num=0, order_init=0x0, skip_order_by=false, group_init=0x0, having_init=0x0, proc_param_init=0x0, select_lex_arg=0x7f6ec0080e10, unit_arg=0x7f6ec00806d0) at /data/src/10.2/sql/sql_select.cc:811
#19 0x0000555ca1c55d03 in st_select_lex_unit::prepare (this=0x7f6ec00806d0, thd_arg=0x7f6ec0000af0, sel_result=0x0, additional_options=0) at /data/src/10.2/sql/sql_union.cc:586
#20 0x0000555ca1b8dbdc in mysql_test_select (stmt=0x7f6ec0006600, tables=0x0) at /data/src/10.2/sql/sql_prepare.cc:1602
#21 0x0000555ca1b8f7e0 in check_prepared_statement (stmt=0x7f6ec0006600) at /data/src/10.2/sql/sql_prepare.cc:2400
#22 0x0000555ca1b9362d in Prepared_statement::prepare (this=0x7f6ec0006600, packet=0x7f6ec00125f0 "SELECT SUBSTR('foo', 1, IFNULL(1", '0' <repeats 16 times>, ",'1'))", packet_len=54) at /data/src/10.2/sql/sql_prepare.cc:4064
#23 0x0000555ca1b905cc in mysql_sql_stmt_prepare (thd=0x7f6ec0000af0) at /data/src/10.2/sql/sql_prepare.cc:2878
#24 0x0000555ca1b6a8ce in mysql_execute_command (thd=0x7f6ec0000af0) at /data/src/10.2/sql/sql_parse.cc:3535
#25 0x0000555ca1b77d53 in mysql_parse (thd=0x7f6ec0000af0, rawbuf=0x7f6ec0012238 "PREPARE stmt FROM \"SELECT SUBSTR('foo', 1, IFNULL(1", '0' <repeats 16 times>, ",'1'))\"", length=74, parser_state=0x7f6f24541660, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:7733
#26 0x0000555ca1b6607f in dispatch_command (command=COM_QUERY, thd=0x7f6ec0000af0, packet=0x7f6ec0066f41 "PREPARE stmt FROM \"SELECT SUBSTR('foo', 1, IFNULL(1", '0' <repeats 16 times>, ",'1'))\"", packet_length=74, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:1824
#27 0x0000555ca1b64afa in do_command (thd=0x7f6ec0000af0) at /data/src/10.2/sql/sql_parse.cc:1377
#28 0x0000555ca1cbaa6f in do_handle_one_connection (connect=0x555ca53a0740) at /data/src/10.2/sql/sql_connect.cc:1336
#29 0x0000555ca1cba7da in handle_one_connection (arg=0x555ca53a0740) at /data/src/10.2/sql/sql_connect.cc:1241
#30 0x00007f6f276144a4 in start_thread (arg=0x7f6f24542700) at pthread_create.c:456
#31 0x00007f6f25748d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

Also reproducible with ucs2.
Reproducible on 10.1-10.5 debug builds.
No obvious problem on a non-debug build, other than an ugly warning message

Warnings:
Warning	1292	Truncated incorrect INTEGER value: '?????????'

Same with ucs2



 Comments   
Comment by Roel Van de Paar [ 2023-01-28 ]

Also ran into this one. May be lightly sporadic.

SET SESSION collation_connection=ucs2_icelandic_ci;
SELECT(TRUNCATE(1,TIME_FORMAT(1,1)));

Leads to:

11.0.1 b075191ba8598af6aff5549e6e19f6255aef258a (Debug)

Core was generated by `/test/MD090123-mariadb-11.0.1-linux-x86_64-dbg/bin/mariadbd --no-defaults --lc-'.
Program terminated with signal SIGABRT, Aborted.
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=22703691038080)
    at ./nptl/pthread_kill.c:44
[Current thread is 1 (Thread 0x14a61d707d80 (LWP 1263227))]
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=22703691038080) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=22703691038080) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=22703691038080, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x000014a61d9f8476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x000014a61d9de7f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x000014a61d9de71b in __assert_fail_base (fmt=0x14a61db93150 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x5574c2e949a0 "! is_set() || m_can_overwrite_status", file=0x5574c2e94710 "/test/11.0_dbg/sql/sql_error.cc", line=457, function=<optimized out>) at ./assert/assert.c:92
#6  0x000014a61d9efe96 in __GI___assert_fail (assertion=0x5574c2e949a0 "! is_set() || m_can_overwrite_status", file=0x5574c2e94710 "/test/11.0_dbg/sql/sql_error.cc", line=457, function=0x5574c2e94918 "void Diagnostics_area::set_error_status(uint, const char*, const char*, const Sql_user_condition_identity&, const Sql_condition*)") at ./assert/assert.c:101
#7  0x00005574c23fdee4 in Diagnostics_area::set_error_status (this=this@entry=0x5574c55362e8, sql_errno=sql_errno@entry=6, message=message@entry=0x7ffd57720210 "Error on delete of '/tmp/#sql-temptable-13467b-1-2.MAD' (Errcode: 2 \"No such file or directory\")", sqlstate=sqlstate@entry=0x5574c2eb0038 "HY000", ucid=@0x7ffd57720000: {m_user_condition_value = 0x0}, error_condition=0x0) at /test/11.0_dbg/sql/sql_error.cc:457
#8  0x00005574c23defbe in THD::raise_condition (this=this@entry=0x5574c55304e8, cond=cond@entry=0x7ffd5771fff0) at /test/11.0_dbg/sql/sql_class.cc:1118
#9  0x00005574c2332bd0 in THD::raise_condition (this=this@entry=0x5574c55304e8, sql_errno=sql_errno@entry=6, sqlstate=sqlstate@entry=0x5574c2e79918 "", level=<optimized out>, msg=msg@entry=0x7ffd57720210 "Error on delete of '/tmp/#sql-temptable-13467b-1-2.MAD' (Errcode: 2 \"No such file or directory\")") at /test/11.0_dbg/sql/sql_class.h:4872
#10 0x00005574c2327996 in my_message_sql (error=6, str=0x7ffd57720210 "Error on delete of '/tmp/#sql-temptable-13467b-1-2.MAD' (Errcode: 2 \"No such file or directory\")", MyFlags=4) at /test/11.0_dbg/sql/mysqld.cc:3391
#11 0x00005574c2da4411 in my_error (nr=nr@entry=6, MyFlags=MyFlags@entry=4) at /test/11.0_dbg/mysys/my_error.c:124
#12 0x00005574c2da40c8 in my_delete (name=name@entry=0x7ffd57720770 "/tmp/#sql-temptable-13467b-1-2.MAD", MyFlags=MyFlags@entry=16) at /test/11.0_dbg/mysys/my_delete.c:53
#13 0x00005574c2daba96 in my_handler_delete_with_symlink (filename=0x7ffd57720770 "/tmp/#sql-temptable-13467b-1-2.MAD", sync_dir=sync_dir@entry=16) at /test/11.0_dbg/mysys/my_symlink2.c:190
#14 0x00005574c296abea in inline_mysql_file_delete_with_symlink (name=name@entry=0x5574c5561d10 "/tmp/#sql-temptable-13467b-1-2", ext=ext@entry=0x5574c307e905 ".MAD", flags=flags@entry=16) at /test/11.0_dbg/include/mysql/psi/mysql_file.h:1396
#15 0x00005574c296ac40 in maria_delete_table_files (name=0x5574c5561d10 "/tmp/#sql-temptable-13467b-1-2", temporary=<optimized out>, flags=16) at /test/11.0_dbg/storage/maria/ma_delete_table.c:103
#16 0x00005574c291242b in ha_maria::drop_table (this=0x5574c5561d90, name=0x5574c5561d10 "/tmp/#sql-temptable-13467b-1-2") at /test/11.0_dbg/storage/maria/ha_maria.cc:2861
#17 0x00005574c2714ab4 in handler::ha_drop_table (this=0x5574c5561d90, name=0x5574c5561d10 "/tmp/#sql-temptable-13467b-1-2") at /test/11.0_dbg/sql/handler.cc:5365
#18 0x00005574c24ab6f3 in free_tmp_table (thd=thd@entry=0x5574c55304e8, entry=0x5574c5560bd0) at /test/11.0_dbg/sql/sql_select.cc:21132
#19 0x00005574c23c7678 in close_thread_tables (thd=thd@entry=0x5574c55304e8) at /test/11.0_dbg/sql/sql_base.cc:880
#20 0x00005574c23c7926 in close_thread_tables_for_query (thd=thd@entry=0x5574c55304e8) at /test/11.0_dbg/sql/sql_base.cc:790
#21 0x00005574c244d796 in mysql_execute_command (thd=thd@entry=0x5574c55304e8, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /test/11.0_dbg/sql/sql_parse.cc:6067
#22 0x00005574c244e934 in mysql_parse (thd=0x5574c55304e8, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x7ffd57720fb0) at /test/11.0_dbg/sql/sql_parse.cc:8000
#23 0x00005574c244ee6a in bootstrap (file=0x5574c40ad660 <instrumented_stdin>) at /test/11.0_dbg/sql/sql_class.h:243
#24 0x00005574c233003e in mysqld_main (argc=<optimized out>, argv=<optimized out>) at /test/11.0_dbg/sql/mysqld.cc:5921
#25 0x00005574c2324736 in main (argc=<optimized out>, argv=<optimized out>) at /test/11.0_dbg/sql/main.cc:34

Note that MySQL 8.0.31 (but not older versions) is affected by this bug also:

MySQL 8.0.31 (Debug)

mysqld: /test/8.0_dbg/strings/ctype-ucs2.cc:646: longlong my_strtoll10_mb2(const CHARSET_INFO*, const char*, const char**, int*): Assertion `(*endptr - s) % 2 == 0' failed.

MySQL 8.0.31 (Debug)

Core was generated by `/test/MS070123-mysql-8.0.31-linux-x86_64-dbg/bin/mysqld --no-defaults --core-fi'.
Program terminated with signal SIGABRT, Aborted.
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=22960131728960)
    at ./nptl/pthread_kill.c:44
[Current thread is 1 (Thread 0x14e1d27ee640 (LWP 1572194))]
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=22960131728960) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=22960131728960) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=22960131728960, signo=6) at ./nptl/pthread_kill.c:89
#3  0x000055fa9886ab78 in my_write_core (sig=sig@entry=6) at /test/8.0_dbg/mysys/stacktrace.cc:295
#4  0x000055fa974eddc2 in handle_fatal_signal (sig=6) at /test/8.0_dbg/sql/signal_handler.cc:230
#5  <signal handler called>
#6  __pthread_kill_implementation (no_tid=0, signo=6, threadid=22960131728960) at ./nptl/pthread_kill.c:44
#7  __pthread_kill_internal (signo=6, threadid=22960131728960) at ./nptl/pthread_kill.c:78
#8  __GI___pthread_kill (threadid=22960131728960, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#9  0x000014e24720d476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#10 0x000014e2471f37f3 in __GI_abort () at ./stdlib/abort.c:79
#11 0x000014e2471f371b in __assert_fail_base (fmt=0x14e2473a8150 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x55fa9a54d5ba "(*endptr - s) % 2 == 0", file=0x55fa9a54cf98 "/test/8.0_dbg/strings/ctype-ucs2.cc", line=646, function=<optimized out>) at ./assert/assert.c:92
#12 0x000014e247204e96 in __GI___assert_fail (assertion=0x55fa9a54d5ba "(*endptr - s) % 2 == 0", file=0x55fa9a54cf98 "/test/8.0_dbg/strings/ctype-ucs2.cc", line=646, function=0x55fa9a54cf48 "longlong my_strtoll10_mb2(const CHARSET_INFO*, const char*, const char**, int*)") at ./assert/assert.c:101
#13 0x000055fa992ae3cd in my_strtoll10_mb2 (cs=0x55fa9aed6dc0 <my_charset_ucs2_icelandic_uca_ci>, nptr=0x14e188019930 "1", endptr=0x14e1d27eaab8, error=0x14e1d27eaab4) at /test/8.0_dbg/strings/ctype-ucs2.cc:646
#14 0x000055fa9768ad3f in longlong_from_string_with_check (cs=0x55fa9aed6dc0 <my_charset_ucs2_icelandic_uca_ci>, cptr=0x14e188019930 "1", end=0x14e188019931 "", unsigned_target=0) at /test/8.0_dbg/sql/item.cc:3509
#15 0x000055fa9768b27f in Item::val_int_from_string (this=0x14e18800aff8) at /test/8.0_dbg/sql/item.cc:491
#16 0x000055fa97529eb1 in Item_str_func::val_int (this=<optimized out>) at /test/8.0_dbg/sql/item_strfunc.h:114
#17 0x000055fa9772680e in Item_func_round::int_op (this=0x14e18800a538) at /test/8.0_dbg/sql/item_func.cc:3499
#18 0x000055fa97735ba2 in Item_func_numhybrid::val_int (this=0x14e18800a538) at /test/8.0_dbg/sql/item_func.cc:1730
#19 0x000055fa9767842e in Item::send (this=0x14e18800a538, protocol=0x14e188005640, buffer=<optimized out>) at /test/8.0_dbg/sql/item.cc:7313
#20 0x000055fa9724c8ea in THD::send_result_set_row (this=this@entry=0x14e188015930, row_items=@0x14e188009918: {m_blocks = 0x14e18800b168, m_begin_idx = 64, m_end_idx = 65, m_capacity = 128, m_root = 0x14e188018110, m_generation = 2}) at /test/8.0_dbg/sql/sql_class.cc:2878
#21 0x000055fa97935528 in Query_result_send::send_data (this=<optimized out>, thd=0x14e188015930, items=@0x14e188009918: {m_blocks = 0x14e18800b168, m_begin_idx = 64, m_end_idx = 65, m_capacity = 128, m_root = 0x14e188018110, m_generation = 2}) at /test/8.0_dbg/sql/query_result.cc:100
#22 0x000055fa97419978 in Query_expression::ExecuteIteratorQuery (this=this@entry=0x14e1880097f0, thd=thd@entry=0x14e188015930) at /test/8.0_dbg/sql/sql_union.cc:1769
#23 0x000055fa97419d4b in Query_expression::execute (this=this@entry=0x14e1880097f0, thd=thd@entry=0x14e188015930) at /test/8.0_dbg/sql/sql_union.cc:1807
#24 0x000055fa9738178f in Sql_cmd_dml::execute_inner (this=0x14e188026608, thd=0x14e188015930) at /test/8.0_dbg/sql/sql_select.cc:787
#25 0x000055fa9738cabe in Sql_cmd_dml::execute (this=0x14e188026608, thd=0x14e188015930) at /test/8.0_dbg/sql/sql_select.cc:587
#26 0x000055fa9730a765 in mysql_execute_command (thd=thd@entry=0x14e188015930, first_level=first_level@entry=true) at /test/8.0_dbg/sql/sql_parse.cc:4677
#27 0x000055fa9730c3c5 in dispatch_sql_command (thd=0x14e188015930, parser_state=parser_state@entry=0x14e1d27ec9b0) at /test/8.0_dbg/sql/sql_parse.cc:5312
#28 0x000055fa9730edf6 in dispatch_command (thd=<optimized out>, thd@entry=0x14e188015930, com_data=com_data@entry=0x14e1d27ed320, command=COM_QUERY) at /test/8.0_dbg/sql/sql_parse.cc:2032
#29 0x000055fa97311497 in do_command (thd=thd@entry=0x14e188015930) at /test/8.0_dbg/sql/sql_parse.cc:1435
#30 0x000055fa974db26f in handle_connection (arg=arg@entry=0x55fa9ea4ef30) at /test/8.0_dbg/sql/conn_handler/connection_handler_per_thread.cc:302
#31 0x000055fa992f8d54 in pfs_spawn_thread (arg=0x55fa9eb0d140) at /test/8.0_dbg/storage/perfschema/pfs.cc:2986
#32 0x000014e24725fb43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#33 0x000014e2472f1a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

MySQL 5.7.40 simply outputs "1".

All Stacks/UniqueID's seen (for both testcases) across versions (including in MySQL 8.0) so far:

(*endptr - (const char *) s) % 2 == 0|SIGABRT|my_strtoll10_mb2|Value_source::Converter_strtoll10::Converter_strtoll10|Value_source::Converter_strtoll10_with_warn::Converter_strtoll10_with_warn|Value_source::longlong_from_string_with_check
(*endptr - (const char *) s) % 2 == 0|SIGABRT|my_strtoll10_mb2|charset_info_st::strtoll10|Value_source::Converter_strtoll10::Converter_strtoll10|Value_source::Converter_strtoll10_with_warn::Converter_strtoll10_with_warn
(*endptr - s) % 2 == 0|SIGABRT|my_strtoll10_mb2|longlong_from_string_with_check|Item::val_int_from_string|Item_str_func::val_int

No UBSAN/ASAN issues observed

Comment by Roel Van de Paar [ 2023-01-28 ]

Slightly improved original testcase

SET CHARACTER_SET_CONNECTION=utf16;
PREPARE s FROM "SELECT SUBSTR(1,1,IFNULL(1,'a'))";

Comment by Roel Van de Paar [ 2023-08-04 ]

Please also test any bugfix against

SET collation_connection=ucs2_czech_ci;
SELECT (COT (-1)%(1/1)+(MAKETIME(0,TIME_FORMAT(0,0),0)%ABS(-1)/1/CEILING (1)))*(DATEDIFF (0,0)/DAYOFYEAR (1))*(CON
V(0,0,0)*1);

Comment by Alexander Barkov [ 2023-08-15 ]

Also repeatable with:

SET CHARACTER_SET_CONNECTION= utf16;
PREPARE stmt FROM "SELECT SUBSTR('foo', 1, IF(1,10000000000000000,'1'))";

Generated at Thu Feb 08 09:20:13 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.