[MDEV-27307] main.ctype_utf8mb4_uca_allkeys tests fail with Valgrind/MSAN Created: 2021-12-18  Updated: 2021-12-21  Resolved: 2021-12-21

Status: Closed
Project: MariaDB Server
Component/s: Character Sets, Tests
Affects Version/s: 10.5, 10.6, 10.7, 10.8
Fix Version/s: 10.8.0, 10.5.14, 10.6.6, 10.7.2

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Alexander Barkov
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-21263 Allow packed values of non-sorted fie... Closed
relates to MDEV-27154 allkeys.txt based tests for Unicode-4... Closed

 Description   

10.8 c9fcea14

 
main.ctype_utf8mb4_uca_allkeys400        [ fail ]  Found warnings/errors in server log file!
 
        Test ended at 2021-12-19 01:54:31
 
line
 
==3545764== Thread 6:
 
==3545764== Uninitialised byte(s) found during client check request
 
==3545764==    at 0xDA49B1: my_b_write(st_io_cache*, unsigned char const*, unsigned long) (my_sys.h:526)
 
==3545764==    by 0xDA743F: write_keys(Sort_param*, SORT_INFO*, unsigned int, st_io_cache*, st_io_cache*) (filesort.cc:1085)
 
==3545764==    by 0xDA09DD: find_all_keys(THD*, Sort_param*, SQL_SELECT*, SORT_INFO*, st_io_cache*, st_io_cache*, Bounded_queue<unsigned char, unsigned char>*, unsigned long long*) (filesort.cc:969)
 
==3545764==    by 0xD9DA67: filesort(THD*, TABLE*, Filesort*, Filesort_tracker*, JOIN*, unsigned long long) (filesort.cc:357)
 
==3545764==    by 0xA385B4: create_sort_index(THD*, JOIN*, st_join_table*, Filesort*) (sql_select.cc:24408)
 
==3545764==    by 0xA3812D: st_join_table::sort_table() (sql_select.cc:22082)
 
==3545764==    by 0xA10E75: join_init_read_record(st_join_table*) (sql_select.cc:22021)
 
==3545764==    by 0x9E8D33: sub_select(JOIN*, st_join_table*, bool) (sql_select.cc:21067)
 
==3545764==    by 0xA15554: do_select(JOIN*, Procedure*) (sql_select.cc:20617)
 
==3545764==    by 0xA14201: JOIN::exec_inner() (sql_select.cc:4735)
 
==3545764==    by 0xA12F14: JOIN::exec() (sql_select.cc:4513)
 
==3545764==    by 0x9E9AC2: mysql_select(THD*, TABLE_LIST*, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:4993)
 
==3545764==    by 0x9E926A: handle_select(THD*, LEX*, select_result*, unsigned long) (sql_select.cc:545)
 
==3545764==    by 0x9807A2: execute_sqlcom_select(THD*, TABLE_LIST*) (sql_parse.cc:6253)
 
==3545764==    by 0x973A01: mysql_execute_command(THD*, bool) (sql_parse.cc:3944)
 
==3545764==    by 0x96A2DF: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:8028)
 
==3545764==  Address 0xbc71647 is 55 bytes inside a block of size 262,152 alloc'd
 
==3545764==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
 
==3545764==    by 0x17D1B0A: my_malloc (my_malloc.c:90)
 
==3545764==    by 0xD9BC3E: Filesort_buffer::alloc_sort_buffer(unsigned int, unsigned int) (filesort_utils.cc:136)
 
==3545764==    by 0xDA8000: SORT_INFO::alloc_sort_buffer(unsigned int, unsigned int) (filesort.h:174)
 
==3545764==    by 0xD9D750: filesort(THD*, TABLE*, Filesort*, Filesort_tracker*, JOIN*, unsigned long long) (filesort.cc:323)
 
==3545764==    by 0xA385B4: create_sort_index(THD*, JOIN*, st_join_table*, Filesort*) (sql_select.cc:24408)
 
==3545764==    by 0xA3812D: st_join_table::sort_table() (sql_select.cc:22082)
 
==3545764==    by 0xA10E75: join_init_read_record(st_join_table*) (sql_select.cc:22021)
 
==3545764==    by 0x9E8D33: sub_select(JOIN*, st_join_table*, bool) (sql_select.cc:21067)
 
==3545764==    by 0xA15554: do_select(JOIN*, Procedure*) (sql_select.cc:20617)
 
==3545764==    by 0xA14201: JOIN::exec_inner() (sql_select.cc:4735)
 
==3545764==    by 0xA12F14: JOIN::exec() (sql_select.cc:4513)
 
==3545764==    by 0x9E9AC2: mysql_select(THD*, TABLE_LIST*, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:4993)
 
==3545764==    by 0x9E926A: handle_select(THD*, LEX*, select_result*, unsigned long) (sql_select.cc:545)
 
==3545764==    by 0x9807A2: execute_sqlcom_select(THD*, TABLE_LIST*) (sql_parse.cc:6253)
 
==3545764==    by 0x973A01: mysql_execute_command(THD*, bool) (sql_parse.cc:3944)
 
==3545764== Syscall param write(buf) points to uninitialised byte(s)
 
==3545764==    at 0x4F212CF: __libc_write (write.c:26)
 
==3545764==    by 0x4F212CF: write (write.c:24)
 
==3545764==    by 0x17D7A67: my_write (my_write.c:49)
 
==3545764==    by 0x17A6987: inline_mysql_file_write (mysql_file.h:1176)
 
==3545764==    by 0x17A7EC1: _my_b_cache_write (mf_iocache.c:1526)
 
==3545764==    by 0x17A4CAC: my_b_flush_io_cache (mf_iocache.c:1727)
 
==3545764==    by 0x17A515A: _my_b_write (mf_iocache.c:559)
 
==3545764==    by 0xDA4A23: my_b_write(st_io_cache*, unsigned char const*, unsigned long) (my_sys.h:536)
 
==3545764==    by 0xDA743F: write_keys(Sort_param*, SORT_INFO*, unsigned int, st_io_cache*, st_io_cache*) (filesort.cc:1085)
 
==3545764==    by 0xDA09DD: find_all_keys(THD*, Sort_param*, SQL_SELECT*, SORT_INFO*, st_io_cache*, st_io_cache*, Bounded_queue<unsigned char, unsigned char>*, unsigned long long*) (filesort.cc:969)
 
==3545764==    by 0xD9DA67: filesort(THD*, TABLE*, Filesort*, Filesort_tracker*, JOIN*, unsigned long long) (filesort.cc:357)
 
==3545764==    by 0xA385B4: create_sort_index(THD*, JOIN*, st_join_table*, Filesort*) (sql_select.cc:24408)
 
==3545764==    by 0xA3812D: st_join_table::sort_table() (sql_select.cc:22082)
 
==3545764==    by 0xA10E75: join_init_read_record(st_join_table*) (sql_select.cc:22021)
 
==3545764==    by 0x9E8D33: sub_select(JOIN*, st_join_table*, bool) (sql_select.cc:21067)
 
==3545764==    by 0xA15554: do_select(JOIN*, Procedure*) (sql_select.cc:20617)
 
==3545764==    by 0xA14201: JOIN::exec_inner() (sql_select.cc:4735)
 
==3545764==  Address 0xbcb1697 is 55 bytes inside a block of size 65,560 alloc'd
 
==3545764==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
 
==3545764==    by 0x17D1B0A: my_malloc (my_malloc.c:90)
 
==3545764==    by 0x17A3516: init_io_cache_ext (mf_iocache.c:248)
 
==3545764==    by 0x17A3F1B: init_io_cache (mf_iocache.c:301)
 
==3545764==    by 0x17A1E22: open_cached_file (mf_cache.c:45)
 
==3545764==    by 0xDA72AD: write_keys(Sort_param*, SORT_INFO*, unsigned int, st_io_cache*, st_io_cache*) (filesort.cc:1068)
 
==3545764==    by 0xDA09DD: find_all_keys(THD*, Sort_param*, SQL_SELECT*, SORT_INFO*, st_io_cache*, st_io_cache*, Bounded_queue<unsigned char, unsigned char>*, unsigned long long*) (filesort.cc:969)
 
==3545764==    by 0xD9DA67: filesort(THD*, TABLE*, Filesort*, Filesort_tracker*, JOIN*, unsigned long long) (filesort.cc:357)
 
==3545764==    by 0xA385B4: create_sort_index(THD*, JOIN*, st_join_table*, Filesort*) (sql_select.cc:24408)
 
==3545764==    by 0xA3812D: st_join_table::sort_table() (sql_select.cc:22082)
 
==3545764==    by 0xA10E75: join_init_read_record(st_join_table*) (sql_select.cc:22021)
 
==3545764==    by 0x9E8D33: sub_select(JOIN*, st_join_table*, bool) (sql_select.cc:21067)
 
==3545764==    by 0xA15554: do_select(JOIN*, Procedure*) (sql_select.cc:20617)
 
==3545764==    by 0xA14201: JOIN::exec_inner() (sql_select.cc:4735)
 
==3545764==    by 0xA12F14: JOIN::exec() (sql_select.cc:4513)
 
==3545764==    by 0x9E9AC2: mysql_select(THD*, TABLE_LIST*, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:4993)
 
==3545764== Uninitialised byte(s) found during client check request
 
==3545764==    at 0xDA49B1: my_b_write(st_io_cache*, unsigned char const*, unsigned long) (my_sys.h:526)
 
==3545764==    by 0xDA743F: write_keys(Sort_param*, SORT_INFO*, unsigned int, st_io_cache*, st_io_cache*) (filesort.cc:1085)
 
==3545764==    by 0xDA0CDB: find_all_keys(THD*, Sort_param*, SQL_SELECT*, SORT_INFO*, st_io_cache*, st_io_cache*, Bounded_queue<unsigned char, unsigned char>*, unsigned long long*) (filesort.cc:1019)
 
==3545764==    by 0xD9DA67: filesort(THD*, TABLE*, Filesort*, Filesort_tracker*, JOIN*, unsigned long long) (filesort.cc:357)
 
==3545764==    by 0xA385B4: create_sort_index(THD*, JOIN*, st_join_table*, Filesort*) (sql_select.cc:24408)
 
==3545764==    by 0xA3812D: st_join_table::sort_table() (sql_select.cc:22082)
 
==3545764==    by 0xA10E75: join_init_read_record(st_join_table*) (sql_select.cc:22021)
 
==3545764==    by 0x9E8D33: sub_select(JOIN*, st_join_table*, bool) (sql_select.cc:21067)
 
==3545764==    by 0xA15554: do_select(JOIN*, Procedure*) (sql_select.cc:20617)
 
==3545764==    by 0xA14201: JOIN::exec_inner() (sql_select.cc:4735)
 
==3545764==    by 0xA12F14: JOIN::exec() (sql_select.cc:4513)
 
==3545764==    by 0x9E9AC2: mysql_select(THD*, TABLE_LIST*, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:4993)
 
==3545764==    by 0x9E926A: handle_select(THD*, LEX*, select_result*, unsigned long) (sql_select.cc:545)
 
==3545764==    by 0x9807A2: execute_sqlcom_select(THD*, TABLE_LIST*) (sql_parse.cc:6253)
 
==3545764==    by 0x973A01: mysql_execute_command(THD*, bool) (sql_parse.cc:3944)
 
==3545764==    by 0x96A2DF: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:8028)
 
==3545764==  Address 0xbc71810 is 512 bytes inside a block of size 262,152 alloc'd
 
==3545764==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
 
==3545764==    by 0x17D1B0A: my_malloc (my_malloc.c:90)
 
==3545764==    by 0xD9BC3E: Filesort_buffer::alloc_sort_buffer(unsigned int, unsigned int) (filesort_utils.cc:136)
 
==3545764==    by 0xDA8000: SORT_INFO::alloc_sort_buffer(unsigned int, unsigned int) (filesort.h:174)
 
==3545764==    by 0xD9D750: filesort(THD*, TABLE*, Filesort*, Filesort_tracker*, JOIN*, unsigned long long) (filesort.cc:323)
 
==3545764==    by 0xA385B4: create_sort_index(THD*, JOIN*, st_join_table*, Filesort*) (sql_select.cc:24408)
 
==3545764==    by 0xA3812D: st_join_table::sort_table() (sql_select.cc:22082)
 
==3545764==    by 0xA10E75: join_init_read_record(st_join_table*) (sql_select.cc:22021)
 
==3545764==    by 0x9E8D33: sub_select(JOIN*, st_join_table*, bool) (sql_select.cc:21067)
 
==3545764==    by 0xA15554: do_select(JOIN*, Procedure*) (sql_select.cc:20617)
 
==3545764==    by 0xA14201: JOIN::exec_inner() (sql_select.cc:4735)
 
==3545764==    by 0xA12F14: JOIN::exec() (sql_select.cc:4513)
 
==3545764==    by 0x9E9AC2: mysql_select(THD*, TABLE_LIST*, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:4993)
 
==3545764==    by 0x9E926A: handle_select(THD*, LEX*, select_result*, unsigned long) (sql_select.cc:545)
 
==3545764==    by 0x9807A2: execute_sqlcom_select(THD*, TABLE_LIST*) (sql_parse.cc:6253)
 
==3545764==    by 0x973A01: mysql_execute_command(THD*, bool) (sql_parse.cc:3944)

10.8-based development branch

 
main.ctype_utf8mb4_uca_allkeys520        w2 [ fail ]
 
...
 
Uninitialized bytes in __msan_check_mem_is_initialized at offset 31 inside [0x7f01932a0018, 35)
 
==568806==WARNING: MemorySanitizer: use-of-uninitialized-value
 
    #0 0x5599af618258 in my_b_write(st_io_cache*, unsigned char const*, unsigned long) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/include/my_sys.h:526:3
 
    #1 0x5599af618258 in write_keys(Sort_param*, SORT_INFO*, unsigned int, st_io_cache*, st_io_cache*) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/filesort.cc:1085:9
 
    #2 0x5599af5f8043 in find_all_keys(THD*, Sort_param*, SQL_SELECT*, SORT_INFO*, st_io_cache*, st_io_cache*, Bounded_queue<unsigned char, unsigned char>*, unsigned long long*) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/filesort.cc:969:15
 
    #3 0x5599af5f8043 in filesort(THD*, TABLE*, Filesort*, Filesort_tracker*, JOIN*, unsigned long long) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/filesort.cc:357:13
 
    #4 0x5599aeb7dce5 in create_sort_index(THD*, JOIN*, st_join_table*, Filesort*) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_select.cc:24416:14
 
    #5 0x5599aeb7cad2 in st_join_table::sort_table() /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_select.cc:22082:7
 
    #6 0x5599aeae3391 in join_init_read_record(st_join_table*) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_select.cc:22021:29
 
    #7 0x5599aea4f1b0 in sub_select(JOIN*, st_join_table*, bool) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_select.cc:21067:12
 
    #8 0x5599aeaee19e in do_select(JOIN*, Procedure*) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_select.cc:20617:14
 
    #9 0x5599aeaee19e in JOIN::exec_inner() /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_select.cc:4735:50
 
    #10 0x5599aeae9dec in JOIN::exec() /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_select.cc:4513:3
 
    #11 0x5599aea520b2 in mysql_select(THD*, TABLE_LIST*, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_select.cc:4993:9
 
    #12 0x5599aea50df8 in handle_select(THD*, LEX*, select_result*, unsigned long) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_select.cc:545:10
 
    #13 0x5599ae927521 in execute_sqlcom_select(THD*, TABLE_LIST*) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_parse.cc:6253:12
 
    #14 0x5599ae9012df in mysql_execute_command(THD*, bool) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_parse.cc:3944:12
 
    #15 0x5599ae9fa2c5 in Prepared_statement::execute(String*, bool) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_prepare.cc:5210:14
 
    #16 0x5599ae9e4e6d in Prepared_statement::execute_loop(String*, bool, unsigned char*, unsigned char*) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_prepare.cc:4633:10
 
    #17 0x5599ae9e2d69 in mysql_stmt_execute_common(THD*, unsigned long, unsigned char*, unsigned char*, unsigned long, bool, bool) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_prepare.cc:3572:11
 
    #18 0x5599ae9e1fae in mysqld_stmt_execute(THD*, char*, unsigned int) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_prepare.cc:3343:3
 
    #19 0x5599ae8dcd66 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_parse.cc:1819:5
 
    #20 0x5599ae8eb62c in do_command(THD*, bool) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_parse.cc:1402:17
 
    #21 0x5599aef5b136 in do_handle_one_connection(CONNECT*, bool) /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_connect.cc:1418:11
 
    #22 0x5599aef5a685 in handle_one_connection /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/sql_connect.cc:1312:5
 
    #23 0x5599b02b6f21 in pfs_spawn_thread /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/storage/perfschema/pfs.cc:2201:3
 
    #24 0x7f019bb5d608 in start_thread /build/glibc-eX1tMB/glibc-2.31/nptl/pthread_create.c:477:8
 
    #25 0x7f019b862292 in clone /build/glibc-eX1tMB/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95
 
 
 
  Memory was marked as uninitialized
 
    #0 0x5599ae2d3a6e in __msan_allocated_memory (/home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/sql/mariadbd+0x728a6e)
 
    #1 0x5599b1661950 in my_malloc /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/mysys/my_malloc.c:113:7
 
 
 
SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/jenkins/workspace/sandbox-elenst/MSAN-ASAN/src/include/my_sys.h:526:3 in my_b_write(st_io_cache*, unsigned char const*, unsigned long)



 Comments   
Comment by Alexander Barkov [ 2021-12-20 ]

It's also repeatable starting from 10.5 with mtr --valgrind run with this smaller script:

--source include/have_utf32.inc
 
--source include/have_utf8mb4.inc
 
 
 
SET NAMES latin1;
 
 
 
CREATE TABLE t1 (
 
  code INT NOT NULL,
 
  str VARCHAR(1) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL
 
) ENGINE=MyISAM;
 
 
 
DELIMITER $$;
 
FOR i IN 0x0000..0x2FFF
 
DO
 
  INSERT INTO t1 VALUES (i, CHAR(i USING utf32));
 
END FOR;
 
$$
 
DELIMITER ;$$
 
SELECT COUNT(*) FROM t1;
 
 
 
SELECT HEX(code), HEX(str) FROM t1 ORDER BY HEX(str);
 
 
 
DROP TABLE t1;

and this script:

SET NAMES latin1;
 
 
 
CREATE TABLE t1 (
 
  code INT NOT NULL,
 
  str VARCHAR(5) CHARACTER SET latin1 NOT NULL
 
) ENGINE=MyISAM;
 
 
 
DELIMITER $$;
 
FOR i IN 0..50
 
DO
 
  INSERT INTO t1 VALUES (i, REPEAT('a',1));
 
END FOR;
 
$$
 
DELIMITER ;$$
 
SELECT COUNT(*) FROM t1;
 
 
 
SET sort_buffer_size=1024;
 
--disable_result_log
 
SELECT HEX(code), HEX(str) FROM t1 ORDER BY HEX(str);
 
--enable_result_log
 
SET sort_buffer_size=DEFAULT;
 
 
 
DROP TABLE t1;

Comment by Alexander Barkov [ 2021-12-21 ]

The problem was introduced by:

commit f52bf92014efae6a1da9c2f26a7e3792ed5f5396
 
Author: Varun Gupta <varun.gupta@mariadb.com>
 
Date:   Tue Jan 21 01:37:47 2020 +0530
 
 
 
    MDEV-21263: Allow packed values of non-sorted fields in the sort buffer

Generated at Thu Feb 08 09:51:55 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.