[MDEV-22689] MSAN use-of-uninitialized-value in decode_bytes() Created: 2020-05-25  Updated: 2020-06-15  Resolved: 2020-06-15

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - MyISAM
Affects Version/s: 10.5
Fix Version/s: 10.5.4

Type: Bug Priority: Major
Reporter: Marko Mäkelä Assignee: Michael Widenius
Resolution: Fixed Votes: 0
Labels: MSAN, MyISAM

Issue Links:
Blocks
blocks MDEV-20377 Make WITH_MSAN more usable Closed

 Description   

The test main.myisampack is failing as follows:

10.5 d8ea11a33fba12331c98c04ff44c815a662faccb

main.myisampack                          w1 [ fail ]
        Test ended at 2020-05-25 11:16:20
 
CURRENT_TEST: main.myisampack
==401784==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x55df34ad8500 in decode_bytes /mariadb/10.5m/storage/myisam/mi_packrec.c:1236:14
    #1 0x55df34ad783e in uf_zerofill_normal /mariadb/10.5m/storage/myisam/mi_packrec.c:1011:3
    #2 0x55df34acdae6 in _mi_pack_rec_unpack /mariadb/10.5m/storage/myisam/mi_packrec.c:754:5
    #3 0x55df34a23e33 in sort_get_next_record /mariadb/10.5m/storage/myisam/mi_check.c:3629:11
    #4 0x55df34a3aee2 in sort_key_read /mariadb/10.5m/storage/myisam/mi_check.c:3124:14
    #5 0x55df34b40252 in find_all_keys /mariadb/10.5m/storage/myisam/sort.c:311:18
    #6 0x55df34b40252 in _create_index_by_sort /mariadb/10.5m/storage/myisam/sort.c:227:17
    #7 0x55df34a35d48 in mi_repair_by_sort /mariadb/10.5m/storage/myisam/mi_check.c:2403:9
    #8 0x55df349fb9b9 in myisamchk /mariadb/10.5m/storage/myisam/myisamchk.c:1024:19
    #9 0x55df349f400d in main /mariadb/10.5m/storage/myisam/myisamchk.c:87:19
    #10 0x7f6381967e0a in __libc_start_main csu/../csu/libc-start.c:308:16
    #11 0x55df34978649 in _start (/dev/shm/10.5-msan/storage/myisam/myisamchk+0x49649)
 
  Memory was marked as uninitialized
    #0 0x55df3498493e in __msan_allocated_memory (/dev/shm/10.5-msan/storage/myisam/myisamchk+0x5593e)
    #1 0x55df34c4bcc7 in my_malloc /mariadb/10.5m/mysys/my_malloc.c:111:7
 
SUMMARY: MemorySanitizer: use-of-uninitialized-value /mariadb/10.5m/storage/myisam/mi_packrec.c:1236:14 in decode_bytes
Exiting
mysqltest: At line 34: exec of '/dev/shm/10.5-msan/storage/myisam/myisamchk -srq /dev/shm/10.5-msan/mysql-test/var/1/mysqld.1/data//test/t1' failed, error: 19712, status: 77, errno: 11
Output from before failure:
FLUSH TABLES;

The code in decode_bytes() is as follows:

	/* First use info in quick_table */
    low_byte=(bit_buff->current_byte >> (bits - table_bits)) & table_and;
    low_byte=decode_tree->table[low_byte];
    if (low_byte & IS_CHAR)

The failing statement is right before the if statement.

Note: Please adjust the affectedVersion and fixVersion. I only tested 10.5.



 Comments   
Comment by Michael Widenius [ 2020-06-02 ]

Tested with valgrind and no issue
. As I can't run MSAN and I don't see any obvious issues with the code, there is not much that I can do here

I also find a bit strange that there could be any issues as there are checksums for rows and data
and if there would be any wrong transformations the checksums should notice that.

hm..
If can you find out where in myisampack.test things fail (in other words which myisampack command was executed) then I could just run that myisampack command under valgrind and check what could be wrong.

Comment by Marko Mäkelä [ 2020-06-02 ]

I tried the following patch:

diff --git a/mysql-test/main/myisampack.test b/mysql-test/main/myisampack.test
index 1f97a28e6fd..ecd4c371b21 100644
--- a/mysql-test/main/myisampack.test
+++ b/mysql-test/main/myisampack.test
@@ -31,7 +31,9 @@ INSERT INTO t1 VALUES
 FLUSH TABLES;
 let $MYSQLD_DATADIR= `select @@datadir`;
 --exec $MYISAMPACK -s $MYSQLD_DATADIR/test/t1
---exec $MYISAMCHK -srq $MYSQLD_DATADIR/test/t1
+--echo exec $MYISAMCHK -srq $MYSQLD_DATADIR/test/t1
+sleep 3600;
+
 --exec $MYISAMCHK -s --unpack $MYSQLD_DATADIR/test/t1
 CHECK TABLE t1 EXTENDED;
 DROP TABLE t1;
diff --git a/storage/myisam/mi_packrec.c b/storage/myisam/mi_packrec.c
index d5bfdac3008..43481e03f43 100644
--- a/storage/myisam/mi_packrec.c
+++ b/storage/myisam/mi_packrec.c
@@ -1188,7 +1188,7 @@ static void decode_bytes(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,uchar *to,
 static void decode_bytes(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff, uchar *to,
 			 uchar *end)
 {
-  reg1 uint bits,low_byte;
+  uint bits,low_byte;
   reg3 uint16 *pos;
   reg4 uint table_bits,table_and;
   MI_DECODE_TREE *decode_tree;
@@ -1233,7 +1233,9 @@ static void decode_bytes(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff, uchar *to,
     }
 	/* First use info in quick_table */
     low_byte=(bit_buff->current_byte >> (bits - table_bits)) & table_and;
+    __msan_check_mem_is_initialized(&low_byte, sizeof low_byte);
     low_byte=decode_tree->table[low_byte];
+    __msan_check_mem_is_initialized(&low_byte, sizeof low_byte);
     if (low_byte & IS_CHAR)
     {
       *to++ = (low_byte & 255);		/* Found char in quick table */

You may want to use VALGRIND_CHECK_MEM_IS_DEFINED() instead.
The first invocation will report that low_byte is uninitialized. gdb seems to confirm that even without a need to invoke __msan_dump_shadow():

10.5 6df2f2db11bf0a559db54e18315ad37e6a4d5573

Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50	../sysdeps/unix/sysv/linux/raise.c: Tiedostoa tai hakemistoa ei ole.
(gdb) up 5
#5  0x00005555556fcd58 in decode_bytes (rec=<optimized out>, 
    bit_buff=0x7fffffffd820, to=0x7070000000d2 '\245' <repeats 14 times>, 
    end=0x7070000000d3 '\245' <repeats 13 times>)
    at /mariadb/10.5m/storage/myisam/mi_packrec.c:1236
1236	    __msan_check_mem_is_initialized(&low_byte, sizeof low_byte);
(gdb) p *bit_buff
$1 = {current_byte = 3785729445, bits = 8, 
  pos = 0x707000000149 "\245\245\245\245\245\245\245", 
  end = 0x707000000146 "\245\245\245\245\245\245\245\245\245\245", 
  blob_pos = 0x0, blob_end = 0x0, error = 0}

Comment by Michael Widenius [ 2020-06-08 ]

Fixed in my 10.5 tree, will be pushed shortly together with other changes

Comment by Michael Widenius [ 2020-06-15 ]

Fix pushed into 10.5.4

This was not user visible issue as the huffman code lookup tables would automatically ignore any of the uninitialised bits
Fixed by adding a end-zero byte to the bit-stream buffer.

Generated at Thu Feb 08 09:16:42 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.