[MDEV-32667] dict_stats_save_index_stat() reads uninitialized index->stats_error_printed Created: 2023-11-03  Updated: 2023-11-03

Status: Confirmed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.4, 10.5
Fix Version/s: 10.4, 10.5

Type: Bug Priority: Major
Reporter: Ramesh Sivaraman Assignee: Thirunarayanan Balathandayuthapani
Resolution: Unresolved Votes: 0
Labels: MSAN


 Description    

CREATE TABLE t (a INT) ENGINE=InnoDB;
 
INSERT INTO t (a) VALUES (0);
 
XA BEGIN 'a';
 
SELECT * FROM mysql.innodb_index_stats LOCK IN SHARE MODE;
 
INSERT INTO t (a) VALUES (0);

Leads to:

10.5.23 b06ac9a8cd2146e89270cc2150d306d8ed1b33fb (Optimized, UBASAN)

 
/test/10.5_opt_san/storage/innobase/dict/dict0stats.cc:2407:14: runtime error: load of value 190, which is not a valid value for type 'bool'
 
    #0 0x561fbcf92411 in dict_stats_save_index_stat(dict_index_t*, long, char const*, unsigned long, unsigned long*, char const*, trx_t*) /test/10.5_opt_san/storage/innobase/dict/dict0stats.cc:2407
 
    #1 0x561fc104d73c in dict_stats_save /test/10.5_opt_san/storage/innobase/dict/dict0stats.cc:2595
 
    #2 0x561fc1056056 in dict_stats_update(dict_table_t*, dict_stats_upd_option_t) /test/10.5_opt_san/storage/innobase/dict/dict0stats.cc:3235
 
    #3 0x561fc1065d14 in dict_stats_process_entry_from_recalc_pool /test/10.5_opt_san/storage/innobase/dict/dict0stats_bg.cc:399
 
    #4 0x561fc1065d14 in dict_stats_func /test/10.5_opt_san/storage/innobase/dict/dict0stats_bg.cc:418
 
    #5 0x561fc13e7e1c in tpool::thread_pool_generic::timer_generic::run() /test/10.5_opt_san/tpool/tpool_generic.cc:361
 
    #6 0x561fc13e7e1c in tpool::thread_pool_generic::timer_generic::execute(void*) /test/10.5_opt_san/tpool/tpool_generic.cc:382
 
    #7 0x561fc13f0295 in tpool::task::execute() /test/10.5_opt_san/tpool/task.cc:52
 
    #8 0x561fc13db98b in tpool::thread_pool_generic::worker_main(tpool::worker_data*) /test/10.5_opt_san/tpool/tpool_generic.cc:599
 
    #9 0x561fc13e701a in void std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>(std::__invoke_memfun_deref, void (tpool::thread_pool_generic::*&&)(tpool::worker_data*), tpool::thread_pool_generic*&&, tpool::worker_data*&&) /usr/include/c++/9/bits/invoke.h:73
 
    #10 0x561fc13e701a in std::__invoke_result<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>::type std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>(void (tpool::thread_pool_generic::*&&)(tpool::worker_data*), tpool::thread_pool_generic*&&, tpool::worker_data*&&) /usr/include/c++/9/bits/invoke.h:95
 
    #11 0x561fc13e701a in void std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/9/thread:244
 
    #12 0x561fc13e701a in std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator()() /usr/include/c++/9/thread:251
 
    #13 0x561fc13e701a in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run() /usr/include/c++/9/thread:195
 
    #14 0x148c08187de3  (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xd6de3)
 
    #15 0x148c082a1608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477
 
    #16 0x148c07516132 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f132)
 
 
 
2023-11-03  5:16:20 0 [ERROR] InnoDB: Cannot save index statistics for table `test`.`t`, index `GEN_CLUST_INDEX`, stat name "n_diff_pfx01": Lock wait timeout

Setup:

Compiled with GCC >=7.5.0 (I use GCC 11.4.0) and:
 
    -DWITH_ASAN=ON -DWITH_ASAN_SCOPE=ON -DWITH_UBSAN=ON -DWSREP_LIB_WITH_ASAN=ON
 
Set before execution:
 
    export UBSAN_OPTIONS=print_stacktrace=1

Bug confirmed present in:
MariaDB: 10.4.32 (dbg), 10.4.32 (opt), 10.5.23 (dbg), 10.5.23 (opt)

Bug (or feature/syntax) confirmed not present in:
MariaDB: 10.6.16 (dbg), 10.6.16 (opt), 10.9.8 (dbg), 10.9.8 (opt), 10.10.7 (dbg), 10.10.7 (opt), 10.11.6 (dbg), 10.11.6 (opt), 11.0.4 (dbg), 11.0.4 (opt), 11.1.3 (dbg), 11.1.3 (opt), 11.2.2 (dbg), 11.2.2 (opt), 11.3.0 (dbg), 11.3.0 (opt)

 Comments   
Comment by Marko Mäkelä [ 2023-11-03 ]

Here is a simpler test:

--source include/have_innodb.inc
 
CREATE TABLE t (a INT) ENGINE=InnoDB STATS_PERSISTENT=1 STATS_AUTO_RECALC=1;
 
BEGIN;
 
SELECT COUNT(*)>=0 FROM mysql.innodb_index_stats LOCK IN SHARE MODE;
 
INSERT INTO t VALUES(0),(0);
 
SELECT sleep(5);
 
COMMIT;
 
DROP TABLE t;

I can reproduce the error with MemorySanitizer:

10.5 b06ac9a8cd2146e89270cc2150d306d8ed1b33fb

 
==40299==WARNING: MemorySanitizer: use-of-uninitialized-value
 
    #0 0x55c9668d8770 in dict_stats_save_index_stat(dict_index_t*, long, char const*, unsigned long, unsigned long*, char const*, trx_t*) /mariadb/10.5/storage/innobase/dict/dict0stats.cc:2406:7
 
    #1 0x55c9668ec8fa in dict_stats_save(dict_table_t*, unsigned long const*) /mariadb/10.5/storage/innobase/dict/dict0stats.cc:2595:10
 
    #2 0x55c9668f376f in dict_stats_update(dict_table_t*, dict_stats_upd_option_t) /mariadb/10.5/storage/innobase/dict/dict0stats.cc
 
    #3 0x55c9668d6cd8 in dict_stats_process_entry_from_recalc_pool() /mariadb/10.5/storage/innobase/dict/dict0stats_bg.cc:399:3
 
    #4 0x55c9668d6cd8 in dict_stats_func(void*) /mariadb/10.5/storage/innobase/dict/dict0stats_bg.cc:418:9

If I lock mysql.innodb_table_stats instead of mysql.innodb_index_stats, an error will be reported:

10.5 b06ac9a8cd2146e89270cc2150d306d8ed1b33fb

 
2023-11-03  9:17:10 0 [ERROR] InnoDB: Cannot save table statistics for table `test`.`t`: Lock wait timeout

Comment by Marko Mäkelä [ 2023-11-03 ]

The problem is that dict_stats_table_clone_create() does not initialize the flag stats_error_printed in either dict_table_t or dict_index_t. Because dict_stats_save_index_stat() is operating on a copy of a dict_index_t object, it appears that dict_index_t::stats_error_printed will always be false for actual metadata objects, and uninitialized in dict_stats_save_index_stat().

We must not bloat dict_table_t and dict_index_t with the stats_error_printed fields. Instead, we should store these flags in some stack-based data structure in dict_stats_update().

I do not know why the metadata is being copied at all. Starting with MDEV-16678 in 10.5 there should be no good reason for that. In 10.4 we may need the copy. Yes, we could protect the metadata by holding a shared dict_sys.latch, but there is no way to upgrade it to exclusive, for the dict_stats_save() call. (Maybe we could acquire the latch in SX mode? Saving the statistics is already hang-prone until 10.6; see MDEV-15020.)

Generated at Thu Feb 08 10:33:05 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.