[MDEV-24329] TSAN: Data race in srv_mon_default_on at srv/srv0mon.cc:2099 Created: 2020-12-02  Updated: 2023-04-27

Status: Open
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.3, 10.4, 10.5, 10.6
Fix Version/s: 10.4, 10.5, 10.6

Type: Bug Priority: Major
Reporter: Roel Van de Paar Assignee: Marko Mäkelä
Resolution: Unresolved Votes: 0
Labels: affects-tests, not-10.1, not-10.2, not-critical, tsan

Issue Links:
Blocks
is blocked by MDEV-21212 buf_page_get_gen -> buf_pool->stat.n_... Closed
Relates
relates to MDEV-24332 TSAN: Data race in os_file_pread from... Open

 Description   

Simply starting the server (both optimized and debug) generates this data race, once on startup and once at shutdown:

10.5.9 1435f35bdabd078f0c4e744ab4bdfd8d4acca3ea (Optimized)

... startup ...
SUMMARY: ThreadSanitizer: data race /test/10.5_opt_san/storage/innobase/srv/srv0mon.cc:2099 in srv_mon_default_on()
... mysqladmin shutdown ...
SUMMARY: ThreadSanitizer: data race /test/10.5_opt_san/storage/innobase/srv/srv0mon.cc:2099 in srv_mon_default_on()

10.5.9 1435f35bdabd078f0c4e744ab4bdfd8d4acca3ea (Optimized)

WARNING: ThreadSanitizer: data race (pid=2539623)
  Write of size 8 at 0x558b8f1a7e78 by main thread:
    #0 srv_mon_default_on() /test/10.5_opt_san/storage/innobase/srv/srv0mon.cc:2099 (mariadbd+0x14decc1)
    #1 innodb_init /test/10.5_opt_san/storage/innobase/handler/ha_innodb.cc:3961 (mariadbd+0x136065c)
    #2 ha_initialize_handlerton(st_plugin_int*) /test/10.5_opt_san/sql/handler.cc:645 (mariadbd+0xcf9547)
    #3 plugin_initialize /test/10.5_opt_san/sql/sql_plugin.cc:1459 (mariadbd+0x8d3f74)
    #4 plugin_init(int*, char**, int) /test/10.5_opt_san/sql/sql_plugin.cc:1751 (mariadbd+0x8d5cc3)
    #5 init_server_components /test/10.5_opt_san/sql/mysqld.cc:4913 (mariadbd+0x72043d)
    #6 mysqld_main(int, char**) /test/10.5_opt_san/sql/mysqld.cc:5496 (mariadbd+0x725bf4)
    #7 main /test/10.5_opt_san/sql/main.cc:25 (mariadbd+0x6b8455)
 
  Previous read of size 8 at 0x558b8f1a7e78 by thread T7:
    #0 srv_master_do_idle_tasks /test/10.5_opt_san/storage/innobase/srv/srv0srv.cc:1791 (mariadbd+0x14e179b)
    #1 srv_master_callback /test/10.5_opt_san/storage/innobase/srv/srv0srv.cc:1853 (mariadbd+0x14e179b)
    #2 tpool::thread_pool_generic::timer_generic::run() /test/10.5_opt_san/tpool/tpool_generic.cc:309 (mariadbd+0x16f2c8e)
    #3 tpool::thread_pool_generic::timer_generic::execute(void*) /test/10.5_opt_san/tpool/tpool_generic.cc:329 (mariadbd+0x16f2c8e)
    #4 tpool::task::execute() /test/10.5_opt_san/tpool/task.cc:52 (mariadbd+0x16f48df)
    #5 tpool::thread_pool_generic::worker_main(tpool::worker_data*) /test/10.5_opt_san/tpool/tpool_generic.cc:546 (mariadbd+0x16f1b54)
    #6 void std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>(std::__invoke_memfun_deref, void (tpool::thread_pool_generic::*&&)(tpool::worker_data*), tpool::thread_pool_generic*&&, tpool::worker_data*&&) /usr/include/c++/9/bits/invoke.h:73 (mariadbd+0x16f2a47)
    #7 std::__invoke_result<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>::type std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>(void (tpool::thread_pool_generic::*&&)(tpool::worker_data*), tpool::thread_pool_generic*&&, tpool::worker_data*&&) /usr/include/c++/9/bits/invoke.h:95 (mariadbd+0x16f2a47)
    #8 void std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/9/thread:244 (mariadbd+0x16f2a47)
    #9 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator()() /usr/include/c++/9/thread:251 (mariadbd+0x16f2a47)
    #10 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run() /usr/include/c++/9/thread:195 (mariadbd+0x16f2a47)
    #11 <null> <null> (libstdc++.so.6+0xd6d83)
 
  Location is global 'monitor_set_tbl' of size 32 at 0x558b8f1a7e60 (mariadbd+0x000002b2ae78)
 
  Thread T7 (tid=2539637, running) created by thread T1 at:
    #0 pthread_create <null> (libtsan.so.0+0x5ea99)
    #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd7048)
    #2 tpool::thread_pool_generic::maybe_wake_or_create_thread() /test/10.5_opt_san/tpool/tpool_generic.cc:802 (mariadbd+0x16f2787)
    #3 tpool::thread_pool_generic::submit_task(tpool::task*) /test/10.5_opt_san/tpool/tpool_generic.cc:821 (mariadbd+0x16f2787)
    #4 tpool::thread_pool_generic::timer_generic::submit_task(void*) /test/10.5_opt_san/tpool/tpool_generic.cc:335 (mariadbd+0x16f28c9)
    #5 process_timers /test/10.5_opt_san/mysys/thr_timer.c:271 (mariadbd+0x17b289b)
    #6 timer_handler /test/10.5_opt_san/mysys/thr_timer.c:315 (mariadbd+0x17b289b)
    #7 <null> <null> (libtsan.so.0+0x2d1af)
 
SUMMARY: ThreadSanitizer: data race /test/10.5_opt_san/storage/innobase/srv/srv0mon.cc:2099 in srv_mon_default_on()

/*************************************************************//**
Turn on monitor counters that are marked as default ON. */
void
srv_mon_default_on(void)
/*====================*/
{
  ulint   ix;
 
  for (ix = 0; ix < NUM_MONITOR; ix++) {
    if (innodb_counter_info[ix].monitor_type
        & MONITOR_DEFAULT_ON) {
      /* Turn on monitor counters that are default on */
      MONITOR_ON(ix);                                         <------------------------ Here
      MONITOR_INIT(ix);
      MONITOR_SET_START(ix);
    }
  }
}

Setup:

Compiled with GCC >=7.5.0 (I use GCC 9.3.0) and:
    -DWITH_TSAN=ON -DWSREP_LIB_WITH_TSAN=ON -DMUTEXTYPE=sys
Set before execution:
    export TSAN_OPTIONS=suppress_equal_stacks=1:suppress_equal_addresses=1:history_size=7:verbosity=1:exitcode=0

Bug confirmed present in:
MariaDB: 10.3.28 (dbg), 10.3.28 (opt), 10.4.18 (dbg), 10.4.18 (opt), 10.5.9 (dbg), 10.5.9 (opt), 10.6.0 (opt), 10.6.0 (dbg)

Bug confirmed not present in:
MariaDB: 10.1.49 (dbg), 10.1.49 (opt), 10.2.37 (dbg), 10.2.37 (opt)



 Comments   
Comment by Roel Van de Paar [ 2020-12-02 ]

First issue (during startup) happens between:

2020-12-02 13:28:36 0 [Note] InnoDB: Loading buffer pool(s) from /test/TSAN_MD011220-mariadb-10.5.9-linux-x86_64-opt/data/ib_buffer_pool

And:

2020-12-02 13:28:36 0 [Note] Plugin 'FEEDBACK' is disabled.

Generated at Thu Feb 08 09:29:10 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.