Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-25064

rpl.rpl_parallel_temptable failed in bb, ASAN heap-use-after-free in std::__atomic_base<long>::store

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Duplicate
    • 10.2.2, 10.2(EOL), 10.3(EOL), 10.4(EOL), 10.5, 10.6, 10.7(EOL), 10.8(EOL), 10.9(EOL), 10.10(EOL)
    • N/A
    • Tests

    Description

      http://buildbot.askmonty.org/buildbot/builders/kvm-asan/builds/5389/steps/mtr_nm/logs/stdio

      10.6 f691d9865becfd242ba44cc63

      rpl.rpl_parallel_temptable 'innodb,stmt' w3 [ fail ]
              Test ended at 2021-03-04 18:34:59
       
      CURRENT_TEST: rpl.rpl_parallel_temptable
      mysqltest: In included file "./include/stop_slave.inc": 
      included from /home/buildbot/buildbot/build/mariadb-10.6.0/mysql-test/suite/rpl/t/rpl_parallel_temptable.test at line 257:
      At line 82: query 'STOP SLAVE' failed: 2013: Lost connection to MySQL server during query
       
      The result from queries just before the failure was:
      < snip >
      INSERT INTO t4 VALUES (33);
      INSERT INTO t1 SELECT a, "optimistic" FROM t4;
      DROP TEMPORARY TABLE t4;
      SELECT * FROM t1 WHERE a >= 30 ORDER BY a;
      a	b
      30	conservative
      31	conservative
      32	optimistic
      33	optimistic
      include/save_master_gtid.inc
      connection server_2;
      include/start_slave.inc
      include/sync_with_master_gtid.inc
      SELECT * FROM t1 WHERE a >= 30 ORDER BY a;
      a	b
      30	conservative
      31	conservative
      32	optimistic
      33	optimistic
      include/stop_slave.inc
       
      More results from queries before failure can be found in /dev/shm/var/3/log/rpl_parallel_temptable.log
       
       
      Server [mysqld.2 - pid: 61746, winpid: 61746, exit: 256] failed during test run
      Server log from this test:
      ----------SERVER LOG START-----------
      2021-03-04 18:34:51 0 [Note] /home/buildbot/buildbot/build/mariadb-10.6.0/sql/mariadbd (mysqld 10.6.0-MariaDB-debug-log) starting as process 61747 ...
      2021-03-04 18:34:51 0 [Warning] Could not increase number of max_open_files to more than 1024 (request: 32186)
      2021-03-04 18:34:51 0 [Warning] Changed limits: max_open_files: 1024  max_connections: 151 (was 151)  table_cache: 421 (was 2000)
      2021-03-04 18:34:51 0 [Warning] setrlimit could not change the size of core files to 'infinity';  We may not be able to generate a core file on signals
      2021-03-04 18:34:51 0 [Note] Plugin 'partition' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'SEQUENCE' is disabled.
      2021-03-04 18:34:51 0 [Note] InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!!
      2021-03-04 18:34:51 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2021-03-04 18:34:51 0 [Note] InnoDB: Number of pools: 1
      2021-03-04 18:34:51 0 [Note] InnoDB: Using generic crc32 instructions
      2021-03-04 18:34:51 0 [Note] mariadbd: O_TMPFILE is not supported on /dev/shm/var/tmp/3/mysqld.2 (disabling future attempts)
      2021-03-04 18:34:51 0 [Note] InnoDB: Using Linux native AIO
      2021-03-04 18:34:51 0 [Note] InnoDB: Initializing buffer pool, total size = 8388608, chunk size = 8388608
      2021-03-04 18:34:51 0 [Note] InnoDB: Completed initialization of buffer pool
      2021-03-04 18:34:51 0 [Note] InnoDB: Setting O_DIRECT on file ./ibdata1 failed
      2021-03-04 18:34:51 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
      2021-03-04 18:34:51 0 [Note] InnoDB: 128 rollback segments are active.
      2021-03-04 18:34:51 0 [Note] InnoDB: Creating shared tablespace for temporary tables
      2021-03-04 18:34:51 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
      2021-03-04 18:34:51 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
      2021-03-04 18:34:51 0 [Note] InnoDB: 10.6.0 started; log sequence number 45179; transaction id 25
      2021-03-04 18:34:51 0 [Note] InnoDB: Loading buffer pool(s) from /dev/shm/var/3/mysqld.2/data/ib_buffer_pool
      2021-03-04 18:34:51 0 [Note] Plugin 'INNODB_FT_CONFIG' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'INNODB_SYS_TABLESTATS' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'INNODB_FT_DELETED' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'INNODB_CMP' is disabled.
      2021-03-04 18:34:51 0 [Note] InnoDB: Buffer pool(s) load completed at 210304 18:34:51
      2021-03-04 18:34:51 0 [Note] Plugin 'THREAD_POOL_WAITS' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'INNODB_CMP_RESET' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'THREAD_POOL_QUEUES' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'FEEDBACK' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'INNODB_FT_INDEX_TABLE' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'THREAD_POOL_GROUPS' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'INNODB_CMP_PER_INDEX_RESET' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'INNODB_FT_INDEX_CACHE' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'INNODB_FT_BEING_DELETED' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'INNODB_CMPMEM_RESET' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'INNODB_FT_DEFAULT_STOPWORD' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'INNODB_SYS_TABLESPACES' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'user_variables' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'INNODB_TABLESPACES_ENCRYPTION' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'THREAD_POOL_STATS' is disabled.
      2021-03-04 18:34:51 0 [Note] Plugin 'unix_socket' is disabled.
      2021-03-04 18:34:51 0 [Warning] /home/buildbot/buildbot/build/mariadb-10.6.0/sql/mariadbd: unknown option '--loose-pam-debug'
      2021-03-04 18:34:51 0 [Warning] /home/buildbot/buildbot/build/mariadb-10.6.0/sql/mariadbd: unknown option '--loose-aria'
      2021-03-04 18:34:51 0 [Note] Server socket created on IP: '127.0.0.1'.
      2021-03-04 18:34:51 0 [Note] Reading of all Master_info entries succeeded
      2021-03-04 18:34:51 0 [Note] Added new Master_info '' to hash table
      2021-03-04 18:34:51 0 [Note] /home/buildbot/buildbot/build/mariadb-10.6.0/sql/mariadbd: ready for connections.
      Version: '10.6.0-MariaDB-debug-log'  socket: '/dev/shm/var/tmp/3/mysqld.2.sock'  port: 16061  Source distribution
      2021-03-04 18:34:52 5 [Note] Deleted Master_info file '/dev/shm/var/3/mysqld.2/data/master.info'.
      2021-03-04 18:34:52 5 [Note] Deleted Master_info file '/dev/shm/var/3/mysqld.2/data/relay-log.info'.
      2021-03-04 18:34:52 5 [Note] Master connection name: ''  Master_info_file: 'master.info'  Relay_info_file: 'relay-log.info'
      2021-03-04 18:34:52 5 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='', master_port='3306', master_log_file='', master_log_pos='4'. New state master_host='127.0.0.1', master_port='16060', master_log_file='master-bin.000001', master_log_pos='4'.
      2021-03-04 18:34:52 7 [Note] Slave I/O thread: Start asynchronous replication to master 'root@127.0.0.1:16060' in log 'master-bin.000001' at position 4
      2021-03-04 18:34:52 8 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 4, relay log './slave-relay-bin.000001' position: 4
      2021-03-04 18:34:52 7 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16060',replication started in log 'master-bin.000001' at position 4
      2021-03-04 18:34:52 8 [Note] Error reading relay log event: slave SQL thread was killed
      2021-03-04 18:34:52 8 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 329, master: 127.0.0.1:16060
      2021-03-04 18:34:52 7 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 329, master 127.0.0.1:16060
      2021-03-04 18:34:52 5 [Note] Master connection name: ''  Master_info_file: 'master.info'  Relay_info_file: 'relay-log.info'
      2021-03-04 18:34:52 5 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='16060', master_log_file='master-bin.000001', master_log_pos='329'. New state master_host='127.0.0.1', master_port='16060', master_log_file='master-bin.000001', master_log_pos='329'.
      2021-03-04 18:34:52 5 [Note] Previous Using_Gtid=No. New Using_Gtid=Current_Pos
      2021-03-04 18:34:52 9 [Note] Slave I/O thread: Start asynchronous replication to master 'root@127.0.0.1:16060' in log 'master-bin.000001' at position 329
      2021-03-04 18:34:52 9 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16060',replication starts at GTID position ''
      2021-03-04 18:34:52 10 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 329, relay log './slave-relay-bin.000001' position: 4; GTID position ''
      2021-03-04 18:34:53 10 [Note] Error reading relay log event: slave SQL thread was killed
      2021-03-04 18:34:53 10 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 508; GTID position '0-1-1', master: 127.0.0.1:16060
      2021-03-04 18:34:53 9 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 508; GTID position 0-1-1, master 127.0.0.1:16060
      2021-03-04 18:34:55 16 [Note] Slave I/O thread: Start asynchronous replication to master 'root@127.0.0.1:16060' in log 'master-bin.000001' at position 508
      2021-03-04 18:34:55 16 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16060',replication starts at GTID position '0-1-1'
      2021-03-04 18:34:55 17 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 508, relay log './slave-relay-bin.000001' position: 4; GTID position '0-1-1'
      2021-03-04 18:34:55 17 [Note] Error reading relay log event: slave SQL thread was killed
      2021-03-04 18:34:55 17 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000002' at position 585; GTID position '0-1-2,1-1-23,2-1-13', master: 127.0.0.1:16060
      2021-03-04 18:34:55 16 [Note] Slave I/O thread exiting, read up to log 'master-bin.000002', position 585; GTID position 1-1-23,2-1-13,0-1-2, master 127.0.0.1:16060
      2021-03-04 18:34:57 23 [Note] Slave I/O thread: Start asynchronous replication to master 'root@127.0.0.1:16060' in log 'master-bin.000002' at position 585
      2021-03-04 18:34:57 23 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16060',replication starts at GTID position '1-1-23,0-1-2,2-1-13'
      2021-03-04 18:34:57 24 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000002' at position 585, relay log './slave-relay-bin.000001' position: 4; GTID position '0-1-2,1-1-23,2-1-13'
      2021-03-04 18:34:57 27 [Warning] Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statement accesses nontransactional table as well as transactional or temporary table, and writes to any of them Statement: INSERT INTO t1 SELECT a, 'default' FROM t3
      2021-03-04 18:34:57 28 [Warning] Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statement accesses nontransactional table as well as transactional or temporary table, and writes to any of them Statement: INSERT INTO t1 SELECT a, 'server_1' FROM t2
      2021-03-04 18:34:57 26 [Warning] Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statement accesses nontransactional table as well as transactional or temporary table, and writes to any of them Statement: INSERT INTO t1 SELECT a+2, '+server_1' FROM t2
      2021-03-04 18:34:57 29 [Warning] Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statement accesses nontransactional table as well as transactional or temporary table, and writes to any of them Statement: INSERT INTO t1 SELECT a+4, '++server_1' FROM t2
      2021-03-04 18:34:57 24 [Note] Error reading relay log event: slave SQL thread was killed
      2021-03-04 18:34:57 24 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000004' at position 1354; GTID position '0-1-9,1-1-31,2-1-17', master: 127.0.0.1:16060
      2021-03-04 18:34:57 23 [Note] Slave I/O thread exiting, read up to log 'master-bin.000004', position 1354; GTID position 1-1-32,0-1-9,2-1-17, master 127.0.0.1:16060
      2021-03-04 18:34:57 30 [Note] Slave I/O thread: Start asynchronous replication to master 'root@127.0.0.1:16060' in log 'master-bin.000004' at position 1354
      2021-03-04 18:34:57 30 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16060',replication starts at GTID position '1-1-31,0-1-9,2-1-17'
      2021-03-04 18:34:57 31 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000004' at position 1354, relay log './slave-relay-bin.000001' position: 4; GTID position '0-1-9,1-1-31,2-1-17'
      =================================================================
      ==61747==ERROR: AddressSanitizer: heap-use-after-free on address 0x6180000a5790 at pc 0x55b4e3c52d82 bp 0x7f67a6c47a40 sp 0x7f67a6c47a30
      WRITE of size 8 at 0x6180000a5790 thread T48
          #0 0x55b4e3c52d81 in std::__atomic_base<long>::store(long, std::memory_order) /usr/include/c++/9/bits/atomic_base.h:397
          #1 0x55b4e3c52d81 in Atomic_relaxed<long>::operator=(long) /home/buildbot/buildbot/build/mariadb-10.6.0/include/my_atomic_wrapper.h:46
          #2 0x55b4e40ec094 in trx_t::commit_tables() /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/trx/trx0trx.cc:1239
          #3 0x55b4e40ecf17 in trx_t::commit_in_memory(mtr_t const*) /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/trx/trx0trx.cc:1333
          #4 0x55b4e40e4abd in trx_t::commit_low(mtr_t*) /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/trx/trx0trx.cc:1507
          #5 0x55b4e40e4bf2 in trx_t::commit() /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/trx/trx0trx.cc:1521
          #6 0x55b4e40e534a in trx_commit_for_mysql(trx_t*) /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/trx/trx0trx.cc:1638
      2021-03-04 18:34:58 31 [Note] Error reading relay log event: slave SQL thread was killed
          #7 0x55b4e3bdaff7 in innobase_commit_low(trx_t*) /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/handler/ha_innodb.cc:3827
          #8 0x55b4e3bdb724 in innobase_commit_ordered_2 /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/handler/ha_innodb.cc:3933
          #9 0x55b4e3bdc1f1 in innobase_commit /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/handler/ha_innodb.cc:4037
          #10 0x55b4e30d48a8 in commit_one_phase_2 /home/buildbot/buildbot/build/mariadb-10.6.0/sql/handler.cc:1942
          #11 0x55b4e30d45bc in ha_commit_one_phase(THD*, bool) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/handler.cc:1921
          #12 0x55b4e30d290c in ha_commit_trans(THD*, bool) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/handler.cc:1715
          #13 0x55b4e2d447b8 in trans_commit(THD*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/transaction.cc:266
          #14 0x55b4e28e2dc7 in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_parse.cc:5573
          #15 0x55b4e28f2f5f in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_parse.cc:7972
          #16 0x55b4e34471db in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/log_event_server.cc:1906
          #17 0x55b4e34443fc in Query_log_event::do_apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/log_event_server.cc:1581
          #18 0x55b4e2669fd2 in Log_event::apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/log_event.h:1498
          #19 0x55b4e264bb66 in apply_event_and_update_pos_apply /home/buildbot/buildbot/build/mariadb-10.6.0/sql/slave.cc:3850
          #20 0x55b4e264c8ed in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/slave.cc:4036
          #21 0x55b4e2e464d8 in rpt_handle_event /home/buildbot/buildbot/build/mariadb-10.6.0/sql/rpl_parallel.cc:62
          #22 0x55b4e2e4dd3d in handle_rpl_parallel_thread /home/buildbot/buildbot/build/mariadb-10.6.0/sql/rpl_parallel.cc:1363
          #23 0x55b4e39f56a0 in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.6.0/storage/perfschema/pfs.cc:2201
          #24 0x7f67b704e608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
          #25 0x7f67b6c22102 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x122102)
       
      0x6180000a5790 is located 784 bytes inside of 816-byte region [0x6180000a5480,0x6180000a57b0)
      freed by thread T45 here:
          #0 0x7f67b729d7cf in __interceptor_free (/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf)
          #1 0x55b4e3c48f5d in ut_allocator<unsigned char, true>::deallocate(unsigned char*, unsigned long) /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/include/ut0new.h:426
          #2 0x55b4e3e0d745 in mem_heap_block_free(mem_block_info_t*, mem_block_info_t*) /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/mem/mem0mem.cc:416
          #3 0x55b4e429cd34 in mem_heap_free /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/include/mem0mem.ic:419
          #4 0x55b4e429ff3d in dict_mem_table_free(dict_table_t*) /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/dict/dict0mem.cc:246
          #5 0x55b4e4262a5e in dict_sys_t::remove(dict_table_t*, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/dict/dict0dict.cc:2063
          #6 0x55b4e3f8c4ad in row_drop_table_for_mysql(char const*, trx_t*, enum_sql_command, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/row/row0mysql.cc:3320
          #7 0x55b4e3c46817 in ha_innobase::delete_table(char const*, enum_sql_command) (/home/buildbot/buildbot/build/mariadb-10.6.0/sql/mariadbd+0x2de4817)
          #8 0x55b4e3c0ffd4 in ha_innobase::delete_table(char const*) /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/handler/ha_innodb.cc:13146
          #9 0x55b4e30cd713 in hton_drop_table /home/buildbot/buildbot/build/mariadb-10.6.0/sql/handler.cc:564
          #10 0x55b4e2f254df in THD::rm_temporary_table(handlerton*, char const*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/temporary_tables.cc:703
          #11 0x55b4e2f2a505 in THD::free_tmp_table_share(TMP_TABLE_SHARE*, bool) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/temporary_tables.cc:1460
          #12 0x55b4e2f250da in THD::drop_temporary_table(TABLE*, bool*, bool) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/temporary_tables.cc:669
          #13 0x55b4e2b461a8 in mysql_rm_table_no_locks(THD*, TABLE_LIST*, bool, bool, bool, bool, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_table.cc:2365
          #14 0x55b4e2b44dbe in mysql_rm_table(THD*, TABLE_LIST*, bool, bool, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_table.cc:2137
          #15 0x55b4e28de484 in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_parse.cc:4903
          #16 0x55b4e28f2f5f in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_parse.cc:7972
          #17 0x55b4e34471db in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/log_event_server.cc:1906
          #18 0x55b4e34443fc in Query_log_event::do_apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/log_event_server.cc:1581
          #19 0x55b4e2669fd2 in Log_event::apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/log_event.h:1498
          #20 0x55b4e264bb66 in apply_event_and_update_pos_apply /home/buildbot/buildbot/build/mariadb-10.6.0/sql/slave.cc:3850
          #21 0x55b4e264c8ed in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/slave.cc:4036
          #22 0x55b4e2e464d8 in rpt_handle_event /home/buildbot/buildbot/build/mariadb-10.6.0/sql/rpl_parallel.cc:62
          #23 0x55b4e2e4dd3d in handle_rpl_parallel_thread /home/buildbot/buildbot/build/mariadb-10.6.0/sql/rpl_parallel.cc:1363
          #24 0x55b4e39f56a0 in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.6.0/storage/perfschema/pfs.cc:2201
          #25 0x7f67b704e608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
       
      previously allocated by thread T48 here:
          #0 0x7f67b729dbc8 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10dbc8)
          #1 0x55b4e3c48b1c in ut_allocator<unsigned char, true>::allocate(unsigned long, unsigned char const*, unsigned int, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/include/ut0new.h:377
          #2 0x55b4e3e0c9f9 in mem_heap_create_block_func(mem_block_info_t*, unsigned long, char const*, unsigned int, unsigned long) /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/mem/mem0mem.cc:277
          #3 0x55b4e3e0d339 in mem_heap_add_block(mem_block_info_t*, unsigned long) /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/mem/mem0mem.cc:378
          #4 0x55b4e429c9c1 in mem_heap_alloc /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/include/mem0mem.ic:193
          #5 0x55b4e429c7d5 in mem_heap_zalloc /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/include/mem0mem.ic:162
          #6 0x55b4e429ee1e in dict_mem_table_create(char const*, fil_space_t*, unsigned long, unsigned long, unsigned long, unsigned long) /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/dict/dict0mem.cc:152
          #7 0x55b4e3c40c3d in create_table_info_t::create_table_def() (/home/buildbot/buildbot/build/mariadb-10.6.0/sql/mariadbd+0x2ddec3d)
          #8 0x55b4e3c0ba3e in create_table_info_t::create_table(bool) /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/handler/ha_innodb.cc:12196
          #9 0x55b4e3c45ec6 in ha_innobase::create(char const*, TABLE*, HA_CREATE_INFO*, bool, trx_t*) (/home/buildbot/buildbot/build/mariadb-10.6.0/sql/mariadbd+0x2de3ec6)
          #10 0x55b4e3c0e43d in ha_innobase::create(char const*, TABLE*, HA_CREATE_INFO*) /home/buildbot/buildbot/build/mariadb-10.6.0/storage/innobase/handler/ha_innodb.cc:12792
          #11 0x55b4e30f1e17 in handler::ha_create(char const*, TABLE*, HA_CREATE_INFO*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/handler.cc:5094
          #12 0x55b4e30f650f in ha_create_table(THD*, char const*, char const*, char const*, HA_CREATE_INFO*, st_mysql_const_unsigned_lex_string*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/handler.cc:5559
          #13 0x55b4e2b5cb22 in create_table_impl /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_table.cc:5376
          #14 0x55b4e2b5d496 in mysql_create_table_no_lock(THD*, st_mysql_const_lex_string const*, st_mysql_const_lex_string const*, Table_specification_st*, Alter_info*, bool*, int, TABLE_LIST*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_table.cc:5460
          #15 0x55b4e2b5e102 in mysql_create_table(THD*, TABLE_LIST*, Table_specification_st*, Alter_info*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_table.cc:5564
          #16 0x55b4e2b8f194 in Sql_cmd_create_table_like::execute(THD*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_table.cc:12248
          #17 0x55b4e28e55ed in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_parse.cc:5946
          #18 0x55b4e28f2f5f in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_parse.cc:7972
          #19 0x55b4e34471db in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/log_event_server.cc:1906
          #20 0x55b4e34443fc in Query_log_event::do_apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/log_event_server.cc:1581
          #21 0x55b4e2669fd2 in Log_event::apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/log_event.h:1498
          #22 0x55b4e264bb66 in apply_event_and_update_pos_apply /home/buildbot/buildbot/build/mariadb-10.6.0/sql/slave.cc:3850
          #23 0x55b4e264c8ed in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/slave.cc:4036
          #24 0x55b4e2e464d8 in rpt_handle_event /home/buildbot/buildbot/build/mariadb-10.6.0/sql/rpl_parallel.cc:62
          #25 0x55b4e2e4dd3d in handle_rpl_parallel_thread /home/buildbot/buildbot/build/mariadb-10.6.0/sql/rpl_parallel.cc:1363
          #26 0x55b4e39f56a0 in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.6.0/storage/perfschema/pfs.cc:2201
          #27 0x7f67b704e608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
       
      Thread T48 created by T43 here:
          #0 0x7f67b71ca805 in pthread_create (/lib/x86_64-linux-gnu/libasan.so.5+0x3a805)
          #1 0x55b4e39f0694 in my_thread_create /home/buildbot/buildbot/build/mariadb-10.6.0/storage/perfschema/my_thread.h:38
          #2 0x55b4e39f5a93 in pfs_spawn_thread_v1 /home/buildbot/buildbot/build/mariadb-10.6.0/storage/perfschema/pfs.cc:2252
          #3 0x55b4e2e4516d in inline_mysql_thread_create /home/buildbot/buildbot/build/mariadb-10.6.0/include/mysql/psi/mysql_thread.h:1139
          #4 0x55b4e2e4f69d in rpl_parallel_change_thread_count /home/buildbot/buildbot/build/mariadb-10.6.0/sql/rpl_parallel.cc:1607
          #5 0x55b4e2e50773 in rpl_parallel_activate_pool(rpl_parallel_thread_pool*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/rpl_parallel.cc:1747
          #6 0x55b4e2653cbf in handle_slave_sql /home/buildbot/buildbot/build/mariadb-10.6.0/sql/slave.cc:5240
          #7 0x55b4e39f56a0 in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.6.0/storage/perfschema/pfs.cc:2201
          #8 0x7f67b704e608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
       
      Thread T43 created by T15 here:
          #0 0x7f67b71ca805 in pthread_create (/lib/x86_64-linux-gnu/libasan.so.5+0x3a805)
          #1 0x55b4e39f0694 in my_thread_create /home/buildbot/buildbot/build/mariadb-10.6.0/storage/perfschema/my_thread.h:38
          #2 0x55b4e39f5a93 in pfs_spawn_thread_v1 /home/buildbot/buildbot/build/mariadb-10.6.0/storage/perfschema/pfs.cc:2252
          #3 0x55b4e26325fe in inline_mysql_thread_create /home/buildbot/buildbot/build/mariadb-10.6.0/include/mysql/psi/mysql_thread.h:1139
          #4 0x55b4e263a4bb in start_slave_thread(unsigned int, void* (*)(void*), st_mysql_mutex*, st_mysql_mutex*, st_mysql_cond*, unsigned int volatile*, unsigned long volatile*, Master_info*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/slave.cc:1148
          #5 0x55b4e263affa in start_slave_threads(THD*, bool, bool, Master_info*, char const*, char const*, int) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/slave.cc:1274
          #6 0x55b4e296f4f2 in start_slave(THD*, Master_info*, bool) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_repl.cc:3241
          #7 0x55b4e28d85d6 in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_parse.cc:4201
          #8 0x55b4e28f2f5f in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_parse.cc:7972
          #9 0x55b4e28c9a60 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_parse.cc:1886
          #10 0x55b4e28c67a2 in do_command(THD*, bool) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_parse.cc:1397
          #11 0x55b4e2d040cb in do_handle_one_connection(CONNECT*, bool) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_connect.cc:1410
          #12 0x55b4e2d03a2f in handle_one_connection /home/buildbot/buildbot/build/mariadb-10.6.0/sql/sql_connect.cc:1312
          #13 0x55b4e39f56a0 in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.6.0/storage/perfschema/pfs.cc:2201
          #14 0x7f67b704e608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
       
      Thread T15 created by T0 here:
          #0 0x7f67b71ca805 in pthread_create (/lib/x86_64-linux-gnu/libasan.so.5+0x3a805)
          #1 0x55b4e39f0694 in my_thread_create /home/buildbot/buildbot/build/mariadb-10.6.0/storage/perfschema/my_thread.h:38
          #2 0x55b4e39f5a93 in pfs_spawn_thread_v1 /home/buildbot/buildbot/build/mariadb-10.6.0/storage/perfschema/pfs.cc:2252
          #3 0x55b4e25bfcda in inline_mysql_thread_create /home/buildbot/buildbot/build/mariadb-10.6.0/include/mysql/psi/mysql_thread.h:1139
          #4 0x55b4e25d591f in create_thread_to_handle_connection(CONNECT*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/mysqld.cc:5795
          #5 0x55b4e25d5f94 in create_new_thread(CONNECT*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/mysqld.cc:5854
          #6 0x55b4e25d62ea in handle_accepted_socket(st_mysql_socket, st_mysql_socket) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/mysqld.cc:5919
          #7 0x55b4e25d6f0b in handle_connections_sockets() /home/buildbot/buildbot/build/mariadb-10.6.0/sql/mysqld.cc:6046
          #8 0x55b4e25d5135 in mysqld_main(int, char**) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/mysqld.cc:5690
          #9 0x55b4e25bf00c in main /home/buildbot/buildbot/build/mariadb-10.6.0/sql/main.cc:25
          #10 0x7f67b6b270b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
       
      Thread T45 created by T43 here:
          #0 0x7f67b71ca805 in pthread_create (/lib/x86_64-linux-gnu/libasan.so.5+0x3a805)
          #1 0x55b4e39f0694 in my_thread_create /home/buildbot/buildbot/build/mariadb-10.6.0/storage/perfschema/my_thread.h:38
          #2 0x55b4e39f5a93 in pfs_spawn_thread_v1 /home/buildbot/buildbot/build/mariadb-10.6.0/storage/perfschema/pfs.cc:2252
          #3 0x55b4e2e4516d in inline_mysql_thread_create /home/buildbot/buildbot/build/mariadb-10.6.0/include/mysql/psi/mysql_thread.h:1139
          #4 0x55b4e2e4f69d in rpl_parallel_change_thread_count /home/buildbot/buildbot/build/mariadb-10.6.0/sql/rpl_parallel.cc:1607
          #5 0x55b4e2e50773 in rpl_parallel_activate_pool(rpl_parallel_thread_pool*) /home/buildbot/buildbot/build/mariadb-10.6.0/sql/rpl_parallel.cc:1747
          #6 0x55b4e2653cbf in handle_slave_sql /home/buildbot/buildbot/build/mariadb-10.6.0/sql/slave.cc:5240
          #7 0x55b4e39f56a0 in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.6.0/storage/perfschema/pfs.cc:2201
          #8 0x7f67b704e608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
       
      SUMMARY: AddressSanitizer: heap-use-after-free /usr/include/c++/9/bits/atomic_base.h:397 in std::__atomic_base<long>::store(long, std::memory_order)
      Shadow bytes around the buggy address:
        0x0c308000caa0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x0c308000cab0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x0c308000cac0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x0c308000cad0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x0c308000cae0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
      =>0x0c308000caf0: fd fd[fd]fd fd fd fa fa fa fa fa fa fa fa fa fa
        0x0c308000cb00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x0c308000cb10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x0c308000cb20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x0c308000cb30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x0c308000cb40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      Shadow byte legend (one shadow byte represents 8 application bytes):
        Addressable:           00
        Partially addressable: 01 02 03 04 05 06 07 
        Heap left redzone:       fa
        Freed heap region:       fd
        Stack left redzone:      f1
        Stack mid redzone:       f2
        Stack right redzone:     f3
        Stack after return:      f5
        Stack use after scope:   f8
        Global redzone:          f9
        Global init order:       f6
        Poisoned by user:        f7
        Container overflow:      fc
        Array cookie:            ac
        Intra object redzone:    bb
        ASan internal:           fe
        Left alloca redzone:     ca
        Right alloca redzone:    cb
        Shadow gap:              cc
      ==61747==ABORTING
      210304 18:34:58 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed, 
      something is definitely wrong and this may fail.
       
      Server version: 10.6.0-MariaDB-debug-log
      key_buffer_size=1048576
      read_buffer_size=131072
      max_used_connections=2
      max_threads=153
      thread_count=10
      It is possible that mysqld could use up to 
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63794 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x62b000299288
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x7f67a6c4a850 thread_stack 0x5fc00
      ??:0(__interceptor_tcgetattr)[0x7f67b71fcd30]
      mysys/stacktrace.c:212(my_print_stacktrace)[0x55b4e45ffb7b]
      sql/signal_handler.cc:212(handle_fatal_signal)[0x55b4e30c4faf]
      ??:0(__restore_rt)[0x7f67b705a3c0]
      ??:0(gsignal)[0x7f67b6b4618b]
      ??:0(abort)[0x7f67b6b25859]
      ??:0(__sanitizer_set_report_fd)[0x7f67b72bb6a2]
      ??:0(__sanitizer_get_module_and_offset_for_pc)[0x7f67b72c624c]
      ??:0(__sanitizer_ptr_cmp)[0x7f67b72a78ec]
      ??:0(__asan_on_error)[0x7f67b72a7363]
      ??:0(__asan_report_store8)[0x7f67b72a874e]
      bits/atomic_base.h:397(std::__atomic_base<long>::store(long, std::memory_order))[0x55b4e3c52d82]
      trx/trx0trx.cc:1240(trx_t::commit_tables())[0x55b4e40ec095]
      trx/trx0trx.cc:1334(trx_t::commit_in_memory(mtr_t const*))[0x55b4e40ecf18]
      trx/trx0trx.cc:1508(trx_t::commit_low(mtr_t*))[0x55b4e40e4abe]
      trx/trx0trx.cc:1514(trx_t::commit())[0x55b4e40e4bf3]
      trx/trx0trx.cc:1639(trx_commit_for_mysql(trx_t*))[0x55b4e40e534b]
      handler/ha_innodb.cc:3827(innobase_commit_low(trx_t*))[0x55b4e3bdaff8]
      handler/ha_innodb.cc:3935(innobase_commit_ordered_2(trx_t*, THD*))[0x55b4e3bdb725]
      handler/ha_innodb.cc:4047(innobase_commit(handlerton*, THD*, bool))[0x55b4e3bdc1f2]
      sql/handler.cc:1942(commit_one_phase_2(THD*, bool, THD_TRANS*, bool))[0x55b4e30d48a9]
      sql/handler.cc:1921(ha_commit_one_phase(THD*, bool))[0x55b4e30d45bd]
      sql/handler.cc:1715(ha_commit_trans(THD*, bool))[0x55b4e30d290d]
      sql/transaction.cc:266(trans_commit(THD*))[0x55b4e2d447b9]
      sql/sql_parse.cc:5573(mysql_execute_command(THD*))[0x55b4e28e2dc8]
      sql/sql_parse.cc:7972(mysql_parse(THD*, char*, unsigned int, Parser_state*))[0x55b4e28f2f60]
      sql/log_event_server.cc:1908(Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int))[0x55b4e34471dc]
      sql/log_event_server.cc:1582(Query_log_event::do_apply_event(rpl_group_info*))[0x55b4e34443fd]
      sql/log_event.h:1498(Log_event::apply_event(rpl_group_info*))[0x55b4e2669fd3]
      sql/slave.cc:3850(apply_event_and_update_pos_apply(Log_event*, THD*, rpl_group_info*, int))[0x55b4e264bb67]
      sql/slave.cc:4037(apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*))[0x55b4e264c8ee]
      sql/rpl_parallel.cc:62(rpt_handle_event(rpl_parallel_thread::queued_event*, rpl_parallel_thread*))[0x55b4e2e464d9]
      sql/rpl_parallel.cc:1363(handle_rpl_parallel_thread)[0x55b4e2e4dd3e]
      perfschema/pfs.cc:2203(pfs_spawn_thread)[0x55b4e39f56a1]
      ??:0(start_thread)[0x7f67b704e609]
      ??:0(clone)[0x7f67b6c22103]
       
      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      Query (0x612000106753): COMMIT
       
      Connection ID (thread ID): 36
      Status: NOT_KILLED
       
      Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off
       
      The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
      information that should help you find out what is causing the crash.
      Writing a core file...
      Working directory at /dev/shm/var/3/mysqld.2/data
      Resource Limits:
      Limit                     Soft Limit           Hard Limit           Units     
      Max cpu time              unlimited            unlimited            seconds   
      Max file size             unlimited            unlimited            bytes     
      Max data size             unlimited            unlimited            bytes     
      Max stack size            8388608              unlimited            bytes     
      Max core file size        0                    0                    bytes     
      Max resident set          unlimited            unlimited            bytes     
      Max processes             23441                23441                processes 
      Max open files            1024                 1024                 files     
      Max locked memory         67108864             67108864             bytes     
      Max address space         unlimited            unlimited            bytes     
      Max file locks            unlimited            unlimited            locks     
      Max pending signals       23441                23441                signals   
      Max msgqueue size         819200               819200               bytes     
      Max nice priority         0                    0                    
      Max realtime priority     0                    0                    
      Max realtime timeout      unlimited            unlimited            us        
      Core pattern: |/usr/share/apport/apport %p %s %c %d %P %E
       
      ----------SERVER LOG END-------------
      

      Attachments

        Issue Links

          Activity

            I noticed another case of this:

            10.6 11597e02f3bcca6d3f7b7992c0fa63e4

            CURRENT_TEST: rpl.rpl_parallel_temptable
            mysqltest: In included file "./include/stop_slave.inc": 
            included from /home/buildbot/buildbot/build/mariadb-10.6.1/mysql-test/suite/rpl/t/rpl_parallel_temptable.test at line 257:
            At line 82: query 'STOP SLAVE' failed: 2013: Lost connection to server during query
            …
            2021-05-04 16:31:32 30 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16060',replication starts at GTID position '1-1-31,0-1-9,2-1-17'
            2021-05-04 16:31:32 31 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000004' at position 1354, relay log './slave-relay-bin.000001' position: 4; GTID position '0-1-9,1-1-31,2-1-17'
            =================================================================
            ==60687==ERROR: AddressSanitizer: heap-use-after-free on address 0x618000078390 at pc 0x55e70a84bf68 bp 0x7f20db357fe0 sp 0x7f20db357fd0
            WRITE of size 8 at 0x618000078390 thread T44
            …
                #2 0x55e70acf42b4 in trx_t::commit_tables() /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/trx/trx0trx.cc:1229
            …
                #13 0x55e709911d15 in trans_commit(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/transaction.cc:266
                #14 0x55e7094a6f6e in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:5613
                #15 0x55e7094b7021 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:8018
                #16 0x55e70a02e117 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1906
                #17 0x55e70a02b338 in Query_log_event::do_apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1581
                #18 0x55e709229afe in Log_event::apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event.h:1498
                #19 0x55e70920b42c in apply_event_and_update_pos_apply /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:3850
                #20 0x55e70920c1b8 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:4036
                #21 0x55e709a1515d in rpt_handle_event /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:62
                #22 0x55e709a1cddd in handle_rpl_parallel_thread /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:1378
            …
            0x618000078390 is located 784 bytes inside of 816-byte region [0x618000078080,0x6180000783b0)
            freed by thread T42 here:
            …
                #6 0x55e70ab91864 in row_drop_table_for_mysql(char const*, trx_t*, enum_sql_command, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/row/row0mysql.cc:3286
            …
                #10 0x55e709af79a7 in THD::rm_temporary_table(handlerton*, char const*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/temporary_tables.cc:703
                #11 0x55e709afca4d in THD::free_tmp_table_share(TMP_TABLE_SHARE*, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/temporary_tables.cc:1462
                #12 0x55e709af7598 in THD::drop_temporary_table(TABLE*, bool*, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/temporary_tables.cc:669
                #13 0x55e70970f9bb in mysql_rm_table_no_locks(THD*, TABLE_LIST*, bool, bool, bool, bool, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:2365
                #14 0x55e70970e5d8 in mysql_rm_table(THD*, TABLE_LIST*, bool, bool, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:2138
                #15 0x55e7094a2468 in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:4943
                #16 0x55e7094b7021 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:8018
                #17 0x55e70a02e117 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1906
                #18 0x55e70a02b338 in Query_log_event::do_apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1581
                #19 0x55e709229afe in Log_event::apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event.h:1498
                #20 0x55e70920b42c in apply_event_and_update_pos_apply /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:3850
                #21 0x55e70920c1b8 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:4036
                #22 0x55e709a1515d in rpt_handle_event /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:62
                #23 0x55e709a1cddd in handle_rpl_parallel_thread /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:1378
            …
            previously allocated by thread T45 here:
            …
                #15 0x55e709728902 in mysql_create_table(THD*, TABLE_LIST*, Table_specification_st*, Alter_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:5580
                #16 0x55e70975a08f in Sql_cmd_create_table_like::execute(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:12288
                #17 0x55e7094a9823 in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:5986
                #18 0x55e7094b7021 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:8018
                #19 0x55e70a02e117 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1906
                #20 0x55e70a02b338 in Query_log_event::do_apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1581
                #21 0x55e709229afe in Log_event::apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event.h:1498
                #22 0x55e70920b42c in apply_event_and_update_pos_apply /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:3850
                #23 0x55e70920c1b8 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:4036
                #24 0x55e709a1515d in rpt_handle_event /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:62
                #25 0x55e709a1cddd in handle_rpl_parallel_thread /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:1378
            

            Starting with MDEV-515, trx_t::mod_tables will include temporary tables. That is probably why this failure pattern started to occur on 10.6.

            Starting with MariaDB Server 10.2.2, InnoDB treats temporary tables specially. For replication, the most relevant difference is that InnoDB skips any locking for temporary tables. It assumes that a temporary table is only being accessed by the thread that created it. In the above output, 3 distinct threads have been identified as creating, dropping, and modifying the temporary table.

            marko Marko Mäkelä added a comment - I noticed another case of this : 10.6 11597e02f3bcca6d3f7b7992c0fa63e4 CURRENT_TEST: rpl.rpl_parallel_temptable mysqltest: In included file "./include/stop_slave.inc": included from /home/buildbot/buildbot/build/mariadb-10.6.1/mysql-test/suite/rpl/t/rpl_parallel_temptable.test at line 257: At line 82: query 'STOP SLAVE' failed: 2013: Lost connection to server during query … 2021-05-04 16:31:32 30 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16060',replication starts at GTID position '1-1-31,0-1-9,2-1-17' 2021-05-04 16:31:32 31 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000004' at position 1354, relay log './slave-relay-bin.000001' position: 4; GTID position '0-1-9,1-1-31,2-1-17' ================================================================= ==60687==ERROR: AddressSanitizer: heap-use-after-free on address 0x618000078390 at pc 0x55e70a84bf68 bp 0x7f20db357fe0 sp 0x7f20db357fd0 WRITE of size 8 at 0x618000078390 thread T44 … #2 0x55e70acf42b4 in trx_t::commit_tables() /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/trx/trx0trx.cc:1229 … #13 0x55e709911d15 in trans_commit(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/transaction.cc:266 #14 0x55e7094a6f6e in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:5613 #15 0x55e7094b7021 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:8018 #16 0x55e70a02e117 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1906 #17 0x55e70a02b338 in Query_log_event::do_apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1581 #18 0x55e709229afe in Log_event::apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event.h:1498 #19 0x55e70920b42c in apply_event_and_update_pos_apply /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:3850 #20 0x55e70920c1b8 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:4036 #21 0x55e709a1515d in rpt_handle_event /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:62 #22 0x55e709a1cddd in handle_rpl_parallel_thread /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:1378 … 0x618000078390 is located 784 bytes inside of 816-byte region [0x618000078080,0x6180000783b0) freed by thread T42 here: … #6 0x55e70ab91864 in row_drop_table_for_mysql(char const*, trx_t*, enum_sql_command, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/row/row0mysql.cc:3286 … #10 0x55e709af79a7 in THD::rm_temporary_table(handlerton*, char const*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/temporary_tables.cc:703 #11 0x55e709afca4d in THD::free_tmp_table_share(TMP_TABLE_SHARE*, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/temporary_tables.cc:1462 #12 0x55e709af7598 in THD::drop_temporary_table(TABLE*, bool*, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/temporary_tables.cc:669 #13 0x55e70970f9bb in mysql_rm_table_no_locks(THD*, TABLE_LIST*, bool, bool, bool, bool, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:2365 #14 0x55e70970e5d8 in mysql_rm_table(THD*, TABLE_LIST*, bool, bool, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:2138 #15 0x55e7094a2468 in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:4943 #16 0x55e7094b7021 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:8018 #17 0x55e70a02e117 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1906 #18 0x55e70a02b338 in Query_log_event::do_apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1581 #19 0x55e709229afe in Log_event::apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event.h:1498 #20 0x55e70920b42c in apply_event_and_update_pos_apply /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:3850 #21 0x55e70920c1b8 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:4036 #22 0x55e709a1515d in rpt_handle_event /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:62 #23 0x55e709a1cddd in handle_rpl_parallel_thread /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:1378 … previously allocated by thread T45 here: … #15 0x55e709728902 in mysql_create_table(THD*, TABLE_LIST*, Table_specification_st*, Alter_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:5580 #16 0x55e70975a08f in Sql_cmd_create_table_like::execute(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:12288 #17 0x55e7094a9823 in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:5986 #18 0x55e7094b7021 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:8018 #19 0x55e70a02e117 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1906 #20 0x55e70a02b338 in Query_log_event::do_apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1581 #21 0x55e709229afe in Log_event::apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event.h:1498 #22 0x55e70920b42c in apply_event_and_update_pos_apply /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:3850 #23 0x55e70920c1b8 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:4036 #24 0x55e709a1515d in rpt_handle_event /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:62 #25 0x55e709a1cddd in handle_rpl_parallel_thread /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:1378 Starting with MDEV-515 , trx_t::mod_tables will include temporary tables. That is probably why this failure pattern started to occur on 10.6. Starting with MariaDB Server 10.2.2, InnoDB treats temporary tables specially. For replication, the most relevant difference is that InnoDB skips any locking for temporary tables. It assumes that a temporary table is only being accessed by the thread that created it. In the above output, 3 distinct threads have been identified as creating, dropping, and modifying the temporary table.

            Another recent failure:

            10.6 11597e02f3bcca6d3f7b7992c0fa63e45e614b1c

            rpl.rpl_parallel_temptable 'innodb,stmt' w2 [ fail ]
                    Test ended at 2021-05-04 16:31:36
             
            CURRENT_TEST: rpl.rpl_parallel_temptable
            mysqltest: In included file "./include/stop_slave.inc": 
            included from /home/buildbot/buildbot/build/mariadb-10.6.1/mysql-test/suite/rpl/t/rpl_parallel_temptable.test at line 257:
            At line 82: query 'STOP SLAVE' failed: 2013: Lost connection to server during query
            …
            2021-05-04 16:31:32 30 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16060',replication starts at GTID position '1-1-31,0-1-9,2-1-17'
            2021-05-04 16:31:32 31 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000004' at position 1354, relay log './slave-relay-bin.000001' position: 4; GTID position '0-1-9,1-1-31,2-1-17'
            =================================================================
            ==60687==ERROR: AddressSanitizer: heap-use-after-free on address 0x618000078390 at pc 0x55e70a84bf68 bp 0x7f20db357fe0 sp 0x7f20db357fd0
            WRITE of size 8 at 0x618000078390 thread T44
                #0 0x55e70a84bf67 in std::__atomic_base<long>::store(long, std::memory_order) /usr/include/c++/9/bits/atomic_base.h:397
                #1 0x55e70a84bf67 in Atomic_relaxed<long>::operator=(long) /home/buildbot/buildbot/build/mariadb-10.6.1/include/my_atomic_wrapper.h:48
                #2 0x55e70acf42b4 in trx_t::commit_tables() /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/trx/trx0trx.cc:1229
                #3 0x55e70acf513b in trx_t::commit_in_memory(mtr_t const*) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/trx/trx0trx.cc:1323
                #4 0x55e70aceccbd in trx_t::commit_low(mtr_t*) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/trx/trx0trx.cc:1493
                #5 0x55e70acecdf2 in trx_t::commit() /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/trx/trx0trx.cc:1507
                #6 0x55e70aced54a in trx_commit_for_mysql(trx_t*) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/trx/trx0trx.cc:1624
            2021-05-04 16:31:33 31 [Note] Error reading relay log event: slave SQL thread was killed
                #7 0x55e70a7cff89 in innobase_commit_low(trx_t*) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/handler/ha_innodb.cc:3874
                #8 0x55e70a7d06c4 in innobase_commit_ordered_2 /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/handler/ha_innodb.cc:3980
                #9 0x55e70a7d119c in innobase_commit /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/handler/ha_innodb.cc:4084
                #10 0x55e709cb702a in commit_one_phase_2 /home/buildbot/buildbot/build/mariadb-10.6.1/sql/handler.cc:1944
                #11 0x55e709cb6d35 in ha_commit_one_phase(THD*, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/handler.cc:1923
                #12 0x55e709cb5085 in ha_commit_trans(THD*, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/handler.cc:1717
                #13 0x55e709911d15 in trans_commit(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/transaction.cc:266
                #14 0x55e7094a6f6e in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:5613
                #15 0x55e7094b7021 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:8018
                #16 0x55e70a02e117 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1906
                #17 0x55e70a02b338 in Query_log_event::do_apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1581
                #18 0x55e709229afe in Log_event::apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event.h:1498
                #19 0x55e70920b42c in apply_event_and_update_pos_apply /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:3850
                #20 0x55e70920c1b8 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:4036
                #21 0x55e709a1515d in rpt_handle_event /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:62
                #22 0x55e709a1cddd in handle_rpl_parallel_thread /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:1378
                #23 0x55e70a5e3f0c in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.6.1/storage/perfschema/pfs.cc:2201
                #24 0x7f20eaf13608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
                #25 0x7f20eaae7292 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x122292)
             
            0x618000078390 is located 784 bytes inside of 816-byte region [0x618000078080,0x6180000783b0)
            freed by thread T42 here:
                #0 0x7f20eb1a97cf in __interceptor_free (/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf)
                #1 0x55e70a84007d in ut_allocator<unsigned char, true>::deallocate(unsigned char*, unsigned long) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/include/ut0new.h:424
                #2 0x55e70aa0e695 in mem_heap_block_free(mem_block_info_t*, mem_block_info_t*) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/mem/mem0mem.cc:416
                #3 0x55e70aea95d8 in mem_heap_free /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/include/mem0mem.ic:419
                #4 0x55e70aeac804 in dict_mem_table_free(dict_table_t*) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/dict/dict0mem.cc:246
                #5 0x55e70ae6e6be in dict_sys_t::remove(dict_table_t*, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/dict/dict0dict.cc:2012
                #6 0x55e70ab91864 in row_drop_table_for_mysql(char const*, trx_t*, enum_sql_command, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/row/row0mysql.cc:3286
                #7 0x55e70a83d791 in ha_innobase::delete_table(char const*, enum_sql_command) (/home/buildbot/buildbot/build/mariadb-10.6.1/sql/mariadbd+0x2e46791)
                #8 0x55e70a805b02 in ha_innobase::delete_table(char const*) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/handler/ha_innodb.cc:13164
                #9 0x55e709cafe10 in hton_drop_table /home/buildbot/buildbot/build/mariadb-10.6.1/sql/handler.cc:564
                #10 0x55e709af79a7 in THD::rm_temporary_table(handlerton*, char const*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/temporary_tables.cc:703
                #11 0x55e709afca4d in THD::free_tmp_table_share(TMP_TABLE_SHARE*, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/temporary_tables.cc:1462
                #12 0x55e709af7598 in THD::drop_temporary_table(TABLE*, bool*, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/temporary_tables.cc:669
                #13 0x55e70970f9bb in mysql_rm_table_no_locks(THD*, TABLE_LIST*, bool, bool, bool, bool, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:2365
                #14 0x55e70970e5d8 in mysql_rm_table(THD*, TABLE_LIST*, bool, bool, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:2138
                #15 0x55e7094a2468 in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:4943
                #16 0x55e7094b7021 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:8018
                #17 0x55e70a02e117 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1906
                #18 0x55e70a02b338 in Query_log_event::do_apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1581
                #19 0x55e709229afe in Log_event::apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event.h:1498
                #20 0x55e70920b42c in apply_event_and_update_pos_apply /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:3850
                #21 0x55e70920c1b8 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:4036
                #22 0x55e709a1515d in rpt_handle_event /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:62
                #23 0x55e709a1cddd in handle_rpl_parallel_thread /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:1378
                #24 0x55e70a5e3f0c in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.6.1/storage/perfschema/pfs.cc:2201
                #25 0x7f20eaf13608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
             
            previously allocated by thread T45 here:
                #0 0x7f20eb1a9bc8 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10dbc8)
                #1 0x55e70a83fb68 in ut_allocator<unsigned char, true>::allocate(unsigned long, unsigned char const*, unsigned int, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/include/ut0new.h:375
                #2 0x55e70aa0d939 in mem_heap_create_block_func(mem_block_info_t*, unsigned long, char const*, unsigned int, unsigned long) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/mem/mem0mem.cc:277
                #3 0x55e70aa0e289 in mem_heap_add_block(mem_block_info_t*, unsigned long) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/mem/mem0mem.cc:378
                #4 0x55e70aea9265 in mem_heap_alloc /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/include/mem0mem.ic:193
                #5 0x55e70aea9079 in mem_heap_zalloc /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/include/mem0mem.ic:162
                #6 0x55e70aeab6e5 in dict_mem_table_create(char const*, fil_space_t*, unsigned long, unsigned long, unsigned long, unsigned long) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/dict/dict0mem.cc:152
                #7 0x55e70a837ba3 in create_table_info_t::create_table_def() (/home/buildbot/buildbot/build/mariadb-10.6.1/sql/mariadbd+0x2e40ba3)
                #8 0x55e70a801472 in create_table_info_t::create_table(bool) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/handler/ha_innodb.cc:12219
                #9 0x55e70a83ce58 in ha_innobase::create(char const*, TABLE*, HA_CREATE_INFO*, bool, trx_t*) (/home/buildbot/buildbot/build/mariadb-10.6.1/sql/mariadbd+0x2e45e58)
                #10 0x55e70a803f4b in ha_innobase::create(char const*, TABLE*, HA_CREATE_INFO*) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/handler/ha_innodb.cc:12815
                #11 0x55e709cd4839 in handler::ha_create(char const*, TABLE*, HA_CREATE_INFO*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/handler.cc:5101
                #12 0x55e709cd8f48 in ha_create_table(THD*, char const*, char const*, char const*, HA_CREATE_INFO*, st_mysql_const_unsigned_lex_string*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/handler.cc:5566
                #13 0x55e7097272fc in create_table_impl /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:5392
                #14 0x55e709727c88 in mysql_create_table_no_lock(THD*, st_mysql_const_lex_string const*, st_mysql_const_lex_string const*, Table_specification_st*, Alter_info*, bool*, int, TABLE_LIST*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:5476
                #15 0x55e709728902 in mysql_create_table(THD*, TABLE_LIST*, Table_specification_st*, Alter_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:5580
                #16 0x55e70975a08f in Sql_cmd_create_table_like::execute(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:12288
                #17 0x55e7094a9823 in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:5986
                #18 0x55e7094b7021 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:8018
                #19 0x55e70a02e117 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1906
                #20 0x55e70a02b338 in Query_log_event::do_apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1581
                #21 0x55e709229afe in Log_event::apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event.h:1498
                #22 0x55e70920b42c in apply_event_and_update_pos_apply /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:3850
                #23 0x55e70920c1b8 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:4036
                #24 0x55e709a1515d in rpt_handle_event /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:62
                #25 0x55e709a1cddd in handle_rpl_parallel_thread /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:1378
                #26 0x55e70a5e3f0c in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.6.1/storage/perfschema/pfs.cc:2201
                #27 0x7f20eaf13608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
            

            marko Marko Mäkelä added a comment - Another recent failure : 10.6 11597e02f3bcca6d3f7b7992c0fa63e45e614b1c rpl.rpl_parallel_temptable 'innodb,stmt' w2 [ fail ] Test ended at 2021-05-04 16:31:36   CURRENT_TEST: rpl.rpl_parallel_temptable mysqltest: In included file "./include/stop_slave.inc": included from /home/buildbot/buildbot/build/mariadb-10.6.1/mysql-test/suite/rpl/t/rpl_parallel_temptable.test at line 257: At line 82: query 'STOP SLAVE' failed: 2013: Lost connection to server during query … 2021-05-04 16:31:32 30 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16060',replication starts at GTID position '1-1-31,0-1-9,2-1-17' 2021-05-04 16:31:32 31 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000004' at position 1354, relay log './slave-relay-bin.000001' position: 4; GTID position '0-1-9,1-1-31,2-1-17' ================================================================= ==60687==ERROR: AddressSanitizer: heap-use-after-free on address 0x618000078390 at pc 0x55e70a84bf68 bp 0x7f20db357fe0 sp 0x7f20db357fd0 WRITE of size 8 at 0x618000078390 thread T44 #0 0x55e70a84bf67 in std::__atomic_base<long>::store(long, std::memory_order) /usr/include/c++/9/bits/atomic_base.h:397 #1 0x55e70a84bf67 in Atomic_relaxed<long>::operator=(long) /home/buildbot/buildbot/build/mariadb-10.6.1/include/my_atomic_wrapper.h:48 #2 0x55e70acf42b4 in trx_t::commit_tables() /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/trx/trx0trx.cc:1229 #3 0x55e70acf513b in trx_t::commit_in_memory(mtr_t const*) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/trx/trx0trx.cc:1323 #4 0x55e70aceccbd in trx_t::commit_low(mtr_t*) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/trx/trx0trx.cc:1493 #5 0x55e70acecdf2 in trx_t::commit() /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/trx/trx0trx.cc:1507 #6 0x55e70aced54a in trx_commit_for_mysql(trx_t*) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/trx/trx0trx.cc:1624 2021-05-04 16:31:33 31 [Note] Error reading relay log event: slave SQL thread was killed #7 0x55e70a7cff89 in innobase_commit_low(trx_t*) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/handler/ha_innodb.cc:3874 #8 0x55e70a7d06c4 in innobase_commit_ordered_2 /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/handler/ha_innodb.cc:3980 #9 0x55e70a7d119c in innobase_commit /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/handler/ha_innodb.cc:4084 #10 0x55e709cb702a in commit_one_phase_2 /home/buildbot/buildbot/build/mariadb-10.6.1/sql/handler.cc:1944 #11 0x55e709cb6d35 in ha_commit_one_phase(THD*, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/handler.cc:1923 #12 0x55e709cb5085 in ha_commit_trans(THD*, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/handler.cc:1717 #13 0x55e709911d15 in trans_commit(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/transaction.cc:266 #14 0x55e7094a6f6e in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:5613 #15 0x55e7094b7021 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:8018 #16 0x55e70a02e117 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1906 #17 0x55e70a02b338 in Query_log_event::do_apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1581 #18 0x55e709229afe in Log_event::apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event.h:1498 #19 0x55e70920b42c in apply_event_and_update_pos_apply /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:3850 #20 0x55e70920c1b8 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:4036 #21 0x55e709a1515d in rpt_handle_event /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:62 #22 0x55e709a1cddd in handle_rpl_parallel_thread /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:1378 #23 0x55e70a5e3f0c in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.6.1/storage/perfschema/pfs.cc:2201 #24 0x7f20eaf13608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608) #25 0x7f20eaae7292 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x122292)   0x618000078390 is located 784 bytes inside of 816-byte region [0x618000078080,0x6180000783b0) freed by thread T42 here: #0 0x7f20eb1a97cf in __interceptor_free (/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf) #1 0x55e70a84007d in ut_allocator<unsigned char, true>::deallocate(unsigned char*, unsigned long) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/include/ut0new.h:424 #2 0x55e70aa0e695 in mem_heap_block_free(mem_block_info_t*, mem_block_info_t*) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/mem/mem0mem.cc:416 #3 0x55e70aea95d8 in mem_heap_free /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/include/mem0mem.ic:419 #4 0x55e70aeac804 in dict_mem_table_free(dict_table_t*) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/dict/dict0mem.cc:246 #5 0x55e70ae6e6be in dict_sys_t::remove(dict_table_t*, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/dict/dict0dict.cc:2012 #6 0x55e70ab91864 in row_drop_table_for_mysql(char const*, trx_t*, enum_sql_command, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/row/row0mysql.cc:3286 #7 0x55e70a83d791 in ha_innobase::delete_table(char const*, enum_sql_command) (/home/buildbot/buildbot/build/mariadb-10.6.1/sql/mariadbd+0x2e46791) #8 0x55e70a805b02 in ha_innobase::delete_table(char const*) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/handler/ha_innodb.cc:13164 #9 0x55e709cafe10 in hton_drop_table /home/buildbot/buildbot/build/mariadb-10.6.1/sql/handler.cc:564 #10 0x55e709af79a7 in THD::rm_temporary_table(handlerton*, char const*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/temporary_tables.cc:703 #11 0x55e709afca4d in THD::free_tmp_table_share(TMP_TABLE_SHARE*, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/temporary_tables.cc:1462 #12 0x55e709af7598 in THD::drop_temporary_table(TABLE*, bool*, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/temporary_tables.cc:669 #13 0x55e70970f9bb in mysql_rm_table_no_locks(THD*, TABLE_LIST*, bool, bool, bool, bool, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:2365 #14 0x55e70970e5d8 in mysql_rm_table(THD*, TABLE_LIST*, bool, bool, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:2138 #15 0x55e7094a2468 in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:4943 #16 0x55e7094b7021 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:8018 #17 0x55e70a02e117 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1906 #18 0x55e70a02b338 in Query_log_event::do_apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1581 #19 0x55e709229afe in Log_event::apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event.h:1498 #20 0x55e70920b42c in apply_event_and_update_pos_apply /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:3850 #21 0x55e70920c1b8 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:4036 #22 0x55e709a1515d in rpt_handle_event /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:62 #23 0x55e709a1cddd in handle_rpl_parallel_thread /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:1378 #24 0x55e70a5e3f0c in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.6.1/storage/perfschema/pfs.cc:2201 #25 0x7f20eaf13608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)   previously allocated by thread T45 here: #0 0x7f20eb1a9bc8 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10dbc8) #1 0x55e70a83fb68 in ut_allocator<unsigned char, true>::allocate(unsigned long, unsigned char const*, unsigned int, bool, bool) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/include/ut0new.h:375 #2 0x55e70aa0d939 in mem_heap_create_block_func(mem_block_info_t*, unsigned long, char const*, unsigned int, unsigned long) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/mem/mem0mem.cc:277 #3 0x55e70aa0e289 in mem_heap_add_block(mem_block_info_t*, unsigned long) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/mem/mem0mem.cc:378 #4 0x55e70aea9265 in mem_heap_alloc /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/include/mem0mem.ic:193 #5 0x55e70aea9079 in mem_heap_zalloc /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/include/mem0mem.ic:162 #6 0x55e70aeab6e5 in dict_mem_table_create(char const*, fil_space_t*, unsigned long, unsigned long, unsigned long, unsigned long) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/dict/dict0mem.cc:152 #7 0x55e70a837ba3 in create_table_info_t::create_table_def() (/home/buildbot/buildbot/build/mariadb-10.6.1/sql/mariadbd+0x2e40ba3) #8 0x55e70a801472 in create_table_info_t::create_table(bool) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/handler/ha_innodb.cc:12219 #9 0x55e70a83ce58 in ha_innobase::create(char const*, TABLE*, HA_CREATE_INFO*, bool, trx_t*) (/home/buildbot/buildbot/build/mariadb-10.6.1/sql/mariadbd+0x2e45e58) #10 0x55e70a803f4b in ha_innobase::create(char const*, TABLE*, HA_CREATE_INFO*) /home/buildbot/buildbot/build/mariadb-10.6.1/storage/innobase/handler/ha_innodb.cc:12815 #11 0x55e709cd4839 in handler::ha_create(char const*, TABLE*, HA_CREATE_INFO*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/handler.cc:5101 #12 0x55e709cd8f48 in ha_create_table(THD*, char const*, char const*, char const*, HA_CREATE_INFO*, st_mysql_const_unsigned_lex_string*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/handler.cc:5566 #13 0x55e7097272fc in create_table_impl /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:5392 #14 0x55e709727c88 in mysql_create_table_no_lock(THD*, st_mysql_const_lex_string const*, st_mysql_const_lex_string const*, Table_specification_st*, Alter_info*, bool*, int, TABLE_LIST*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:5476 #15 0x55e709728902 in mysql_create_table(THD*, TABLE_LIST*, Table_specification_st*, Alter_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:5580 #16 0x55e70975a08f in Sql_cmd_create_table_like::execute(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_table.cc:12288 #17 0x55e7094a9823 in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:5986 #18 0x55e7094b7021 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/sql_parse.cc:8018 #19 0x55e70a02e117 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1906 #20 0x55e70a02b338 in Query_log_event::do_apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event_server.cc:1581 #21 0x55e709229afe in Log_event::apply_event(rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/log_event.h:1498 #22 0x55e70920b42c in apply_event_and_update_pos_apply /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:3850 #23 0x55e70920c1b8 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /home/buildbot/buildbot/build/mariadb-10.6.1/sql/slave.cc:4036 #24 0x55e709a1515d in rpt_handle_event /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:62 #25 0x55e709a1cddd in handle_rpl_parallel_thread /home/buildbot/buildbot/build/mariadb-10.6.1/sql/rpl_parallel.cc:1378 #26 0x55e70a5e3f0c in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.6.1/storage/perfschema/pfs.cc:2201 #27 0x7f20eaf13608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
            alice Alice Sherepa added a comment - https://buildbot.askmonty.org/buildbot/builders/kvm-asan/builds/6715/steps/mtr_nm/logs/stdio

            I think that this race condition needs to be fixed in the oldest affected release series (10.2). I believe that the replication of InnoDB temporary tables is more broken starting with 10.2.2. In earlier releases, InnoDB treated temporary tables as persistent ones, and there might not have been any problem. In any case, 10.2 is currently the oldest supported release series.

            marko Marko Mäkelä added a comment - I think that this race condition needs to be fixed in the oldest affected release series (10.2). I believe that the replication of InnoDB temporary tables is more broken starting with 10.2.2. In earlier releases, InnoDB treated temporary tables as persistent ones, and there might not have been any problem. In any case, 10.2 is currently the oldest supported release series.
            alice Alice Sherepa added a comment -

            ./mtr rpl.rpl_xa_survive_disconnect_mixed_engines --mysqld=--slave_parallel_threads=2
            

            220124 16:15:12 [ERROR] mysqld got signal 6 ;
             
            ??:0(__asan_on_error)[0x7fc7ce73a363]
            ??:0(__asan_report_store8)[0x7fc7ce73b74e]
            bits/atomic_base.h:397(std::__atomic_base<long>::store(long, std::memory_order))[0x5562b862f374]
            include/my_atomic_wrapper.h:49(Atomic_relaxed<long>::operator=(long))[0x5562b862b6fa]
            trx/trx0trx.cc:1177(trx_t::commit_tables())[0x5562b8ada841]
            trx/trx0trx.cc:1279(trx_t::commit_in_memory(mtr_t const*))[0x5562b8ad24b9]
            trx/trx0trx.cc:1446(trx_t::commit_persist())[0x5562b8ad325d]
            trx/trx0trx.cc:1460(trx_t::commit())[0x5562b8ad33aa]
            trx/trx0trx.cc:1576(trx_commit_for_mysql(trx_t*))[0x5562b8ad3bbc]
            handler/ha_innodb.cc:4257(innobase_commit_low(trx_t*))[0x5562b85ab8ba]
            handler/ha_innodb.cc:16988(innobase_commit_by_xid(handlerton*, xid_t*))[0x5562b85f6ad7]
            sql/handler.cc:2253(xacommit_handlerton(THD*, st_plugin_int**, void*))[0x5562b7b2cfcd]
            sql/sql_plugin.cc:2509(plugin_foreach_with_mask(THD*, char (*)(THD*, st_plugin_int**, void*), int, unsigned int, void*))[0x5562b72df2bd]
            sql/handler.cc:2289(ha_commit_or_rollback_by_xid(xid_t*, bool))[0x5562b7b2d2db]
            sql/xa.cc:639(trans_xa_commit(THD*))[0x5562b7987030]
            sql/sql_parse.cc:5854(mysql_execute_command(THD*, bool))[0x5562b726d83c]
            sql/sql_parse.cc:8027(mysql_parse(THD*, char*, unsigned int, Parser_state*))[0x5562b727bcac]
            sql/log_event_server.cc:2151(Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int))[0x5562b7ef9ae0]
            sql/log_event_server.cc:1613(Query_log_event::do_apply_event(rpl_group_info*))[0x5562b7ef5375]
            sql/log_event.h:1566(Log_event::apply_event(rpl_group_info*))[0x5562b6fdf885]
            sql/slave.cc:3881(apply_event_and_update_pos_apply(Log_event*, THD*, rpl_group_info*, int))[0x5562b6fc0c22]
            sql/slave.cc:4078(apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*))[0x5562b6fc1b38]
            sql/rpl_parallel.cc:62(rpt_handle_event(rpl_parallel_thread::queued_event*, rpl_parallel_thread*))[0x5562b7883d93]
            sql/rpl_parallel.cc:1409(handle_rpl_parallel_thread)[0x5562b788c06f]
            perfschema/pfs.cc:2203(pfs_spawn_thread)[0x5562b83b6cdf]
            nptl/pthread_create.c:478(start_thread)[0x7fc7ce235609]
            ??:0(clone)[0x7fc7cde08293]
             
            Query (0x6120000c4453): XA COMMIT X'78615f747278',X'',1
            

            alice Alice Sherepa added a comment - ./mtr rpl.rpl_xa_survive_disconnect_mixed_engines --mysqld=--slave_parallel_threads=2 220124 16:15:12 [ERROR] mysqld got signal 6 ;   ??:0(__asan_on_error)[0x7fc7ce73a363] ??:0(__asan_report_store8)[0x7fc7ce73b74e] bits/atomic_base.h:397(std::__atomic_base<long>::store(long, std::memory_order))[0x5562b862f374] include/my_atomic_wrapper.h:49(Atomic_relaxed<long>::operator=(long))[0x5562b862b6fa] trx/trx0trx.cc:1177(trx_t::commit_tables())[0x5562b8ada841] trx/trx0trx.cc:1279(trx_t::commit_in_memory(mtr_t const*))[0x5562b8ad24b9] trx/trx0trx.cc:1446(trx_t::commit_persist())[0x5562b8ad325d] trx/trx0trx.cc:1460(trx_t::commit())[0x5562b8ad33aa] trx/trx0trx.cc:1576(trx_commit_for_mysql(trx_t*))[0x5562b8ad3bbc] handler/ha_innodb.cc:4257(innobase_commit_low(trx_t*))[0x5562b85ab8ba] handler/ha_innodb.cc:16988(innobase_commit_by_xid(handlerton*, xid_t*))[0x5562b85f6ad7] sql/handler.cc:2253(xacommit_handlerton(THD*, st_plugin_int**, void*))[0x5562b7b2cfcd] sql/sql_plugin.cc:2509(plugin_foreach_with_mask(THD*, char (*)(THD*, st_plugin_int**, void*), int, unsigned int, void*))[0x5562b72df2bd] sql/handler.cc:2289(ha_commit_or_rollback_by_xid(xid_t*, bool))[0x5562b7b2d2db] sql/xa.cc:639(trans_xa_commit(THD*))[0x5562b7987030] sql/sql_parse.cc:5854(mysql_execute_command(THD*, bool))[0x5562b726d83c] sql/sql_parse.cc:8027(mysql_parse(THD*, char*, unsigned int, Parser_state*))[0x5562b727bcac] sql/log_event_server.cc:2151(Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int))[0x5562b7ef9ae0] sql/log_event_server.cc:1613(Query_log_event::do_apply_event(rpl_group_info*))[0x5562b7ef5375] sql/log_event.h:1566(Log_event::apply_event(rpl_group_info*))[0x5562b6fdf885] sql/slave.cc:3881(apply_event_and_update_pos_apply(Log_event*, THD*, rpl_group_info*, int))[0x5562b6fc0c22] sql/slave.cc:4078(apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*))[0x5562b6fc1b38] sql/rpl_parallel.cc:62(rpt_handle_event(rpl_parallel_thread::queued_event*, rpl_parallel_thread*))[0x5562b7883d93] sql/rpl_parallel.cc:1409(handle_rpl_parallel_thread)[0x5562b788c06f] perfschema/pfs.cc:2203(pfs_spawn_thread)[0x5562b83b6cdf] nptl/pthread_create.c:478(start_thread)[0x7fc7ce235609] ??:0(clone)[0x7fc7cde08293]   Query (0x6120000c4453): XA COMMIT X'78615f747278',X'',1
            Elkin Andrei Elkin added a comment -

            marko, sending the ticket to you first for examining the top of the stack.

            Elkin Andrei Elkin added a comment - marko , sending the ticket to you first for examining the top of the stack.

            Elkin, the crash stack trace that alice posted looks similar to what I posted in my very first comment to this issue, along with some analysis.

            It would be much more useful to see the AddressSanitizer output itself than the crash that might have been the (desired) result of ASAN_OPTIONS=abort_on_error=1.

            marko Marko Mäkelä added a comment - Elkin , the crash stack trace that alice posted looks similar to what I posted in my very first comment to this issue, along with some analysis. It would be much more useful to see the AddressSanitizer output itself than the crash that might have been the (desired) result of ASAN_OPTIONS=abort_on_error=1 .

            My first comment to this bug (2021-05-05) includes full root cause analysis.

            marko Marko Mäkelä added a comment - My first comment to this bug (2021-05-05) includes full root cause analysis.

            Duplicate of MDEV-10356

            monty Michael Widenius added a comment - Duplicate of MDEV-10356

            I am able to reproduce this bug on today’s 10.6, running the following on /dev/shm:

            ./mtr --parallel=80 --big-test --repeat=1000 rpl.rpl_parallel_temptable{,,,,,,,}{,,,,,,,,,}
            

            rpl.rpl_parallel_temptable 'innodb,stmt' w28 [ 20 fail ]
                    Test ended at 2023-09-06 16:01:47
             
            CURRENT_TEST: rpl.rpl_parallel_temptable
            mysqltest: In included file "./include/stop_slave.inc": 
            included from /mariadb/10.6m/mysql-test/suite/rpl/t/rpl_parallel_temptable.test at line 264:
            At line 82: query 'STOP SLAVE' failed: <Unknown> (2013): Lost connection to server during query
            

            marko Marko Mäkelä added a comment - I am able to reproduce this bug on today’s 10.6, running the following on /dev/shm : ./mtr --parallel=80 --big-test --repeat=1000 rpl.rpl_parallel_temptable{,,,,,,,}{,,,,,,,,,} rpl.rpl_parallel_temptable 'innodb,stmt' w28 [ 20 fail ] Test ended at 2023-09-06 16:01:47   CURRENT_TEST: rpl.rpl_parallel_temptable mysqltest: In included file "./include/stop_slave.inc": included from /mariadb/10.6m/mysql-test/suite/rpl/t/rpl_parallel_temptable.test at line 264: At line 82: query 'STOP SLAVE' failed: <Unknown> (2013): Lost connection to server during query

            Here is another attempt, this time with some more details:

            ASAN_OPTIONS=abort_on_error=1:fast_unwind_on_malloc=0 ./mtr --parallel=80 --repeat=1000 rpl.rpl_parallel_temptable{,,,,,,,}{,,,,,,,,,} --skip-core-file --skip-stack-trace
            

            10.6 de5dba9ebe6a40b0344ce87b7ee6fc613f38c44f

            rpl.rpl_parallel_temptable 'innodb,stmt' w29 [ fail ]
                    Test ended at 2023-09-06 16:39:53
             
            CURRENT_TEST: rpl.rpl_parallel_temptable
            mysqltest: In included file "./include/stop_slave.inc": 
            included from /mariadb/10.6m/mysql-test/suite/rpl/t/rpl_parallel_temptable.test at line 244:
            At line 82: query 'STOP SLAVE' failed: <Unknown> (2013): Lost connection to server during query
            …
            WRITE of size 8 at 0x618000062798 thread T33
            2023-09-06 16:39:49 24 [Note] Error reading relay log event: slave SQL thread was killed
                #0 0x55b880437374 in std::__atomic_base<long>::store(long, std::memory_order) /usr/include/c++/13/bits/atomic_base.h:481
                #1 0x55b880437374 in Atomic_relaxed<long>::store(long, std::memory_order) /mariadb/10.6m/include/my_atomic_wrapper.h:47
                #2 0x55b880437374 in Atomic_relaxed<long>::operator=(long) /mariadb/10.6m/include/my_atomic_wrapper.h:49
                #3 0x55b880437374 in trx_t::commit_tables() /mariadb/10.6m/storage/innobase/trx/trx0trx.cc:1268
                #4 0x55b8804310cc in trx_t::commit_in_memory(mtr_t const*) /mariadb/10.6m/storage/innobase/trx/trx0trx.cc:1421
                #5 0x55b8804310cc in trx_t::commit_low(mtr_t*) /mariadb/10.6m/storage/innobase/trx/trx0trx.cc:1567
                #6 0x55b880432b58 in trx_t::commit_persist() /mariadb/10.6m/storage/innobase/trx/trx0trx.cc:1581
                #7 0x55b880432cd0 in trx_t::commit() /mariadb/10.6m/storage/innobase/trx/trx0trx.cc:1590
                #8 0x55b88043343f in trx_commit_for_mysql(trx_t*) /mariadb/10.6m/storage/innobase/trx/trx0trx.cc:1707
                #9 0x55b87fdb6cec in innobase_commit_low(trx_t*) /mariadb/10.6m/storage/innobase/handler/ha_innodb.cc:4390
                #10 0x55b87fdb6dc9 in innobase_commit_ordered_2 /mariadb/10.6m/storage/innobase/handler/ha_innodb.cc:4494
                #11 0x55b87fdb7370 in innobase_commit /mariadb/10.6m/storage/innobase/handler/ha_innodb.cc:4595
                #12 0x55b87eef1cb9 in commit_one_phase_2 /mariadb/10.6m/sql/handler.cc:2100
                #13 0x55b87eef1fc9 in ha_commit_one_phase(THD*, bool) /mariadb/10.6m/sql/handler.cc:2053
                #14 0x55b87eef3d43 in ha_commit_trans(THD*, bool) /mariadb/10.6m/sql/handler.cc:1847
                #15 0x55b87f883a21 in trans_commit(THD*) /mariadb/10.6m/sql/transaction.cc:266
                #16 0x55b87f4a763e in mysql_execute_command(THD*, bool) /mariadb/10.6m/sql/sql_parse.cc:5646
                #17 0x55b87f4ab275 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /mariadb/10.6m/sql/sql_parse.cc:8048
                #18 0x55b87ef7cc00 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /mariadb/10.6m/sql/log_event_server.cc:1927
                #19 0x55b87ef7dc2b in Query_log_event::do_apply_event(rpl_group_info*) /mariadb/10.6m/sql/log_event_server.cc:1605
                #20 0x55b87f1b8123 in Log_event::apply_event(rpl_group_info*) /mariadb/10.6m/sql/log_event.h:1499
                #21 0x55b87f199115 in apply_event_and_update_pos_apply /mariadb/10.6m/sql/slave.cc:3894
                #22 0x55b87f1a5655 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /mariadb/10.6m/sql/slave.cc:4080
                #23 0x55b87f98fa93 in rpt_handle_event /mariadb/10.6m/sql/rpl_parallel.cc:64
                #24 0x55b87f99817e in handle_rpl_parallel_thread /mariadb/10.6m/sql/rpl_parallel.cc:1481
                #25 0x55b87fc2b381 in pfs_spawn_thread /mariadb/10.6m/storage/perfschema/pfs.cc:2201
                #26 0x7f61cbca63eb in start_thread nptl/pthread_create.c:444
                #27 0x7f61cbd26a1b in clone3 ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
             
            0x618000062798 is located 792 bytes inside of 824-byte region [0x618000062480,0x6180000627b8)
            freed by thread T34 here:
                #0 0x7f61ccad7288 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52
                #1 0x55b87fe07cf9 in ut_allocator<unsigned char, true>::deallocate(unsigned char*, unsigned long) /mariadb/10.6m/storage/innobase/include/ut0new.h:424
                #2 0x55b8801f44fd in mem_heap_block_free(mem_block_info_t*, mem_block_info_t*) /mariadb/10.6m/storage/innobase/mem/mem0mem.cc:416
                #3 0x55b87ffd5b7e in mem_heap_free /mariadb/10.6m/storage/innobase/include/mem0mem.inl:419
                #4 0x55b87ffdd33e in dict_mem_table_free(dict_table_t*) /mariadb/10.6m/storage/innobase/dict/dict0mem.cc:234
                #5 0x55b87fe5e41d in dict_sys_t::remove(dict_table_t*, bool, bool) /mariadb/10.6m/storage/innobase/dict/dict0dict.cc:1943
                #6 0x55b87fdeb95a in ha_innobase::delete_table(char const*) /mariadb/10.6m/storage/innobase/handler/ha_innodb.cc:13513
                #7 0x55b87eef6e6d in hton_drop_table /mariadb/10.6m/sql/handler.cc:573
                #8 0x55b87fa5236a in THD::rm_temporary_table(handlerton*, char const*) /mariadb/10.6m/sql/temporary_tables.cc:705
                #9 0x55b87fa53138 in THD::free_tmp_table_share(TMP_TABLE_SHARE*, bool) /mariadb/10.6m/sql/temporary_tables.cc:1468
                #10 0x55b87fa56580 in THD::drop_temporary_table(TABLE*, bool*, bool) /mariadb/10.6m/sql/temporary_tables.cc:673
                #11 0x55b87f6bd60a in mysql_rm_table_no_locks(THD*, TABLE_LIST*, st_mysql_const_lex_string const*, st_ddl_log_state*, bool, bool, bool, bool, bool, bool) /mariadb/10.6m/sql/sql_table.cc:1295
                #12 0x55b87f6c13fd in mysql_rm_table(THD*, TABLE_LIST*, bool, bool, bool, bool) /mariadb/10.6m/sql/sql_table.cc:1043
                #13 0x55b87f4a38d6 in mysql_execute_command(THD*, bool) /mariadb/10.6m/sql/sql_parse.cc:4976
                #14 0x55b87f4ab275 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /mariadb/10.6m/sql/sql_parse.cc:8048
                #15 0x55b87ef7cc00 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /mariadb/10.6m/sql/log_event_server.cc:1927
                #16 0x55b87ef7dc2b in Query_log_event::do_apply_event(rpl_group_info*) /mariadb/10.6m/sql/log_event_server.cc:1605
                #17 0x55b87f1b8123 in Log_event::apply_event(rpl_group_info*) /mariadb/10.6m/sql/log_event.h:1499
                #18 0x55b87f199115 in apply_event_and_update_pos_apply /mariadb/10.6m/sql/slave.cc:3894
                #19 0x55b87f1a5655 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /mariadb/10.6m/sql/slave.cc:4080
                #20 0x55b87f98fa93 in rpt_handle_event /mariadb/10.6m/sql/rpl_parallel.cc:64
                #21 0x55b87f99817e in handle_rpl_parallel_thread /mariadb/10.6m/sql/rpl_parallel.cc:1481
                #22 0x55b87fc2b381 in pfs_spawn_thread /mariadb/10.6m/storage/perfschema/pfs.cc:2201
                #23 0x7f61cbca63eb in start_thread nptl/pthread_create.c:444
                #24 0x7f61cbd26a1b in clone3 ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
             
            previously allocated by thread T33 here:
                #0 0x7f61ccad85bf in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
                #1 0x55b87fe08ccf in ut_allocator<unsigned char, true>::allocate(unsigned long, unsigned char const*, unsigned int, bool, bool) /mariadb/10.6m/storage/innobase/include/ut0new.h:375
                #2 0x55b8801f4871 in mem_heap_create_block_func(mem_block_info_t*, unsigned long, char const*, unsigned int, unsigned long) /mariadb/10.6m/storage/innobase/mem/mem0mem.cc:277
                #3 0x55b8801f4f23 in mem_heap_add_block(mem_block_info_t*, unsigned long) /mariadb/10.6m/storage/innobase/mem/mem0mem.cc:378
                #4 0x55b87ffd5822 in mem_heap_alloc /mariadb/10.6m/storage/innobase/include/mem0mem.inl:193
                #5 0x55b87ffd5966 in mem_heap_zalloc /mariadb/10.6m/storage/innobase/include/mem0mem.inl:162
                #6 0x55b87ffdaaf0 in dict_table_t::create(st_::span<char const> const&, fil_space_t*, unsigned long, unsigned long, unsigned long, unsigned long) /mariadb/10.6m/storage/innobase/dict/dict0mem.cc:148
                #7 0x55b87fe0495d in create_table_info_t::create_table_def() /mariadb/10.6m/storage/innobase/handler/ha_innodb.cc:10726
                #8 0x55b87fdf1566 in create_table_info_t::create_table(bool) /mariadb/10.6m/storage/innobase/handler/ha_innodb.cc:12789
                #9 0x55b87fdf2b40 in ha_innobase::create(char const*, TABLE*, HA_CREATE_INFO*, bool, trx_t*) /mariadb/10.6m/storage/innobase/handler/ha_innodb.cc:13296
                #10 0x55b87fdf2fbf in ha_innobase::create(char const*, TABLE*, HA_CREATE_INFO*) /mariadb/10.6m/storage/innobase/handler/ha_innodb.cc:13340
                #11 0x55b87ef06fb8 in handler::ha_create(char const*, TABLE*, HA_CREATE_INFO*) /mariadb/10.6m/sql/handler.cc:5515
                #12 0x55b87ef08d19 in ha_create_table(THD*, char const*, char const*, char const*, HA_CREATE_INFO*, st_mysql_const_unsigned_lex_string*, bool) /mariadb/10.6m/sql/handler.cc:5983
                #13 0x55b87f6e0314 in create_table_impl /mariadb/10.6m/sql/sql_table.cc:4422
                #14 0x55b87f6e1891 in mysql_create_table_no_lock(THD*, st_ddl_log_state*, st_ddl_log_state*, Table_specification_st*, Alter_info*, bool*, int, TABLE_LIST*) /mariadb/10.6m/sql/sql_table.cc:4521
                #15 0x55b87f6e249a in mysql_create_table(THD*, TABLE_LIST*, Table_specification_st*, Alter_info*) /mariadb/10.6m/sql/sql_table.cc:4687
                #16 0x55b87f6f2905 in Sql_cmd_create_table_like::execute(THD*) /mariadb/10.6m/sql/sql_table.cc:11972
                #17 0x55b87f4a97fb in mysql_execute_command(THD*, bool) /mariadb/10.6m/sql/sql_parse.cc:6024
                #18 0x55b87f4ab275 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /mariadb/10.6m/sql/sql_parse.cc:8048
                #19 0x55b87ef7cc00 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /mariadb/10.6m/sql/log_event_server.cc:1927
                #20 0x55b87ef7dc2b in Query_log_event::do_apply_event(rpl_group_info*) /mariadb/10.6m/sql/log_event_server.cc:1605
                #21 0x55b87f1b8123 in Log_event::apply_event(rpl_group_info*) /mariadb/10.6m/sql/log_event.h:1499
                #22 0x55b87f199115 in apply_event_and_update_pos_apply /mariadb/10.6m/sql/slave.cc:3894
                #23 0x55b87f1a5655 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /mariadb/10.6m/sql/slave.cc:4080
                #24 0x55b87f98fa93 in rpt_handle_event /mariadb/10.6m/sql/rpl_parallel.cc:64
                #25 0x55b87f99817e in handle_rpl_parallel_thread /mariadb/10.6m/sql/rpl_parallel.cc:1481
                #26 0x55b87fc2b381 in pfs_spawn_thread /mariadb/10.6m/storage/perfschema/pfs.cc:2201
                #27 0x7f61cbca63eb in start_thread nptl/pthread_create.c:444
            

            The output mentions two threads: T33, which did CREATE TEMPORARY TABLE and crashes on COMMIT, and T34, which executed DROP TEMPORARY TABLE before all transactions that used the temporary table were committed. This is similar to my first comment here, except that in that comment, the CREATE, COMMIT, and DROP were all executed in different threads.

            marko Marko Mäkelä added a comment - Here is another attempt, this time with some more details: ASAN_OPTIONS=abort_on_error=1:fast_unwind_on_malloc=0 . /mtr --parallel=80 --repeat=1000 rpl.rpl_parallel_temptable{,,,,,,,}{,,,,,,,,,} --skip-core- file --skip-stack-trace 10.6 de5dba9ebe6a40b0344ce87b7ee6fc613f38c44f rpl.rpl_parallel_temptable 'innodb,stmt' w29 [ fail ] Test ended at 2023-09-06 16:39:53   CURRENT_TEST: rpl.rpl_parallel_temptable mysqltest: In included file "./include/stop_slave.inc": included from /mariadb/10.6m/mysql-test/suite/rpl/t/rpl_parallel_temptable.test at line 244: At line 82: query 'STOP SLAVE' failed: <Unknown> (2013): Lost connection to server during query … WRITE of size 8 at 0x618000062798 thread T33 2023-09-06 16:39:49 24 [Note] Error reading relay log event: slave SQL thread was killed #0 0x55b880437374 in std::__atomic_base<long>::store(long, std::memory_order) /usr/include/c++/13/bits/atomic_base.h:481 #1 0x55b880437374 in Atomic_relaxed<long>::store(long, std::memory_order) /mariadb/10.6m/include/my_atomic_wrapper.h:47 #2 0x55b880437374 in Atomic_relaxed<long>::operator=(long) /mariadb/10.6m/include/my_atomic_wrapper.h:49 #3 0x55b880437374 in trx_t::commit_tables() /mariadb/10.6m/storage/innobase/trx/trx0trx.cc:1268 #4 0x55b8804310cc in trx_t::commit_in_memory(mtr_t const*) /mariadb/10.6m/storage/innobase/trx/trx0trx.cc:1421 #5 0x55b8804310cc in trx_t::commit_low(mtr_t*) /mariadb/10.6m/storage/innobase/trx/trx0trx.cc:1567 #6 0x55b880432b58 in trx_t::commit_persist() /mariadb/10.6m/storage/innobase/trx/trx0trx.cc:1581 #7 0x55b880432cd0 in trx_t::commit() /mariadb/10.6m/storage/innobase/trx/trx0trx.cc:1590 #8 0x55b88043343f in trx_commit_for_mysql(trx_t*) /mariadb/10.6m/storage/innobase/trx/trx0trx.cc:1707 #9 0x55b87fdb6cec in innobase_commit_low(trx_t*) /mariadb/10.6m/storage/innobase/handler/ha_innodb.cc:4390 #10 0x55b87fdb6dc9 in innobase_commit_ordered_2 /mariadb/10.6m/storage/innobase/handler/ha_innodb.cc:4494 #11 0x55b87fdb7370 in innobase_commit /mariadb/10.6m/storage/innobase/handler/ha_innodb.cc:4595 #12 0x55b87eef1cb9 in commit_one_phase_2 /mariadb/10.6m/sql/handler.cc:2100 #13 0x55b87eef1fc9 in ha_commit_one_phase(THD*, bool) /mariadb/10.6m/sql/handler.cc:2053 #14 0x55b87eef3d43 in ha_commit_trans(THD*, bool) /mariadb/10.6m/sql/handler.cc:1847 #15 0x55b87f883a21 in trans_commit(THD*) /mariadb/10.6m/sql/transaction.cc:266 #16 0x55b87f4a763e in mysql_execute_command(THD*, bool) /mariadb/10.6m/sql/sql_parse.cc:5646 #17 0x55b87f4ab275 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /mariadb/10.6m/sql/sql_parse.cc:8048 #18 0x55b87ef7cc00 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /mariadb/10.6m/sql/log_event_server.cc:1927 #19 0x55b87ef7dc2b in Query_log_event::do_apply_event(rpl_group_info*) /mariadb/10.6m/sql/log_event_server.cc:1605 #20 0x55b87f1b8123 in Log_event::apply_event(rpl_group_info*) /mariadb/10.6m/sql/log_event.h:1499 #21 0x55b87f199115 in apply_event_and_update_pos_apply /mariadb/10.6m/sql/slave.cc:3894 #22 0x55b87f1a5655 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /mariadb/10.6m/sql/slave.cc:4080 #23 0x55b87f98fa93 in rpt_handle_event /mariadb/10.6m/sql/rpl_parallel.cc:64 #24 0x55b87f99817e in handle_rpl_parallel_thread /mariadb/10.6m/sql/rpl_parallel.cc:1481 #25 0x55b87fc2b381 in pfs_spawn_thread /mariadb/10.6m/storage/perfschema/pfs.cc:2201 #26 0x7f61cbca63eb in start_thread nptl/pthread_create.c:444 #27 0x7f61cbd26a1b in clone3 ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81   0x618000062798 is located 792 bytes inside of 824-byte region [0x618000062480,0x6180000627b8) freed by thread T34 here: #0 0x7f61ccad7288 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52 #1 0x55b87fe07cf9 in ut_allocator<unsigned char, true>::deallocate(unsigned char*, unsigned long) /mariadb/10.6m/storage/innobase/include/ut0new.h:424 #2 0x55b8801f44fd in mem_heap_block_free(mem_block_info_t*, mem_block_info_t*) /mariadb/10.6m/storage/innobase/mem/mem0mem.cc:416 #3 0x55b87ffd5b7e in mem_heap_free /mariadb/10.6m/storage/innobase/include/mem0mem.inl:419 #4 0x55b87ffdd33e in dict_mem_table_free(dict_table_t*) /mariadb/10.6m/storage/innobase/dict/dict0mem.cc:234 #5 0x55b87fe5e41d in dict_sys_t::remove(dict_table_t*, bool, bool) /mariadb/10.6m/storage/innobase/dict/dict0dict.cc:1943 #6 0x55b87fdeb95a in ha_innobase::delete_table(char const*) /mariadb/10.6m/storage/innobase/handler/ha_innodb.cc:13513 #7 0x55b87eef6e6d in hton_drop_table /mariadb/10.6m/sql/handler.cc:573 #8 0x55b87fa5236a in THD::rm_temporary_table(handlerton*, char const*) /mariadb/10.6m/sql/temporary_tables.cc:705 #9 0x55b87fa53138 in THD::free_tmp_table_share(TMP_TABLE_SHARE*, bool) /mariadb/10.6m/sql/temporary_tables.cc:1468 #10 0x55b87fa56580 in THD::drop_temporary_table(TABLE*, bool*, bool) /mariadb/10.6m/sql/temporary_tables.cc:673 #11 0x55b87f6bd60a in mysql_rm_table_no_locks(THD*, TABLE_LIST*, st_mysql_const_lex_string const*, st_ddl_log_state*, bool, bool, bool, bool, bool, bool) /mariadb/10.6m/sql/sql_table.cc:1295 #12 0x55b87f6c13fd in mysql_rm_table(THD*, TABLE_LIST*, bool, bool, bool, bool) /mariadb/10.6m/sql/sql_table.cc:1043 #13 0x55b87f4a38d6 in mysql_execute_command(THD*, bool) /mariadb/10.6m/sql/sql_parse.cc:4976 #14 0x55b87f4ab275 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /mariadb/10.6m/sql/sql_parse.cc:8048 #15 0x55b87ef7cc00 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /mariadb/10.6m/sql/log_event_server.cc:1927 #16 0x55b87ef7dc2b in Query_log_event::do_apply_event(rpl_group_info*) /mariadb/10.6m/sql/log_event_server.cc:1605 #17 0x55b87f1b8123 in Log_event::apply_event(rpl_group_info*) /mariadb/10.6m/sql/log_event.h:1499 #18 0x55b87f199115 in apply_event_and_update_pos_apply /mariadb/10.6m/sql/slave.cc:3894 #19 0x55b87f1a5655 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /mariadb/10.6m/sql/slave.cc:4080 #20 0x55b87f98fa93 in rpt_handle_event /mariadb/10.6m/sql/rpl_parallel.cc:64 #21 0x55b87f99817e in handle_rpl_parallel_thread /mariadb/10.6m/sql/rpl_parallel.cc:1481 #22 0x55b87fc2b381 in pfs_spawn_thread /mariadb/10.6m/storage/perfschema/pfs.cc:2201 #23 0x7f61cbca63eb in start_thread nptl/pthread_create.c:444 #24 0x7f61cbd26a1b in clone3 ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81   previously allocated by thread T33 here: #0 0x7f61ccad85bf in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69 #1 0x55b87fe08ccf in ut_allocator<unsigned char, true>::allocate(unsigned long, unsigned char const*, unsigned int, bool, bool) /mariadb/10.6m/storage/innobase/include/ut0new.h:375 #2 0x55b8801f4871 in mem_heap_create_block_func(mem_block_info_t*, unsigned long, char const*, unsigned int, unsigned long) /mariadb/10.6m/storage/innobase/mem/mem0mem.cc:277 #3 0x55b8801f4f23 in mem_heap_add_block(mem_block_info_t*, unsigned long) /mariadb/10.6m/storage/innobase/mem/mem0mem.cc:378 #4 0x55b87ffd5822 in mem_heap_alloc /mariadb/10.6m/storage/innobase/include/mem0mem.inl:193 #5 0x55b87ffd5966 in mem_heap_zalloc /mariadb/10.6m/storage/innobase/include/mem0mem.inl:162 #6 0x55b87ffdaaf0 in dict_table_t::create(st_::span<char const> const&, fil_space_t*, unsigned long, unsigned long, unsigned long, unsigned long) /mariadb/10.6m/storage/innobase/dict/dict0mem.cc:148 #7 0x55b87fe0495d in create_table_info_t::create_table_def() /mariadb/10.6m/storage/innobase/handler/ha_innodb.cc:10726 #8 0x55b87fdf1566 in create_table_info_t::create_table(bool) /mariadb/10.6m/storage/innobase/handler/ha_innodb.cc:12789 #9 0x55b87fdf2b40 in ha_innobase::create(char const*, TABLE*, HA_CREATE_INFO*, bool, trx_t*) /mariadb/10.6m/storage/innobase/handler/ha_innodb.cc:13296 #10 0x55b87fdf2fbf in ha_innobase::create(char const*, TABLE*, HA_CREATE_INFO*) /mariadb/10.6m/storage/innobase/handler/ha_innodb.cc:13340 #11 0x55b87ef06fb8 in handler::ha_create(char const*, TABLE*, HA_CREATE_INFO*) /mariadb/10.6m/sql/handler.cc:5515 #12 0x55b87ef08d19 in ha_create_table(THD*, char const*, char const*, char const*, HA_CREATE_INFO*, st_mysql_const_unsigned_lex_string*, bool) /mariadb/10.6m/sql/handler.cc:5983 #13 0x55b87f6e0314 in create_table_impl /mariadb/10.6m/sql/sql_table.cc:4422 #14 0x55b87f6e1891 in mysql_create_table_no_lock(THD*, st_ddl_log_state*, st_ddl_log_state*, Table_specification_st*, Alter_info*, bool*, int, TABLE_LIST*) /mariadb/10.6m/sql/sql_table.cc:4521 #15 0x55b87f6e249a in mysql_create_table(THD*, TABLE_LIST*, Table_specification_st*, Alter_info*) /mariadb/10.6m/sql/sql_table.cc:4687 #16 0x55b87f6f2905 in Sql_cmd_create_table_like::execute(THD*) /mariadb/10.6m/sql/sql_table.cc:11972 #17 0x55b87f4a97fb in mysql_execute_command(THD*, bool) /mariadb/10.6m/sql/sql_parse.cc:6024 #18 0x55b87f4ab275 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /mariadb/10.6m/sql/sql_parse.cc:8048 #19 0x55b87ef7cc00 in Query_log_event::do_apply_event(rpl_group_info*, char const*, unsigned int) /mariadb/10.6m/sql/log_event_server.cc:1927 #20 0x55b87ef7dc2b in Query_log_event::do_apply_event(rpl_group_info*) /mariadb/10.6m/sql/log_event_server.cc:1605 #21 0x55b87f1b8123 in Log_event::apply_event(rpl_group_info*) /mariadb/10.6m/sql/log_event.h:1499 #22 0x55b87f199115 in apply_event_and_update_pos_apply /mariadb/10.6m/sql/slave.cc:3894 #23 0x55b87f1a5655 in apply_event_and_update_pos_for_parallel(Log_event*, THD*, rpl_group_info*) /mariadb/10.6m/sql/slave.cc:4080 #24 0x55b87f98fa93 in rpt_handle_event /mariadb/10.6m/sql/rpl_parallel.cc:64 #25 0x55b87f99817e in handle_rpl_parallel_thread /mariadb/10.6m/sql/rpl_parallel.cc:1481 #26 0x55b87fc2b381 in pfs_spawn_thread /mariadb/10.6m/storage/perfschema/pfs.cc:2201 #27 0x7f61cbca63eb in start_thread nptl/pthread_create.c:444 The output mentions two threads: T33, which did CREATE TEMPORARY TABLE and crashes on COMMIT , and T34, which executed DROP TEMPORARY TABLE before all transactions that used the temporary table were committed. This is similar to my first comment here, except that in that comment, the CREATE, COMMIT, and DROP were all executed in different threads.

            I don’t think that this should be closed before the bug that it duplicates is fixed. We need to test the fix.

            marko Marko Mäkelä added a comment - I don’t think that this should be closed before the bug that it duplicates is fixed. We need to test the fix.

            Same issue as MDEV-10356, fix pushed to 10.4.

            knielsen Kristian Nielsen added a comment - Same issue as MDEV-10356 , fix pushed to 10.4.
            julien.fritsch Julien Fritsch added a comment - - edited

            knielsen If it's a duplicate, use "N/A" as a fixversion since it's not in any version's code. I mean this issue. Otherwise, if it is, then please add all the following versions in which the code will be.

            julien.fritsch Julien Fritsch added a comment - - edited knielsen If it's a duplicate, use "N/A" as a fixversion since it's not in any version's code. I mean this issue. Otherwise, if it is, then please add all the following versions in which the code will be.

            julien.fritsch Ok. Is there any way to add the versions other than to manually click through the gui for every single "unreleased" version number within each major release series?

            knielsen Kristian Nielsen added a comment - julien.fritsch Ok. Is there any way to add the versions other than to manually click through the gui for every single "unreleased" version number within each major release series?

            knielsen not yet, but I'm working on a solution and it's almost ready.

            julien.fritsch Julien Fritsch added a comment - knielsen not yet, but I'm working on a solution and it's almost ready.

            People

              knielsen Kristian Nielsen
              alice Alice Sherepa
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.