[MDEV-28264] Heap-use-after-free error while loading the table in dict_sys_t::load_table() Created: 2022-04-08  Updated: 2022-04-20

Status: Open
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.6.2
Fix Version/s: 10.6

Type: Bug Priority: Major
Reporter: Thirunarayanan Balathandayuthapani Assignee: Thirunarayanan Balathandayuthapani
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Problem/Incident
is caused by MDEV-25743 Unnecessary copying of table names in... Closed

 Description    

[rr 4150676 7553]2022-04-07  9:50:23 0 [Warning] InnoDB: Load table `test`.`C` failed, the table has missing foreign key indexes. Turn off 'foreign_key_checks' and try again.
 
[rr 4150676 7557]=================================================================
 
[rr 4150676 7564][rr 4150676 7568]==4150676==ERROR: AddressSanitizer: heap-use-after-free on address 0x61c000004c48 at pc 0x7f6e79e15a6d bp 0x7fffcc9ca8c0 sp 0x7fffcc9ca068
 
[rr 4150676 7571][rr 4150676 7573]READ of size 8 at 0x61c000004c48 thread T0
 
[rr 4150676 13317]    #0 0x7f6e79e15a6c  (/lib/x86_64-linux-gnu/libasan.so.5+0x67a6c)
 
[rr 4150676 13355]    #1 0x55c5e378d28c in dict_sys_t::load_table(st_::span<char const> const&, dict_err_ignore_t) /data/Server/bb-10.6-MDEV-27783A/storage/innobase/dict/dict0load.cc:2531
 
[rr 4150676 13357]    #2 0x55c5e378db76 in dict_load_table_on_id(unsigned long, dict_err_ignore_t) /data/Server/bb-10.6-MDEV-27783A/storage/innobase/dict/dict0load.cc:2601
 
[rr 4150676 13373]    #3 0x55c5e37704ea in dict_table_t* dict_table_open_on_id<false>(unsigned long, bool, dict_table_op_t, THD*, MDL_ticket**) /data/Server/bb-10.6-MDEV-27783A/storage/innobase/dict/dict0dict.cc:860
 
[rr 4150676 13385]    #4 0x55c5e35e479b in trx_resurrect_table_locks /data/Server/bb-10.6-MDEV-27783A/storage/innobase/trx/trx0trx.cc:602
 
[rr 4150676 13387]    #5 0x55c5e35e5716 in trx_resurrect /data/Server/bb-10.6-MDEV-27783A/storage/innobase/trx/trx0trx.cc:685
 
[rr 4150676 13389]    #6 0x55c5e35e5e6d in trx_lists_init_at_db_start() /data/Server/bb-10.6-MDEV-27783A/storage/innobase/trx/trx0trx.cc:735
 
[rr 4150676 13397]    #7 0x55c5e3578170 in srv_start(bool) /data/Server/bb-10.6-MDEV-27783A/storage/innobase/srv/srv0start.cc:1467
 
[rr 4150676 13421]    #8 0x55c5e31041cc in innodb_init /data/Server/bb-10.6-MDEV-27783A/storage/innobase/handler/ha_innodb.cc:4291
 
[rr 4150676 13439]    #9 0x55c5e2813ece in ha_initialize_handlerton(st_plugin_int*) /data/Server/bb-10.6-MDEV-27783A/sql/handler.cc:659
 
[rr 4150676 13449]    #10 0x55c5e2072a9b in plugin_initialize /data/Server/bb-10.6-MDEV-27783A/sql/sql_plugin.cc:1464
 
[rr 4150676 13451]    #11 0x55c5e2074800 in plugin_init(int*, char**, int) /data/Server/bb-10.6-MDEV-27783A/sql/sql_plugin.cc:1757
 
[rr 4150676 13463]    #12 0x55c5e1d7e88d in init_server_components /data/Server/bb-10.6-MDEV-27783A/sql/mysqld.cc:5052
 
[rr 4150676 13465]    #13 0x55c5e1d8024c in mysqld_main(int, char**) /data/Server/bb-10.6-MDEV-27783A/sql/mysqld.cc:5667
 
[rr 4150676 13467]    #14 0x55c5e1d6efbc in main /data/Server/bb-10.6-MDEV-27783A/sql/main.cc:34
 
[rr 4150676 13469]    #15 0x7f6e791b30b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
 
[rr 4150676 13471]    #16 0x55c5e1d6eedd in _start (/data/Server_bin/bb-10.6-MDEV-27783A_asan/bin/mariadbd+0x13f6edd)

dict_sys_t::load_table() function is throwing the error:

dict_table_t *dict_sys_t::load_table(const span<const char> &name,
 
                                     dict_err_ignore_t ignore)
 
{
 
  if (dict_table_t *table= find_table(name))
 
    return table;
 
  dict_names_t fk_list;
 
  dict_table_t *table= dict_load_table_one(name, ignore, fk_list);
 
  while (!fk_list.empty())
 
  {
 
    const char *f= fk_list.front();
 
    const span<const char> name{f, strlen(f)};
 
    if (!find_table(name))
 
      dict_load_table_one(name, ignore, fk_list);
 
    fk_list.pop_front();
 
  }
 
 
 
  return table;
 
}

In above function `dict_load_table_one()` fails because of error `DB_CANNOT_ADD_CONSTRAINT` and it is being
evicted and table has been freed. In the next iteration, it tries to access the name present in fk_list. But the name is part of freed table heap and leads to the ASAN failure.

This issue is specific to 10.6. This function has been added as a part of the following
commit message:

commit 49e2c8f0a6fefdeac50925f758090d6bd099768d
 
Author: Marko Mäkelä <marko.makela@mariadb.com>
 
Date:   Thu May 20 14:58:25 2021 +0300
 
 
 
    MDEV-25743: Unnecessary copying of table names in InnoDB dictionary



 Comments   
Comment by Matthias Leich [ 2022-04-08 ] 

Workflow when hitting the problem
 
1. Start server and generate initial data
 
2. One session runs a DDL/DML mix
 
3. During 2. is ongoing
 
    Mariabackup --backup of that server to destination X
 
    Mariabackup --prepare on destination X
 
    Attempt to start a server on the data at destination X and that fails.
 
 
 
pluto:/data/results/1649323908/TBR-1448/
 
RQG
 
====
 
# git clone https://github.com/mleich1/rqg --branch experimental RQG
 
#
 
# GIT_SHOW: HEAD -> experimental, origin/experimental 358366308288eaadbcb62822ef4faadf6e7aebc8 2022-03-28T18:04:44+02:00
 
# rqg.pl  : Version 4.0.4 (2021-12)
 
#
 
# $RQG_HOME/rqg.pl \
 
# --views \
 
# --grammar=conf/mariadb/partitions_innodb.yy \
 
# --redefine=conf/mariadb/alter_table.yy \
 
# --redefine=conf/mariadb/instant_add.yy \
 
# --redefine=conf/mariadb/modules/alter_table_columns.yy \
 
# --redefine=conf/mariadb/bulk_insert.yy \
 
# --redefine=conf/mariadb/modules/foreign_keys.yy \
 
# --redefine=conf/mariadb/modules/locks.yy \
 
# --redefine=conf/mariadb/modules/sql_mode.yy \
 
# --redefine=conf/mariadb/versioning.yy \
 
# --redefine=conf/mariadb/sequences.yy \
 
# --redefine=conf/mariadb/modules/locks-10.4-extra.yy \
 
# --mysqld=--loose-innodb_lock_schedule_algorithm=fcfs \
 
# --mysqld=--loose-idle_write_transaction_timeout=0 \
 
# --mysqld=--loose-idle_transaction_timeout=0 \
 
# --mysqld=--loose-idle_readonly_transaction_timeout=0 \
 
# --mysqld=--connect_timeout=60 \
 
# --mysqld=--interactive_timeout=28800 \
 
# --mysqld=--slave_net_timeout=60 \
 
# --mysqld=--net_read_timeout=30 \
 
# --mysqld=--net_write_timeout=60 \
 
# --mysqld=--loose-table_lock_wait_timeout=50 \
 
# --mysqld=--wait_timeout=28800 \
 
# --mysqld=--lock-wait-timeout=86400 \
 
# --mysqld=--innodb-lock-wait-timeout=50 \
 
# --no-mask \
 
# --queries=10000000 \
 
# --seed=random \
 
# --reporters=Backtrace \
 
# --reporters=ErrorLog \
 
# --reporters=Deadlock1 \
 
# --validators=None \
 
# --mysqld=--log_output=none \
 
# --mysqld=--log_bin_trust_function_creators=1 \
 
# --mysqld=--loose-debug_assert_on_not_freed_memory=0 \
 
# --engine=InnoDB \
 
# --restart_timeout=360 \
 
# --mysqld=--plugin-load-add=file_key_management.so \
 
# --mysqld=--loose-file-key-management-filename=$RQG_HOME/conf/mariadb/encryption_keys.txt \
 
# --mysqld=--plugin-load-add=provider_lzo.so \
 
# --mysqld=--plugin-load-add=provider_bzip2.so \
 
# --mysqld=--plugin-load-add=provider_lzma.so \
 
# --mysqld=--plugin-load-add=provider_snappy.so \
 
# --mysqld=--plugin-load-add=provider_lz4.so \
 
# --duration=300 \
 
# --mysqld=--loose-innodb_fatal_semaphore_wait_threshold=300 \
 
# --mysqld=--loose-innodb_read_only_compressed=OFF \
 
# --reporters=Mariabackup_linux \
 
# --duration=300 \
 
# --mysqld=--loose-innodb-sync-debug \
 
# --mysqld=--innodb_stats_persistent=off \
 
# --mysqld=--innodb_adaptive_hash_index=off \
 
# --mysqld=--log-bin \
 
# --mysqld=--sync-binlog=1 \
 
# --mysqld=--loose-innodb_evict_tables_on_commit_debug=off \
 
# --mysqld=--loose-max-statement-time=30 \
 
# --threads=1 \
 
# --mysqld=--innodb-use-native-aio=0 \
 
# --mysqld=--loose-gdb \
 
# --mysqld=--loose-debug-gdb \
 
# --rr=Extended \
 
# --rr_options=--wait \
 
# --mysqld=--innodb_rollback_on_timeout=OFF \
 
# --vardir_type=fast \
 
# --mysqld=--innodb_page_size=4K \
 
# --mysqld=--innodb-buffer-pool-size=256M \
 
# --no_mask \
 
# <local settings>

Generated at Thu Feb 08 09:59:22 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.