Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-28264

Heap-use-after-free error while loading the table in dict_sys_t::load_table()

Details

    Description

      [rr 4150676 7553]2022-04-07  9:50:23 0 [Warning] InnoDB: Load table `test`.`C` failed, the table has missing foreign key indexes. Turn off 'foreign_key_checks' and try again.
      [rr 4150676 7557]=================================================================
      [rr 4150676 7564][rr 4150676 7568]==4150676==ERROR: AddressSanitizer: heap-use-after-free on address 0x61c000004c48 at pc 0x7f6e79e15a6d bp 0x7fffcc9ca8c0 sp 0x7fffcc9ca068
      [rr 4150676 7571][rr 4150676 7573]READ of size 8 at 0x61c000004c48 thread T0
      [rr 4150676 13317]    #0 0x7f6e79e15a6c  (/lib/x86_64-linux-gnu/libasan.so.5+0x67a6c)
      [rr 4150676 13355]    #1 0x55c5e378d28c in dict_sys_t::load_table(st_::span<char const> const&, dict_err_ignore_t) /data/Server/bb-10.6-MDEV-27783A/storage/innobase/dict/dict0load.cc:2531
      [rr 4150676 13357]    #2 0x55c5e378db76 in dict_load_table_on_id(unsigned long, dict_err_ignore_t) /data/Server/bb-10.6-MDEV-27783A/storage/innobase/dict/dict0load.cc:2601
      [rr 4150676 13373]    #3 0x55c5e37704ea in dict_table_t* dict_table_open_on_id<false>(unsigned long, bool, dict_table_op_t, THD*, MDL_ticket**) /data/Server/bb-10.6-MDEV-27783A/storage/innobase/dict/dict0dict.cc:860
      [rr 4150676 13385]    #4 0x55c5e35e479b in trx_resurrect_table_locks /data/Server/bb-10.6-MDEV-27783A/storage/innobase/trx/trx0trx.cc:602
      [rr 4150676 13387]    #5 0x55c5e35e5716 in trx_resurrect /data/Server/bb-10.6-MDEV-27783A/storage/innobase/trx/trx0trx.cc:685
      [rr 4150676 13389]    #6 0x55c5e35e5e6d in trx_lists_init_at_db_start() /data/Server/bb-10.6-MDEV-27783A/storage/innobase/trx/trx0trx.cc:735
      [rr 4150676 13397]    #7 0x55c5e3578170 in srv_start(bool) /data/Server/bb-10.6-MDEV-27783A/storage/innobase/srv/srv0start.cc:1467
      [rr 4150676 13421]    #8 0x55c5e31041cc in innodb_init /data/Server/bb-10.6-MDEV-27783A/storage/innobase/handler/ha_innodb.cc:4291
      [rr 4150676 13439]    #9 0x55c5e2813ece in ha_initialize_handlerton(st_plugin_int*) /data/Server/bb-10.6-MDEV-27783A/sql/handler.cc:659
      [rr 4150676 13449]    #10 0x55c5e2072a9b in plugin_initialize /data/Server/bb-10.6-MDEV-27783A/sql/sql_plugin.cc:1464
      [rr 4150676 13451]    #11 0x55c5e2074800 in plugin_init(int*, char**, int) /data/Server/bb-10.6-MDEV-27783A/sql/sql_plugin.cc:1757
      [rr 4150676 13463]    #12 0x55c5e1d7e88d in init_server_components /data/Server/bb-10.6-MDEV-27783A/sql/mysqld.cc:5052
      [rr 4150676 13465]    #13 0x55c5e1d8024c in mysqld_main(int, char**) /data/Server/bb-10.6-MDEV-27783A/sql/mysqld.cc:5667
      [rr 4150676 13467]    #14 0x55c5e1d6efbc in main /data/Server/bb-10.6-MDEV-27783A/sql/main.cc:34
      [rr 4150676 13469]    #15 0x7f6e791b30b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
      [rr 4150676 13471]    #16 0x55c5e1d6eedd in _start (/data/Server_bin/bb-10.6-MDEV-27783A_asan/bin/mariadbd+0x13f6edd)
      

      dict_sys_t::load_table() function is throwing the error:

      dict_table_t *dict_sys_t::load_table(const span<const char> &name,
                                           dict_err_ignore_t ignore)
      {
        if (dict_table_t *table= find_table(name))
          return table;
        dict_names_t fk_list;
        dict_table_t *table= dict_load_table_one(name, ignore, fk_list);
        while (!fk_list.empty())
        {
          const char *f= fk_list.front();
          const span<const char> name{f, strlen(f)};
          if (!find_table(name))
            dict_load_table_one(name, ignore, fk_list);
          fk_list.pop_front();
        }
       
        return table;
      }
      

      In above function `dict_load_table_one()` fails because of error `DB_CANNOT_ADD_CONSTRAINT` and it is being
      evicted and table has been freed. In the next iteration, it tries to access the name present in fk_list. But the name is part of freed table heap and leads to the ASAN failure.

      This issue is specific to 10.6. This function has been added as a part of the following
      commit message:

      commit 49e2c8f0a6fefdeac50925f758090d6bd099768d
      Author: Marko Mäkelä <marko.makela@mariadb.com>
      Date:   Thu May 20 14:58:25 2021 +0300
       
          MDEV-25743: Unnecessary copying of table names in InnoDB dictionary
      

      Attachments

        Issue Links

          Activity

            Workflow when hitting the problem
            1. Start server and generate initial data
            2. One session runs a DDL/DML mix
            3. During 2. is ongoing
                Mariabackup --backup of that server to destination X
                Mariabackup --prepare on destination X
                Attempt to start a server on the data at destination X and that fails.
             
            pluto:/data/results/1649323908/TBR-1448/
            RQG
            ====
            # git clone https://github.com/mleich1/rqg --branch experimental RQG
            #
            # GIT_SHOW: HEAD -> experimental, origin/experimental 358366308288eaadbcb62822ef4faadf6e7aebc8 2022-03-28T18:04:44+02:00
            # rqg.pl  : Version 4.0.4 (2021-12)
            #
            # $RQG_HOME/rqg.pl \
            # --views \
            # --grammar=conf/mariadb/partitions_innodb.yy \
            # --redefine=conf/mariadb/alter_table.yy \
            # --redefine=conf/mariadb/instant_add.yy \
            # --redefine=conf/mariadb/modules/alter_table_columns.yy \
            # --redefine=conf/mariadb/bulk_insert.yy \
            # --redefine=conf/mariadb/modules/foreign_keys.yy \
            # --redefine=conf/mariadb/modules/locks.yy \
            # --redefine=conf/mariadb/modules/sql_mode.yy \
            # --redefine=conf/mariadb/versioning.yy \
            # --redefine=conf/mariadb/sequences.yy \
            # --redefine=conf/mariadb/modules/locks-10.4-extra.yy \
            # --mysqld=--loose-innodb_lock_schedule_algorithm=fcfs \
            # --mysqld=--loose-idle_write_transaction_timeout=0 \
            # --mysqld=--loose-idle_transaction_timeout=0 \
            # --mysqld=--loose-idle_readonly_transaction_timeout=0 \
            # --mysqld=--connect_timeout=60 \
            # --mysqld=--interactive_timeout=28800 \
            # --mysqld=--slave_net_timeout=60 \
            # --mysqld=--net_read_timeout=30 \
            # --mysqld=--net_write_timeout=60 \
            # --mysqld=--loose-table_lock_wait_timeout=50 \
            # --mysqld=--wait_timeout=28800 \
            # --mysqld=--lock-wait-timeout=86400 \
            # --mysqld=--innodb-lock-wait-timeout=50 \
            # --no-mask \
            # --queries=10000000 \
            # --seed=random \
            # --reporters=Backtrace \
            # --reporters=ErrorLog \
            # --reporters=Deadlock1 \
            # --validators=None \
            # --mysqld=--log_output=none \
            # --mysqld=--log_bin_trust_function_creators=1 \
            # --mysqld=--loose-debug_assert_on_not_freed_memory=0 \
            # --engine=InnoDB \
            # --restart_timeout=360 \
            # --mysqld=--plugin-load-add=file_key_management.so \
            # --mysqld=--loose-file-key-management-filename=$RQG_HOME/conf/mariadb/encryption_keys.txt \
            # --mysqld=--plugin-load-add=provider_lzo.so \
            # --mysqld=--plugin-load-add=provider_bzip2.so \
            # --mysqld=--plugin-load-add=provider_lzma.so \
            # --mysqld=--plugin-load-add=provider_snappy.so \
            # --mysqld=--plugin-load-add=provider_lz4.so \
            # --duration=300 \
            # --mysqld=--loose-innodb_fatal_semaphore_wait_threshold=300 \
            # --mysqld=--loose-innodb_read_only_compressed=OFF \
            # --reporters=Mariabackup_linux \
            # --duration=300 \
            # --mysqld=--loose-innodb-sync-debug \
            # --mysqld=--innodb_stats_persistent=off \
            # --mysqld=--innodb_adaptive_hash_index=off \
            # --mysqld=--log-bin \
            # --mysqld=--sync-binlog=1 \
            # --mysqld=--loose-innodb_evict_tables_on_commit_debug=off \
            # --mysqld=--loose-max-statement-time=30 \
            # --threads=1 \
            # --mysqld=--innodb-use-native-aio=0 \
            # --mysqld=--loose-gdb \
            # --mysqld=--loose-debug-gdb \
            # --rr=Extended \
            # --rr_options=--wait \
            # --mysqld=--innodb_rollback_on_timeout=OFF \
            # --vardir_type=fast \
            # --mysqld=--innodb_page_size=4K \
            # --mysqld=--innodb-buffer-pool-size=256M \
            # --no_mask \
            # <local settings>
            
            

            mleich Matthias Leich added a comment - Workflow when hitting the problem 1. Start server and generate initial data 2. One session runs a DDL/DML mix 3. During 2. is ongoing Mariabackup --backup of that server to destination X Mariabackup --prepare on destination X Attempt to start a server on the data at destination X and that fails.   pluto:/data/results/1649323908/TBR-1448/ RQG ==== # git clone https://github.com/mleich1/rqg --branch experimental RQG # # GIT_SHOW: HEAD -> experimental, origin/experimental 358366308288eaadbcb62822ef4faadf6e7aebc8 2022-03-28T18:04:44+02:00 # rqg.pl : Version 4.0.4 (2021-12) # # $RQG_HOME/rqg.pl \ # --views \ # --grammar=conf/mariadb/partitions_innodb.yy \ # --redefine=conf/mariadb/alter_table.yy \ # --redefine=conf/mariadb/instant_add.yy \ # --redefine=conf/mariadb/modules/alter_table_columns.yy \ # --redefine=conf/mariadb/bulk_insert.yy \ # --redefine=conf/mariadb/modules/foreign_keys.yy \ # --redefine=conf/mariadb/modules/locks.yy \ # --redefine=conf/mariadb/modules/sql_mode.yy \ # --redefine=conf/mariadb/versioning.yy \ # --redefine=conf/mariadb/sequences.yy \ # --redefine=conf/mariadb/modules/locks-10.4-extra.yy \ # --mysqld=--loose-innodb_lock_schedule_algorithm=fcfs \ # --mysqld=--loose-idle_write_transaction_timeout=0 \ # --mysqld=--loose-idle_transaction_timeout=0 \ # --mysqld=--loose-idle_readonly_transaction_timeout=0 \ # --mysqld=--connect_timeout=60 \ # --mysqld=--interactive_timeout=28800 \ # --mysqld=--slave_net_timeout=60 \ # --mysqld=--net_read_timeout=30 \ # --mysqld=--net_write_timeout=60 \ # --mysqld=--loose-table_lock_wait_timeout=50 \ # --mysqld=--wait_timeout=28800 \ # --mysqld=--lock-wait-timeout=86400 \ # --mysqld=--innodb-lock-wait-timeout=50 \ # --no-mask \ # --queries=10000000 \ # --seed=random \ # --reporters=Backtrace \ # --reporters=ErrorLog \ # --reporters=Deadlock1 \ # --validators=None \ # --mysqld=--log_output=none \ # --mysqld=--log_bin_trust_function_creators=1 \ # --mysqld=--loose-debug_assert_on_not_freed_memory=0 \ # --engine=InnoDB \ # --restart_timeout=360 \ # --mysqld=--plugin-load-add=file_key_management.so \ # --mysqld=--loose-file-key-management-filename=$RQG_HOME/conf/mariadb/encryption_keys.txt \ # --mysqld=--plugin-load-add=provider_lzo.so \ # --mysqld=--plugin-load-add=provider_bzip2.so \ # --mysqld=--plugin-load-add=provider_lzma.so \ # --mysqld=--plugin-load-add=provider_snappy.so \ # --mysqld=--plugin-load-add=provider_lz4.so \ # --duration=300 \ # --mysqld=--loose-innodb_fatal_semaphore_wait_threshold=300 \ # --mysqld=--loose-innodb_read_only_compressed=OFF \ # --reporters=Mariabackup_linux \ # --duration=300 \ # --mysqld=--loose-innodb-sync-debug \ # --mysqld=--innodb_stats_persistent=off \ # --mysqld=--innodb_adaptive_hash_index=off \ # --mysqld=--log-bin \ # --mysqld=--sync-binlog=1 \ # --mysqld=--loose-innodb_evict_tables_on_commit_debug=off \ # --mysqld=--loose-max-statement-time=30 \ # --threads=1 \ # --mysqld=--innodb-use-native-aio=0 \ # --mysqld=--loose-gdb \ # --mysqld=--loose-debug-gdb \ # --rr=Extended \ # --rr_options=--wait \ # --mysqld=--innodb_rollback_on_timeout=OFF \ # --vardir_type=fast \ # --mysqld=--innodb_page_size=4K \ # --mysqld=--innodb-buffer-pool-size=256M \ # --no_mask \ # <local settings>

            People

              thiru Thirunarayanan Balathandayuthapani
              thiru Thirunarayanan Balathandayuthapani
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.