Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-40070

Crash on innodb_log_archive=ON recovery after reporting "Multi-batch recovery needed"

    XMLWordPrintable

Details

    • Can result in hang or crash
    • InnoDB recovery with innodb_log_file_mmap=ON could crash if the mini-transaction that triggered a multi-batch recovery spans two innodb_log_archive=ON log files.

    Description

      Assertion found while testing MDEV-38936

      mdev-38936 6330b4c9b6bdcd8ec999f0a36c59e94c286bd0c9

      # 2026-06-16T01:41:24 [3013686] | mariadbd: /data/Server/MDEV-38936/storage/innobase/log/log0recv.cc:3842: void log_t::archived_mmap_switch_recovery_complete(): Assertion `archived_mmap_switch()' failed.
      

      StackTrace

      # 2026-06-16T01:43:12 [3013686] | #0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:44
      # 2026-06-16T01:43:12 [3013686] | #1  __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
      # 2026-06-16T01:43:12 [3013686] | #2  __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
      # 2026-06-16T01:43:12 [3013686] | #3  0x000011d00d85526e in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
      # 2026-06-16T01:43:12 [3013686] | #4  0x000011d00d8388ff in __GI_abort () at ./stdlib/abort.c:79
      # 2026-06-16T01:43:12 [3013686] | #5  0x000011d00d83881b in __assert_fail_base (fmt=0x11d00d9e01e8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x5673a301ee8f "archived_mmap_switch()", file=file@entry=0x5673a2f2d398 "/data/Server/MDEV-38936/storage/innobase/log/log0recv.cc", line=line@entry=3842, function=function@entry=0x5673a2f2f2b8 "void log_t::archived_mmap_switch_recovery_complete()") at ./assert/assert.c:94
      # 2026-06-16T01:43:12 [3013686] | #6  0x000011d00d84b507 in __assert_fail (assertion=0x5673a301ee8f "archived_mmap_switch()", file=0x5673a2f2d398 "/data/Server/MDEV-38936/storage/innobase/log/log0recv.cc", line=3842, function=0x5673a2f2f2b8 "void log_t::archived_mmap_switch_recovery_complete()") at ./assert/assert.c:103
      # 2026-06-16T01:43:12 [3013686] | #7  0x00005673a2a4b82c in log_t::archived_mmap_switch_recovery_complete (this=0x5673a4252c80 <log_sys>) at /data/Server/MDEV-38936/storage/innobase/log/log0recv.cc:3842
      # 2026-06-16T01:43:12 [3013686] | #8  0x00005673a2a7be08 in recv_sys_t::parse_mmap<(recv_sys_t::store)2, 1349024115u> (if_exists=<optimized out>) at /data/Server/MDEV-38936/storage/innobase/log/log0recv.cc:3889
      # 2026-06-16T01:43:12 [3013686] | #9  0x00005673a2a4f946 in recv_scan_archive_store (parser=@0x7fffcf7b43e8: 0x5673a2a7b724 <recv_sys_t::parse_mmap<(recv_sys_t::store)2, 1349024115u>(bool)>, last_phase=false) at /data/Server/MDEV-38936/storage/innobase/log/log0recv.cc:4957
      # 2026-06-16T01:43:12 [3013686] | #10 0x00005673a2a50caa in recv_scan_log (last_phase=last_phase@entry=false, parser=parser@entry=0x7fffcf7b43e0) at /data/Server/MDEV-38936/storage/innobase/log/log0recv.cc:5190
      # 2026-06-16T01:43:12 [3013686] | #11 0x00005673a2a59810 in recv_recovery_from_checkpoint_start () at /data/Server/MDEV-38936/storage/innobase/log/log0recv.cc:5877
      # 2026-06-16T01:43:12 [3013686] | #12 0x00005673a2b5aaa7 in srv_start (create_new_db=<optimized out>) at /data/Server/MDEV-38936/storage/innobase/srv/srv0start.cc:1564
      # 2026-06-16T01:43:12 [3013686] | #13 0x00005673a29b8e4d in innodb_init (p=<optimized out>) at /data/Server/MDEV-38936/storage/innobase/handler/ha_innodb.cc:4420
      # 2026-06-16T01:43:12 [3013686] | #14 0x00005673a271d104 in ha_initialize_handlerton (plugin_=0x5673a5f81d80) at /data/Server/MDEV-38936/sql/handler.cc:742
      # 2026-06-16T01:43:12 [3013686] | #15 0x00005673a242ea4b in plugin_do_initialize (plugin=plugin@entry=0x5673a5f81d80, state=@0x7fffcf7b4abc: 4) at /data/Server/MDEV-38936/sql/sql_plugin.cc:1464
      # 2026-06-16T01:43:12 [3013686] | #16 0x00005673a24329b1 in plugin_initialize (tmp_root=tmp_root@entry=0x7fffcf7b50c0, plugin=plugin@entry=0x5673a5f81d80, argc=argc@entry=0x5673a39ff9a0 <remaining_argc>, argv=argv@entry=0x5673a5ce6a80, options_only=false) at /data/Server/MDEV-38936/sql/sql_plugin.cc:1518
      # 2026-06-16T01:43:12 [3013686] | #17 0x00005673a2434893 in plugin_init (argc=argc@entry=0x5673a39ff9a0 <remaining_argc>, argv=<optimized out>, flags=0) at /data/Server/MDEV-38936/sql/sql_plugin.cc:1762
      # 2026-06-16T01:43:12 [3013686] | #18 0x00005673a22e5442 in init_server_components () at /data/Server/MDEV-38936/sql/mysqld.cc:5464
      # 2026-06-16T01:43:12 [3013686] | #19 0x00005673a22e944b in mysqld_main (argc=<optimized out>, argv=<optimized out>) at /data/Server/MDEV-38936/sql/mysqld.cc:6264
      # 2026-06-16T01:43:12 [3013686] | #20 0x00005673a22dd0a6 in main (argc=<optimized out>, argv=<optimized out>) at /data/Server/MDEV-38936/sql/main.cc:34
      

      RQG startup options used

      # $RQG_HOME/rqg.pl \
      # --gendata=conf/mariadb/table_stress.zz \
      # --gendata_sql=conf/mariadb/table_stress.sql \
      # --grammar=conf/mariadb/table_stress_innodb_fk.yy \
      # --redefine=conf/mariadb/redefine_innodb_log_archive.yy \
      # --mysqld=--loose-innodb_log_archive=ON \
      # --mysqld=--loose-innodb_log_file_size=32M \
      # --reporters=LogArchiveRecovery \
      # --mysqld=--loose-innodb_lock_schedule_algorithm=fcfs \
      # --mysqld=--loose-idle_write_transaction_timeout=0 \
      # --mysqld=--loose-idle_transaction_timeout=0 \
      # --mysqld=--loose-idle_readonly_transaction_timeout=0 \
      # --mysqld=--connect_timeout=60 \
      # --mysqld=--interactive_timeout=28800 \
      # --mysqld=--slave_net_timeout=60 \
      # --mysqld=--net_read_timeout=30 \
      # --mysqld=--net_write_timeout=60 \
      # --mysqld=--loose-table_lock_wait_timeout=50 \
      # --mysqld=--wait_timeout=28800 \
      # --no_mask \
      # --queries=10000000 \
      # --seed=random \
      # --reporters=None \
      # --reporters=ErrorLog \
      # --reporters=Deadlock \
      # --validators=None \
      # --mysqld=--log_output=none \
      # --mysqld=--log_bin_trust_function_creators=1 \
      # --mysqld=--loose-debug_assert_on_not_freed_memory=0 \
      # --engine=InnoDB \
      # --restart_timeout=240 \
      # --mysqld=--plugin-load-add=file_key_management.so \
      # --mysqld=--loose-file-key-management-filename=$RQG_HOME/conf/mariadb/encryption_keys.txt \
      # --mysqld=--plugin-load-add=provider_lzo.so \
      # --mysqld=--plugin-load-add=provider_bzip2.so \
      # --mysqld=--plugin-load-add=provider_lzma.so \
      # --mysqld=--plugin-load-add=provider_snappy.so \
      # --mysqld=--plugin-load-add=provider_lz4.so \
      # --mysqld=--loose-innodb_compression_level=1 \
      # --duration=300 \
      # --mysqld=--loose-innodb_fatal_semaphore_wait_threshold=300 \
      # --mysqld=--loose-innodb_data_file_path=ibdata1:1M:autoextend:autoshrink \
      # --redefine=conf/mariadb/redefine_innodb_log_write_ahead_size.yy \
      # --redefine=conf/mariadb/redefine_indexed_vcols.yy \
      # --mysqld=--lock-wait-timeout=86400 \
      # --mysqld=--innodb-lock-wait-timeout=50 \
      # --mysqld=--sql_mode=traditional \
      # --mysqld=--innodb_file_per_table=1 \
      # --mysqld=--loose-innodb_read_only_compressed=OFF \
      # --mysqld=--loose-innodb-sync-debug \
      # --mysqld=--innodb_stats_persistent=on \
      # --mysqld=--innodb_adaptive_hash_index=on \
      # --mysqld=--innodb_random_read_ahead=OFF \
      # --mysqld=--log-bin \
      # --mysqld=--sync-binlog=1 \
      # --mysqld=--loose-innodb_evict_tables_on_commit_debug=on \
      # --threads=2 \
      # --rr=rr record --chaos --wait \
      # --mysqld=--innodb_undo_log_truncate=OFF \
      # --mysqld=--innodb_undo_tablespaces=16 \
      # --mysqld=--innodb_rollback_on_timeout=ON \
      # --mysqld=--innodb_page_size=32K \
      # --mysqld=--innodb-buffer-pool-size=24M \
      # --vardir_type=fast \
      # --no_mask \
      # --batch \
      # --basedir1=/Server_bin/MDEV-38936_debug_Og \
      # --basedir2=/Server_bin/MDEV-38936_debug_Og \
      # --script_debug=_nix_ \
      # --major_runid=1781597031 \
      # --minor_runid=34
      

      RR trace is present on SDP:-
      /data/results/1781597031/TBR-2436

      Please assign bug back to me if simplification is needed

      Attachments

        Issue Links

          Activity

            People

              marko Marko Mäkelä
              saahil Saahil Alam
              Marko Mäkelä Marko Mäkelä
              Thirunarayanan Balathandayuthapani Thirunarayanan Balathandayuthapani
              Saahil Alam Saahil Alam
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.