Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-39594

void mtr_t::commit(): Assertion `!recv_sys.rpo' failed

    XMLWordPrintable

Details

    • Can result in hang or crash

    Description

      This recovery crash looks related to MDEV-37949 ,which is recently merged
      Please take a look :-
      The issue is present on main branch

      origin/main, origin/HEAD f6e44c1b7819f9befa3824f07866822288231528

      # 2026-05-13T04:15:24 [2251354] | mariadbd: /data/Server/main_latest/storage/innobase/mtr/mtr0mtr.cc:547: void mtr_t::commit(): Assertion `!recv_sys.rpo' failed.
      

      Stacktrace

      [rr 2450094 5428]mariadbd: /data/Server/main_latest/storage/innobase/mtr/mtr0mtr.cc:547: void mtr_t::commit(): Assertion `!recv_sys.rpo' failed.
      [New Thread 2450094.2450421]
      [New Thread 2450094.2450595]
      [New Thread 2450094.2450763]
      [New Thread 2450094.2450772]
      [New Thread 2450094.2451098]
      [New Thread 2450094.2452508]
      [New Thread 2450094.2452531]
       
      Thread 1 received signal SIGABRT, Aborted.
      Download failed: Invalid argument.  Continuing without source file ./nptl/./nptl/pthread_kill.c.
      __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:44
      warning: 44     ./nptl/pthread_kill.c: No such file or directory
      (rr) set print addr off
      (rr) bt
      #0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:44
      #1  __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
      #2  __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
      #3  __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
      #4  __GI_abort () at ./stdlib/abort.c:79
      #5  __assert_fail_base (fmt="%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry="!recv_sys.rpo", file=file@entry="/data/Server/main_latest/storage/innobase/mtr/mtr0mtr.cc", 
          line=line@entry=547, function=function@entry="void mtr_t::commit()") at ./assert/assert.c:94
      #6  __assert_fail (assertion="!recv_sys.rpo", file="/data/Server/main_latest/storage/innobase/mtr/mtr0mtr.cc", line=547, function="void mtr_t::commit()") at ./assert/assert.c:103
      #7  mtr_t::commit (this=this@entry=) at /data/Server/main_latest/storage/innobase/mtr/mtr0mtr.cc:547
      #8  innobase_recover_rollback_by_xid (xid=<optimized out>) at /data/Server/main_latest/storage/innobase/handler/ha_innodb.cc:17507
      #9  xarecover_do_commit_or_rollback (hton=, arg=<optimized out>) at /data/Server/main_latest/sql/handler.cc:2969
      #10 xarecover_do_commit_or_rollback_handlerton (hton=<optimized out>, arg=<optimized out>) at /data/Server/main_latest/sql/handler.cc:3000
      #11 tp_foreach (thd=thd@entry=, func=func@entry=<xarecover_do_commit_or_rollback_handlerton(THD*, transaction_participant*, void*)>, arg=arg@entry=) at /data/Server/main_latest/sql/handler.cc:894
      #12 xarecover_complete_and_count (member_arg=, param_arg=) at /data/Server/main_latest/sql/handler.cc:3018
      #13 my_hash_iterate (hash=, action=<xarecover_complete_and_count(void*, void*)>, argument=) at /data/Server/main_latest/mysys/hash.c:808
      #14 ha_recover_complete (commit_list=commit_list@entry=, coord=<optimized out>) at /data/Server/main_latest/sql/handler.cc:3054
      #15 Recovery_context::complete (this=this@entry=, log=log@entry=<mysql_bin_log>, xids=...) at /data/Server/main_latest/sql/log.cc:13871
      #16 MYSQL_BIN_LOG::recover (this=this@entry=<mysql_bin_log>, linfo=linfo@entry=, last_log_name=last_log_name@entry="./mysql-bin.000001", first_log=first_log@entry=, fdle=fdle@entry=, do_xa=do_xa@entry=true)
          at /data/Server/main_latest/sql/log.cc:14478
      #17 MYSQL_BIN_LOG::do_binlog_recovery (this=<mysql_bin_log>, opt_name="mysql-bin", do_xa_recovery=true) at /data/Server/main_latest/sql/log.cc:14675
      #18 MYSQL_BIN_LOG::open (this=<mysql_bin_log>, opt_name="mysql-bin") at /data/Server/main_latest/sql/log.cc:13030
      #19 init_server_components () at /data/Server/main_latest/sql/mysqld.cc:5765
      #20 mysqld_main (argc=<optimized out>, argv=<optimized out>) at /data/Server/main_latest/sql/mysqld.cc:6207
      #21 main (argc=<optimized out>, argv=<optimized out>) at /data/Server/main_latest/sql/main.cc:34
      (rr) frame 7
      #7  mtr_t::commit (this=this@entry=) at /data/Server/main_latest/storage/innobase/mtr/mtr0mtr.cc:547
      547         ut_ad(!recv_sys.rpo);
      (rr) info args
      this = 
      (rr) info locals
      lsns = <optimized out>
      __PRETTY_FUNCTION__ = "void mtr_t::commit()"
      

      RQG startup options

      # $RQG_HOME/rqg.pl \
      # --gendata=conf/mariadb/oltp.zz \
      # --max_gd_duration=900 \
      # --grammar=conf/mariadb/oltp.yy \
      # --redefine=conf/mariadb/redefine_innodb_log_archive.yy \
      # --mysqld=--loose-innodb_log_archive=ON \
      # --mysqld=--loose-innodb_log_file_size=4M \
      # --mysqld=--loose-innodb_log_file_mmap=OFF \
      # --reporters=LogArchiveRecovery \
      # --mysqld=--loose-innodb_lock_schedule_algorithm=fcfs \
      # --mysqld=--loose-idle_write_transaction_timeout=0 \
      # --mysqld=--loose-idle_transaction_timeout=0 \
      # --mysqld=--loose-idle_readonly_transaction_timeout=0 \
      # --mysqld=--connect_timeout=60 \
      # --mysqld=--interactive_timeout=28800 \
      # --mysqld=--slave_net_timeout=60 \
      # --mysqld=--net_read_timeout=30 \
      # --mysqld=--net_write_timeout=60 \
      # --mysqld=--loose-table_lock_wait_timeout=50 \
      # --mysqld=--wait_timeout=28800 \
      # --no_mask \
      # --queries=10000000 \
      # --seed=random \
      # --reporters=None \
      # --reporters=ErrorLog \
      # --reporters=Deadlock \
      # --validators=None \
      # --mysqld=--log_output=none \
      # --mysqld=--log_bin_trust_function_creators=1 \
      # --mysqld=--loose-debug_assert_on_not_freed_memory=0 \
      # --engine=InnoDB \
      # --restart_timeout=240 \
      # --mysqld=--plugin-load-add=file_key_management.so \
      # --mysqld=--loose-file-key-management-filename=$RQG_HOME/conf/mariadb/encryption_keys.txt \
      # --mysqld=--plugin-load-add=provider_lzo.so \
      # --mysqld=--plugin-load-add=provider_bzip2.so \
      # --mysqld=--plugin-load-add=provider_lzma.so \
      # --mysqld=--plugin-load-add=provider_snappy.so \
      # --mysqld=--plugin-load-add=provider_lz4.so \
      # --mysqld=--loose-innodb_compression_level=1 \
      # --duration=300 \
      # --mysqld=--loose-innodb_fatal_semaphore_wait_threshold=300 \
      # --redefine=conf/mariadb/redefine_innodb_log_write_ahead_size.yy \
      # --mysqld=--lock-wait-timeout=86400 \
      # --mysqld=--innodb-lock-wait-timeout=50 \
      # --mysqld=--sql_mode=traditional \
      # --mysqld=--loose-innodb_read_only_compressed=OFF \
      # --mysqld=--innodb_stats_persistent=off \
      # --mysqld=--innodb_adaptive_hash_index=off \
      # --mysqld=--innodb_random_read_ahead=ON \
      # --mysqld=--innodb_log_buffer_size=2M \
      # --mysqld=--log-bin \
      # --mysqld=--sync-binlog=1 \
      # --mysqld=--loose-innodb_evict_tables_on_commit_debug=off \
      # --mysqld=--loose-max-statement-time=30 \
      # --threads=9 \
      # --rr=rr record --chaos --wait \
      # --mysqld=--innodb_undo_log_truncate=OFF \
      # --mysqld=--innodb_undo_tablespaces=0 \
      # --mysqld=--innodb_rollback_on_timeout=OFF \
      # --mysqld=--innodb_page_size=4K \
      # --mysqld=--innodb-buffer-pool-size=256M \
      # --vardir_type=fast \
      # --no_mask \
      # --batch \
      # --basedir1=/Server_bin/main_latest_debug_Og \
      # --basedir2=/Server_bin/main_latest_debug_Og \
      # --script_debug=_nix_ \
      # --major_runid=1778659851 \
      # --minor_runid=71
      #--------
      # rqg_rr_add ->"--innodb-use-native-aio=0" "--loose-innodb-write-io-threads=2" "--loose-innodb-read-io-threads=1" "--loose-gdb" "--loose-debug-gdb"<-
      # vardir ->/dev/shm/rqg/1778659851/71<- fs_type=tmpfs
      

      Steps to repro using datadir

       

      ssh sdp
      rm -rf /tmp/d /tmp/dt /tmp/d.err /tmp/d.sock /tmp/d.pid
      cp -a /data/results/1778659851/TBR-2371-MDEV-37949/1/fbackup/data /tmp/d
      mkdir -p /tmp/dt
       
      /Server_bin/main_latest_debug_Og/bin/mariadbd \
        --no-defaults \
        --basedir=/Server_bin/main_latest_debug_Og \
        --datadir=/tmp/d \
        --tmpdir=/tmp/dt \
        --socket=/tmp/d.sock \
        --pid-file=/tmp/d.pid \
        --port=26000 \
        --log-error=/tmp/d.err \
        --plugin-load-add=file_key_management.so \
        --loose-file-key-management-filename=/home/saahil/RQG_mleich1/conf/mariadb/encryption_keys.txt \
        --innodb_page_size=4K \
        --innodb_log_file_size=4M \
        --innodb_buffer_pool_size=256M \
        --log-bin=mysql-bin \
        --log-bin-index=mysql-bin.index \
        --innodb-log-archive=OFF \
        --innodb-log-recovery-start=0 \
        --innodb-log-recovery-target=12587280 \
        --innodb-log-file-mmap=OFF
       
      on other terminal in logs :-
      tail -f /tmp/d.err
      

      RR trace is present on SDP :-
      /data/results/1778659851/TBR-2371-MDEV-37949

      Attachments

        Issue Links

          Activity

            People

              marko Marko Mäkelä
              saahil Saahil Alam
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.