Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-35658

Assertion `commit_trx' failed during galera_as_master

Details

    Description

      Test started to fail after fix for MDEV-24035 due to assertion `commit_trx` in `innobase_commit()`:

      20:03:36 #10 0x00007b2155c2871b in __assert_fail_base (fmt=0x7b2155ddd130 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x5b938b15e821 "commit_trx", file=0x5b938b161eb0 "/tmp/workspace/mariadb-10.6-build/storage/innobase/handler/ha_innodb.cc", line=4718, function=<optimized out>) at ./assert/assert.c:92
      20:03:36 #11 0x00007b2155c39e96 in __GI___assert_fail (assertion=0x5b938b15e821 "commit_trx", file=0x5b938b161eb0 "/tmp/workspace/mariadb-10.6-build/storage/innobase/handler/ha_innodb.cc", line=4718, function=0x5b938b165080 "int innobase_commit(handlerton*, THD*, bool)") at ./assert/assert.c:101
      20:03:36 #12 0x00005b938a836ea0 in innobase_commit (hton=<optimized out>, thd=0x7b20e0000dc8, commit_trx=<optimized out>) at /tmp/workspace/mariadb-10.6-build/storage/innobase/handler/ha_innodb.cc:4718
      20:03:36 #13 0x00005b938a460c95 in commit_one_phase_2 (thd=thd@entry=0x7b20e0000dc8, all=all@entry=false, trans=trans@entry=0x7b20e00047b8, is_real_trans=is_real_trans@entry=true) at /tmp/workspace/mariadb-10.6-build/sql/handler.cc:2155
      20:03:36 #14 0x00005b938a4627f8 in ha_commit_trans (thd=thd@entry=0x7b20e0000dc8, all=all@entry=false) at /tmp/workspace/mariadb-10.6-build/sql/handler.cc:1973
      20:03:36 #15 0x00005b938a2fe869 in trans_commit_stmt (thd=thd@entry=0x7b20e0000dc8) at /tmp/workspace/mariadb-10.6-build/sql/transaction.cc:497
      20:03:36 #16 0x00005b938a180d23 in mysql_execute_command (thd=thd@entry=0x7b20e0000dc8, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /tmp/workspace/mariadb-10.6-build/sql/sql_parse.cc:6215
      20:03:36 #17 0x00005b938a1820f0 in mysql_parse (thd=thd@entry=0x7b20e0000dc8, rawbuf=rawbuf@entry=0x7b20e0014630 "INSERT INTO t2 VALUES(1)", length=length@entry=24, parser_state=parser_state@entry=0x7b2150932240) at /tmp/workspace/mariadb-10.6-build/sql/sql_parse.cc:8199
      20:03:36 #18 0x00005b938a182f2a in wsrep_mysql_parse (thd=thd@entry=0x7b20e0000dc8, rawbuf=0x7b20e0014630 "INSERT INTO t2 VALUES(1)", length=24, parser_state=parser_state@entry=0x7b2150932240) at /tmp/workspace/mariadb-10.6-build/sql/sql_parse.cc:8010
      20:03:36 #19 0x00005b938a1847df in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x7b20e0000dc8, packet=packet@entry=0x7b20e000bc29 "INSERT INTO t2 VALUES(1)", packet_length=packet_length@entry=24, blocking=blocking@entry=true) at /tmp/workspace/mariadb-10.6-build/sql/sql_class.h:242
      20:03:36 #20 0x00005b938a18740d in do_command (thd=thd@entry=0x7b20e0000dc8, blocking=blocking@entry=true) at /tmp/workspace/mariadb-10.6-build/sql/sql_parse.cc:1422
      

      Attachments

        Issue Links

          Activity

            The assertion is failing, because InnoDB does not expect ha_commit_trans(thd, false) to be called on a transaction that is in the XA PREPARE state. The only allowed actions should be XA COMMIT and XA ROLLBACK. Here the INSERT seems to be executing on an invalid transaction object, on which XA PREPARE had already been executed.

            marko Marko Mäkelä added a comment - The assertion is failing, because InnoDB does not expect ha_commit_trans(thd, false) to be called on a transaction that is in the XA PREPARE state. The only allowed actions should be XA COMMIT and XA ROLLBACK . Here the INSERT seems to be executing on an invalid transaction object, on which XA PREPARE had already been executed.

            The following test case reproduces the issue, when it runs with binlog enabled.

            CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB;
            SET SQL_LOG_BIN=OFF;
            INSERT INTO t1 VALUES(1);  # Assertion `commit_trx' failed
            

            Setting sql_log_bin = off, disables writes to binlog. However, unlike native replication, Galera will still replicate changes (since MDEV-7205) even if binlog is disabled. Setting sql_log_bin=off, will only "partially" disable the binlog.
            I observed that as a result, the INSERT above involves binlog + innodb, thus requiring internal 2 phase commit. So the is INSERT is first prepared, which will make it transition to PREPARED state in innodb, and later committed which causes the new assertion to fail.
            Running the same test with sql_log_bin enabled also results in 2PC, but the execution has one more step for ordered commit, between prepare and commit. Ordered commit causes the transaction state to transition back to TRX_STATE_NOT_STARTED. Thus avoiding the assertion.
            A potential fix is to avoid 2PC when sql_log_bin=off.

            sciascid Daniele Sciascia added a comment - The following test case reproduces the issue, when it runs with binlog enabled. CREATE TABLE t1 (f1 INTEGER PRIMARY KEY) ENGINE=InnoDB; SET SQL_LOG_BIN=OFF; INSERT INTO t1 VALUES(1); # Assertion `commit_trx' failed Setting sql_log_bin = off, disables writes to binlog. However, unlike native replication, Galera will still replicate changes (since MDEV-7205 ) even if binlog is disabled. Setting sql_log_bin=off, will only "partially" disable the binlog. I observed that as a result, the INSERT above involves binlog + innodb, thus requiring internal 2 phase commit. So the is INSERT is first prepared, which will make it transition to PREPARED state in innodb, and later committed which causes the new assertion to fail. Running the same test with sql_log_bin enabled also results in 2PC, but the execution has one more step for ordered commit, between prepare and commit. Ordered commit causes the transaction state to transition back to TRX_STATE_NOT_STARTED. Thus avoiding the assertion. A potential fix is to avoid 2PC when sql_log_bin=off.

            It seems that this is actually waiting for review by someone who is familiar with replication.

            marko Marko Mäkelä added a comment - It seems that this is actually waiting for review by someone who is familiar with replication.
            janlindstrom Jan Lindström added a comment - https://github.com/MariaDB/server/pull/3753
            sysprg Julius Goryavsky added a comment - sciascid janlindstrom Thanks, the fix has been merged with the head revision: https://github.com/MariaDB/server/commit/d698b784c8bc3492c2ac7883a14bb7e824362bf1
            sysprg Julius Goryavsky added a comment - The fix has been merged with the head revision: https://github.com/MariaDB/server/commit/d698b784c8bc3492c2ac7883a14bb7e824362bf1

            People

              sysprg Julius Goryavsky
              sciascid Daniele Sciascia
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.