Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-34064

ERROR 12607 (HY000): This xid is not exist and Assertion `!is_set() || (m_status == DA_OK_BULK && is_bulk_op())' failed from change_master()

Details

    Description

      # mysqld options required for replay: --log-bin 
      INSTALL PLUGIN Spider SONAME 'ha_spider.so';
      CREATE TABLE t (c TEXT) ENGINE=Spider;
      XA BEGIN 'a';
      UPDATE t SET c=+1;
      CHANGE MASTER TO master_host='a',master_port=1,master_user='a',master_demote_to_slave=1;
      

      Leads to, on optimized:

      11.5.0 3f9182126c64bcec359bebe9ebad2a0e559b13e2 (Optimized)

      11.5.0-opt>CHANGE MASTER TO master_host='a',master_port=1,master_user='a',master_demote_to_slave=1;
      ERROR 12607 (HY000): This xid is not exist
      

      And on debug to:

      11.5.0 3f9182126c64bcec359bebe9ebad2a0e559b13e2 (Debug)

      2024-05-03 17:37:48 5 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='', master_port='3306', master_log_file='', master_log_pos='4'. New state master_host='a', master_port='1', master_log_file='', master_log_pos='4'.
      2024-05-03 17:37:48 5 [Note] Previous Using_Gtid=Slave_Pos. New Using_Gtid=Slave_Pos
      mariadbd: /test/11.5_dbg_san/sql/sql_error.cc:335: void Diagnostics_area::set_ok_status(ulonglong, ulonglong, const char*): Assertion `!is_set() || (m_status == DA_OK_BULK && is_bulk_op())' failed.
      240503 17:37:48 [ERROR] mysqld got signal 6 ;
      

      11.5.0 3f9182126c64bcec359bebe9ebad2a0e559b13e2 (Debug)

      mariadbd: /test/11.5_dbg_san/sql/sql_error.cc:335: void Diagnostics_area::set_ok_status(ulonglong, ulonglong, const char*): Assertion `!is_set() || (m_status == DA_OK_BULK && is_bulk_op())' failed.
      

      11.5.0 3f9182126c64bcec359bebe9ebad2a0e559b13e2 (Debug)

      Core was generated by `/test/UBASAN_MD240424-mariadb-11.5.0-linux-x86_64-dbg/bin/mariadbd --no-default'.
      Program terminated with signal SIGABRT, Aborted.
      #0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:44
      [Current thread is 1 (LWP 221613)]
      (gdb) bt
      #0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:44
      #1  __pthread_kill_internal (signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:78
      #2  __GI___pthread_kill (threadid=<optimized out>, signo=6)at ./nptl/pthread_kill.c:89
      #3  0x000056070e6a72eb in my_write_core (sig=6)at /test/11.5_dbg_san/mysys/stacktrace.c:424
      #4  0x000056070bef2a3e in handle_fatal_signal (sig=<optimized out>)at /test/11.5_dbg_san/sql/signal_handler.cc:357
      #5  <signal handler called>
      #6  __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:44
      #7  __pthread_kill_internal (signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:78
      #8  __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6)at ./nptl/pthread_kill.c:89
      #9  0x000014f95a23c406 in __GI_raise (sig=sig@entry=6)at ../sysdeps/posix/raise.c:26
      #10 0x000014f95a22287c in __GI_abort () at ./stdlib/abort.c:79
      #11 0x000014f95a22279b in __assert_fail_base (fmt=0x14f95a3b9f68 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x56070efe5a20 "!is_set() || (m_status == DA_OK_BULK && is_bulk_op())", file=file@entry=0x56070efe5620 "/test/11.5_dbg_san/sql/sql_error.cc", line=line@entry=335, function=function@entry=0x56070efe5a80 "void Diagnostics_area::set_ok_status(ulonglong, ulonglong, const char*)") at ./assert/assert.c:92
      #12 0x000014f95a233b86 in __assert_fail (assertion=0x56070efe5a20 "!is_set() || (m_status == DA_OK_BULK && is_bulk_op())", file=0x56070efe5620 "/test/11.5_dbg_san/sql/sql_error.cc", line=335, function=0x56070efe5a80 "void Diagnostics_area::set_ok_status(ulonglong, ulonglong, const char*)") at ./assert/assert.c:101
      #13 0x000056070a3ec2a2 in Diagnostics_area::set_ok_status (this=this@entry=0x62b000172140, affected_rows=affected_rows@entry=0, last_insert_id=last_insert_id@entry=0, message=message@entry=0x0)at /test/11.5_dbg_san/sql/sql_error.cc:335
      #14 0x000056070a828694 in my_ok (message=0x0, id=0, affected_rows_arg=0, thd=<optimized out>) at /test/11.5_dbg_san/sql/sql_class.h:6009
      #15 change_master (thd=thd@entry=0x62b00016c218, mi=mi@entry=0x62a00000c200, master_info_added=master_info_added@entry=0x14f937dfc300)at /test/11.5_dbg_san/sql/sql_repl.cc:4351
      #16 0x000056070a68ebcc in mysql_execute_command (thd=thd@entry=0x62b00016c218, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false)at /test/11.5_dbg_san/sql/sql_parse.cc:4126
      #17 0x000056070a6b21c6 in mysql_parse (thd=thd@entry=0x62b00016c218, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x14f937dfd810)at /test/11.5_dbg_san/sql/sql_parse.cc:7815
      #18 0x000056070a6c2547 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x62b00016c218, packet=packet@entry=0x62900009b219 "CHANGE MASTER TO master_host='a',master_port=1,master_user='a',master_demote_to_slave=1", packet_length=packet_length@entry=87, blocking=blocking@entry=true)at /test/11.5_dbg_san/sql/sql_parse.cc:1892
      #19 0x000056070a6d1388 in do_command (thd=0x62b00016c218, blocking=blocking@entry=true) at /test/11.5_dbg_san/sql/sql_parse.cc:1405
      #20 0x000056070b10954c in do_handle_one_connection (connect=<optimized out>, connect@entry=0x608000002638, put_in_cache=put_in_cache@entry=true)at /test/11.5_dbg_san/sql/sql_connect.cc:1445
      #21 0x000056070b10aaf5 in handle_one_connection (arg=0x608000002638)at /test/11.5_dbg_san/sql/sql_connect.cc:1347
      #22 0x000014f95a28f18a in start_thread (arg=<optimized out>)at ./nptl/pthread_create.c:444
      #23 0x000014f95a31dbd0 in clone3 ()at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
      

      Please note the CHANGE MASTER command in the stack.

      Attachments

        Issue Links

          Activity

            MTR Testcase:

            --source include/have_binlog_format_mixed.inc
            INSTALL PLUGIN Spider SONAME 'ha_spider.so';
            CREATE TABLE t (c TEXT) ENGINE=Spider;
            XA BEGIN 'a';
            --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            UPDATE t SET c=+1;
            CHANGE MASTER TO master_host='a',master_port=1,master_user='a',master_demote_to_slave=1;
            SELECT SLEEP(5);
            

            Roel Roel Van de Paar added a comment - MTR Testcase: --source include/have_binlog_format_mixed.inc INSTALL PLUGIN Spider SONAME 'ha_spider.so' ; CREATE TABLE t (c TEXT) ENGINE=Spider; XA BEGIN 'a' ; --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE UPDATE t SET c=+1; CHANGE MASTER TO master_host= 'a' ,master_port=1,master_user= 'a' ,master_demote_to_slave=1; SELECT SLEEP(5);
            Roel Roel Van de Paar added a comment - - edited

            It is not a regression per se as master_demote_to_slave was introduced in 10.10 in MDEV-20122

            Roel Roel Van de Paar added a comment - - edited It is not a regression per se as master_demote_to_slave was introduced in 10.10 in MDEV-20122
            ycp Yuchen Pei added a comment - - edited

            Same kind of issue as MDEV-28105 and MDEV-27186, but instead of info(), it is rpl_slave_state::truncate_state_table() ignores error from ha_commit_trans().

            Here's a demo of what I mean. Does it point towards a sensible fix bnestere?

            bb-10.11-mdev-34064-ycp-demo dfcb8b380a186cd47efec1d31b1db7c8e4fe2943
            MDEV-34064 [demo] return error from ha_commit_trans() in rpl_slave_state::truncate_state_table(THD *thd)
             
            just a demo - a similar fix to MDEV-28105 and MDEV-27186.
             
            Whether the 12607 error makes sense is perhaps a different matter:
             
            mysqltest: At line 7: query 'CHANGE MASTER TO master_host='a',master_port=1,master_user='a',master_demote_to_slave=1' failed: <Unknown> (12607): This xid is not exist
             
            2 files changed, 13 insertions(+), 1 deletion(-)
            sql/rpl_gtid.cc                                           | 6 +++++-
            storage/spider/mysql-test/spider/bugfix/t/mdev_34064.test | 8 ++++++++
             
            modified   sql/rpl_gtid.cc
            @@ -461,7 +461,11 @@ rpl_slave_state::truncate_state_table(THD *thd)
                 {
                   ha_commit_trans(thd, FALSE);
                   close_thread_tables(thd);
            -      ha_commit_trans(thd, TRUE);
            +      if ((err= ha_commit_trans(thd, TRUE)))
            +      {
            +        thd->release_transactional_locks();
            +        return err;
            +      }
                 }
                 thd->release_transactional_locks();
               }
            new file   storage/spider/mysql-test/spider/bugfix/t/mdev_34064.test
            @@ -0,0 +1,8 @@
            +--source include/have_binlog_format_mixed.inc
            +INSTALL SONAME 'ha_spider.so';
            +CREATE TABLE t (c TEXT) ENGINE=Spider;
            +XA BEGIN 'a';
            +--error ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            +UPDATE t SET c=+1;
            +--error 12607
            +CHANGE MASTER TO master_host='a',master_port=1,master_user='a',master_demote_to_slave=1;
            

            ycp Yuchen Pei added a comment - - edited Same kind of issue as MDEV-28105 and MDEV-27186 , but instead of info(), it is rpl_slave_state::truncate_state_table() ignores error from ha_commit_trans(). Here's a demo of what I mean. Does it point towards a sensible fix bnestere ? bb-10.11-mdev-34064-ycp-demo dfcb8b380a186cd47efec1d31b1db7c8e4fe2943 MDEV-34064 [demo] return error from ha_commit_trans() in rpl_slave_state::truncate_state_table(THD *thd)   just a demo - a similar fix to MDEV-28105 and MDEV-27186.   Whether the 12607 error makes sense is perhaps a different matter:   mysqltest: At line 7: query 'CHANGE MASTER TO master_host='a',master_port=1,master_user='a',master_demote_to_slave=1' failed: <Unknown> (12607): This xid is not exist   2 files changed, 13 insertions(+), 1 deletion(-) sql/rpl_gtid.cc | 6 +++++- storage/spider/mysql-test/spider/bugfix/t/mdev_34064.test | 8 ++++++++   modified sql/rpl_gtid.cc @@ -461,7 +461,11 @@ rpl_slave_state::truncate_state_table(THD *thd) { ha_commit_trans(thd, FALSE); close_thread_tables(thd); - ha_commit_trans(thd, TRUE); + if ((err= ha_commit_trans(thd, TRUE))) + { + thd->release_transactional_locks(); + return err; + } } thd->release_transactional_locks(); } new file storage/spider/mysql-test/spider/bugfix/t/mdev_34064.test @@ -0,0 +1,8 @@ +--source include/have_binlog_format_mixed.inc +INSTALL SONAME 'ha_spider.so'; +CREATE TABLE t (c TEXT) ENGINE=Spider; +XA BEGIN 'a'; +--error ER_CONNECT_TO_FOREIGN_DATA_SOURCE +UPDATE t SET c=+1; +--error 12607 +CHANGE MASTER TO master_host='a',master_port=1,master_user='a',master_demote_to_slave=1;

            Hi ycp,

            I think the bug runs deeper than that (and not specific to spider), it seems CHANGE MASTER (CM) with master_demote_to_slave=1 will commit any ongoing transaction on the current thread.. If you make the DML within the XA transaction valid, you can see it commit in the binary log as soon as CM executes as a normal transaction. Then if you remove the master_demote_to_slave=1 option, it won't commit. I'll reassign to myself, because it isn't spider.

            Though the general idea of your fix seems quite good to have, though I don't think you need the if statement, you can just assign the commit (and rollback) to err as the function will then naturally release its trans locks and return.

            bnestere Brandon Nesterenko added a comment - Hi ycp , I think the bug runs deeper than that (and not specific to spider), it seems CHANGE MASTER (CM) with master_demote_to_slave=1 will commit any ongoing transaction on the current thread.. If you make the DML within the XA transaction valid, you can see it commit in the binary log as soon as CM executes as a normal transaction. Then if you remove the master_demote_to_slave=1 option, it won't commit. I'll reassign to myself, because it isn't spider. Though the general idea of your fix seems quite good to have, though I don't think you need the if statement, you can just assign the commit (and rollback) to err as the function will then naturally release its trans locks and return.

            People

              bnestere Brandon Nesterenko
              Roel Roel Van de Paar
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.