Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-32551

"Read semi-sync reply magic number error" warnings on master

Details

    Description

      The user gets a lot of these errors randomly in the error log:

      "Read semi-sync reply magic number error"

      This is probably some missing or out of sync packet as part of semi sync replication.

      We should try to repeat this by have our QA and benchmark teams run tests on a master and semi-sync slave to try to repeat the issue.

      Attachments

        1. skr.cnf
          0.2 kB
        2. skr.test
          3 kB

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32940 Semi-sync Slave Falsely Logs that Master has Semi-sync Disabled

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32943 Stopping Semi-sync Slave Reports that it is Falling Back to Async Mode

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32944 Rpl_semi_sync_slave_enabled sysvar Should Not Be Modifiable While IO Thread is Running

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32945 Rpl_semi_sync_master_status is ON Without Any Connected Slaves

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32947 Semi-sync with Multi-source Checksum Verification Failure

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32960 Semi-sync ACKed Transaction can Timeout and Switch Off Semi-sync with Multiple Replicas

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32961 Semi-sync Primary Segfaults on Net Error

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-33331 IO Thread Relay Log Inconsistent Statistics After MDEV-32551

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-33546 Rpl_semi_sync_slave_status is ON When Replication Is Not Configured

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-11853 semisync thread can be killed after sync binlog but before ACK in the sync state

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24773 slave_compressed_protocol doesn't work properly with semi-sync replication

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Task - A task that needs to be done. MDEV-28114 Semi-sync Master ACK Receiver Thread Can Error on COM_QUIT

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-28141 Slave crashes with Packets out of order when connecting to a shutting down master

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-28306 Slave I/O: Fatal error: Failed to run 'after_read_event' hook, Internal MariaDB error code: 1593

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-29369 rpl.rpl_semi_sync_shutdown_await_ack fails regularly with Result content mismatch

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-32365 detailize the semisync replication magic number error

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32385 Semi-Sync Ack_Receiver Thread Should Allow COM_QUIT Command

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-32792 Second Semi-sync Replica Can Hang at Connect Time

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-33327 rpl_seconds_behind_master_spike Sensitive to IO Thread Stop Position

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MXS-4792 Semi-sync replication through MaxScale causes errors on STOP SLAVE

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MXS-4793 Unknown command errors when semi-sync replication runs through MaxScale

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            monty Michael Widenius created issue -
            monty Michael Widenius made changes -
            Field Original Value New Value
            Assignee Roel Van de Paar [ roel ]
            monty Michael Widenius made changes -
            Labels CS0653958
            Roel Roel Van de Paar made changes -
            Roel Roel Van de Paar made changes -
            Roel Roel Van de Paar made changes -
            Roel Roel Van de Paar made changes -
            Roel Roel Van de Paar made changes -
            Roel Roel Van de Paar made changes -
            Roel Roel Van de Paar made changes -
            Assignee Roel Van de Paar [ roel ] Axel Schwenke [ axel ]
            Roel Roel Van de Paar made changes -
            Roel Roel Van de Paar made changes -
            Roel Roel Van de Paar made changes -
            Roel Roel Van de Paar made changes -
            Roel Roel Van de Paar made changes -
            Assignee Axel Schwenke [ axel ]
            Roel Roel Van de Paar made changes -
            Assignee Pandikrishnan Gurusamy [ JIRAUSER52131 ]
            julien.fritsch Julien Fritsch made changes -
            Status Open [ 1 ] Needs Feedback [ 10501 ]
            Roel Roel Van de Paar made changes -
            julien.fritsch Julien Fritsch made changes -
            Labels CS0653958
            julien.fritsch Julien Fritsch made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            Roel Roel Van de Paar made changes -
            Elkin Andrei Elkin made changes -
            Assignee Pandikrishnan Gurusamy [ JIRAUSER52131 ] Brandon Nesterenko [ JIRAUSER48702 ]
            Elkin Andrei Elkin made changes -
            Status Needs Feedback [ 10501 ] Open [ 1 ]
            monty Michael Widenius made changes -
            Comment [ [~Elkin] I tried the following MTR case, but was not able to repeat the issue:
            Please provide a working mtr case so that I can help working on this!

            If the issue happens because of some missing bytes in the protocol, it may be that the code in net_clear() is eating the response on master/slave side.

            I have tried to follow your instructions, but not yet succeed.
            Here is what I have at this point:

            --source include/master-slave.inc

            --let $rpl_connection_name= slave2
            --let $rpl_server_number= 2
            --source include/rpl_connect.inc

            --connection master

            delimiter //;
            create procedure p1(i int)
            begin
              declare k int default 0;
              while k < i do
               select sleep(0.1);
               SET GLOBAL rpl_semi_sync_slave_enabled= OFF;
            STOP SLAVE IO_THREAD;
               select sleep(0.001);
            start slave io_thread;
            SET GLOBAL rpl_semi_sync_slave_enabled= ON;
                  set k=k+1;
              end while;
            end//

            delimiter ;//

            --sync_slave_with_master

            --connection master

            set global rpl_semi_sync_master_enabled = ON;

            --connection slave1
            --send call p1(1000)
            --connection slave2
            --send call p1(1000)

            --connection slave1
            --reap
            SET GLOBAL rpl_semi_sync_slave_enabled= OFF;
            --connection slave2
            --reap
            SET GLOBAL rpl_semi_sync_slave_enabled= OFF;

            --connection master

            SET GLOBAL rpl_semi_sync_master_enabled= OFF;
            drop procedure p1;

            --sync_slave_with_master
            --connection master

            --source include/rpl_end.inc
            ]
            monty Michael Widenius made changes -
            Attachment skr.test [ 72422 ]
            monty Michael Widenius made changes -
            Attachment my.cnf [ 72423 ]
            monty Michael Widenius made changes -
            Attachment my.cnf [ 72423 ]
            monty Michael Widenius made changes -
            Attachment skr.cnf [ 72424 ]
            monty Michael Widenius made changes -
            Attachment skr.test [ 72422 ]
            monty Michael Widenius made changes -
            Attachment skr.test [ 72425 ]
            bnestere Brandon Nesterenko made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            bnestere Brandon Nesterenko made changes -
            monty Michael Widenius made changes -
            Fix Version/s 10.6.17 [ 29518 ]
            Fix Version/s 10.6 [ 24028 ]
            serg Sergei Golubchik made changes -
            Fix Version/s 10.6 [ 24028 ]
            Fix Version/s 10.6.17 [ 29518 ]
            bnestere Brandon Nesterenko made changes -
            Status In Progress [ 3 ] In Testing [ 10301 ]
            bnestere Brandon Nesterenko made changes -
            bnestere Brandon Nesterenko made changes -
            bnestere Brandon Nesterenko made changes -
            bnestere Brandon Nesterenko made changes -
            bnestere Brandon Nesterenko made changes -
            bnestere Brandon Nesterenko made changes -
            bnestere Brandon Nesterenko made changes -
            bnestere Brandon Nesterenko made changes -
            Status In Testing [ 10301 ] Stalled [ 10000 ]
            bnestere Brandon Nesterenko made changes -
            Assignee Brandon Nesterenko [ JIRAUSER48702 ] Michael Widenius [ monty ]
            serg Sergei Golubchik made changes -
            Fix Version/s 10.6.17 [ 29518 ]
            Fix Version/s 10.11.7 [ 29519 ]
            Fix Version/s 11.0.5 [ 29520 ]
            Fix Version/s 11.1.4 [ 29024 ]
            Fix Version/s 11.2.3 [ 29521 ]
            Fix Version/s 10.6 [ 24028 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            bnestere Brandon Nesterenko made changes -
            bnestere Brandon Nesterenko made changes -
            bnestere Brandon Nesterenko made changes -
            mariadb-jira-automation Jira Automation (IT) made changes -
            Zendesk Related Tickets 158556 183178

            People

              monty Michael Widenius
              monty Michael Widenius
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.