Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-14014

Multi-Slave Replication Fail: bogus data in log event

Details

    • 10.2.14, 10.1.32

    Description

      After upgrading to 10.1.20, replication now intermittently stops (randomly and not all slaves of a master at the same time) with the following errors:

      2017-10-01 20:38:20 140079966804736 [ERROR] Master 'trades': Error reading packet from server: bogus data in log event; the first event 'mysql-bin.038184' at 156161357, the last event read from 'mysql-bin.038185' at 60018003, the last byte read from 'mysql-bin.038185' at 60018022. (server_errno=1236)
      2017-10-01 20:38:20 140079966804736 [ERROR] Master 'trades': Slave I/O: Got fatal error 1236 from master when reading data from binary log: 'bogus data in log event; the first event 'mysql-bin.038184' at 156161357, the last event read from 'mysql-bin.038185' at 60018003, the last byte read from 'mysql-bin.038185' at 60018022.', Internal MariaDB error code: 1236
      

      We can fix this by issuing a START SLAVE and everything works without issue but the regular replication failures occurs.

      It looks like upstream bug : https://bugs.mysql.com/bug.php?id=84752

      With setting slave_compressed_protocol=1 and change the sync_binlog value to off, it's working fine. This is not easily reproducible. As per the comment in upstream bug, able to reproduce only with high load on master server and multiple slaves (like 5 or 6)

      Attachments

        Issue Links

          Activity

            niljoshi Nilnandan Joshi created issue -
            elenst Elena Stepanova made changes -
            Field Original Value New Value
            Labels upstream
            elenst Elena Stepanova made changes -
            Fix Version/s 10.1 [ 16100 ]
            Assignee Andrei Elkin [ elkin ]
            serg Sergei Golubchik made changes -
            Description After upgrading to 10.1.20, replication now intermittently stops (randomly and not all slaves of a master at the same time) with the following errors:

            2017-10-01 20:38:20 140079966804736 [ERROR] Master 'trades': Error reading packet from server: bogus data in log event; the first event 'mysql-bin.038184' at 156161357, the last event read from 'mysql-bin.038185' at 60018003, the last byte read from 'mysql-bin.038185' at 60018022. (server_errno=1236)
            2017-10-01 20:38:20 140079966804736 [ERROR] Master 'trades': Slave I/O: Got fatal error 1236 from master when reading data from binary log: 'bogus data in log event; the first event 'mysql-bin.038184' at 156161357, the last event read from 'mysql-bin.038185' at 60018003, the last byte read from 'mysql-bin.038185' at 60018022.', Internal MariaDB error code: 1236

            We can fix this by issuing a START SLAVE and everything works without issue but the regular replication failures occurs.

            It looks like upstream bug : https://bugs.mysql.com/bug.php?id=84752

            With setting slave_compressed_protocol=1 and change the sync_binlog value to off, it's working fine. This is not easily reproducible. As per the comment in upstream bug, able to reproduce only with high load on master server and multiple slaves (like 5 or 6)
            After upgrading to 10.1.20, replication now intermittently stops (randomly and not all slaves of a master at the same time) with the following errors:
            {noformat}
            2017-10-01 20:38:20 140079966804736 [ERROR] Master 'trades': Error reading packet from server: bogus data in log event; the first event 'mysql-bin.038184' at 156161357, the last event read from 'mysql-bin.038185' at 60018003, the last byte read from 'mysql-bin.038185' at 60018022. (server_errno=1236)
            2017-10-01 20:38:20 140079966804736 [ERROR] Master 'trades': Slave I/O: Got fatal error 1236 from master when reading data from binary log: 'bogus data in log event; the first event 'mysql-bin.038184' at 156161357, the last event read from 'mysql-bin.038185' at 60018003, the last byte read from 'mysql-bin.038185' at 60018022.', Internal MariaDB error code: 1236
            {noformat}
            We can fix this by issuing a START SLAVE and everything works without issue but the regular replication failures occurs.

            It looks like upstream bug : https://bugs.mysql.com/bug.php?id=84752

            With setting slave_compressed_protocol=1 and change the sync_binlog value to off, it's working fine. This is not easily reproducible. As per the comment in upstream bug, able to reproduce only with high load on master server and multiple slaves (like 5 or 6)
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            serg Sergei Golubchik made changes -
            Sprint 10.1.30 [ 215 ]
            serg Sergei Golubchik made changes -
            Rank Ranked lower
            serg Sergei Golubchik made changes -
            Fix Version/s 10.2 [ 14601 ]
            Elkin Andrei Elkin made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            Elkin Andrei Elkin made changes -
            Sprint 10.1.30 [ 215 ] 10.1.30, 10.3.5-1 [ 215, 229 ]
            serg Sergei Golubchik made changes -
            Sprint 10.1.30, 10.3.5-1 [ 215, 229 ]
            Elkin Andrei Elkin made changes -
            Sprint 10.2.14 [ 229 ]
            Elkin Andrei Elkin made changes -
            Sprint 10.2.14 [ 229 ] 10.2.14, 10.1.32 [ 229, 235 ]
            Elkin Andrei Elkin made changes -
            Assignee Andrei Elkin [ elkin ] Sergei Golubchik [ serg ]
            Status In Progress [ 3 ] In Review [ 10002 ]
            serg Sergei Golubchik made changes -
            Assignee Sergei Golubchik [ serg ] Andrei Elkin [ elkin ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            Elkin Andrei Elkin made changes -
            Assignee Andrei Elkin [ elkin ] Sergei Golubchik [ serg ]
            Status Stalled [ 10000 ] In Review [ 10002 ]
            serg Sergei Golubchik made changes -
            Assignee Sergei Golubchik [ serg ] Andrei Elkin [ elkin ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            Elkin Andrei Elkin made changes -
            Status Stalled [ 10000 ] In Progress [ 3 ]
            Elkin Andrei Elkin made changes -
            Assignee Andrei Elkin [ elkin ] Sergei Golubchik [ serg ]
            Status In Progress [ 3 ] In Review [ 10002 ]
            Elkin Andrei Elkin made changes -
            Assignee Sergei Golubchik [ serg ] Andrei Elkin [ elkin ]
            Elkin Andrei Elkin made changes -
            Status In Review [ 10002 ] Stalled [ 10000 ]
            julien.fritsch Julien Fritsch made changes -
            Assignee Andrei Elkin [ elkin ] Sergei Golubchik [ serg ]
            serg Sergei Golubchik made changes -
            Labels upstream need_feedback upstream
            serg Sergei Golubchik made changes -
            Fix Version/s 10.1.35 [ 23116 ]
            Fix Version/s 10.2.17 [ 23111 ]
            Fix Version/s 10.3.8 [ 23113 ]
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.1 [ 16100 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            crishoj Christian Rishøj made changes -
            julien.fritsch Julien Fritsch made changes -
            Labels need_feedback upstream upstream
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 82945 ] MariaDB v4 [ 152947 ]
            mariadb-jira-automation Jira Automation (IT) made changes -
            Zendesk Related Tickets 184868

            People

              serg Sergei Golubchik
              niljoshi Nilnandan Joshi
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.