[MDEV-16146] MariaDB slave stops with incompatible heartbeat Created: 2018-05-11  Updated: 2021-05-04  Resolved: 2021-05-01

Status: Closed
Project: MariaDB Server
Component/s: Replication
Affects Version/s: 5.5.56
Fix Version/s: 10.2.38, 10.3.29, 10.4.19, 10.5.10

Type: Bug Priority: Major
Reporter: Diptesh Patel Assignee: Sujatha Sivakumar (Inactive)
Resolution: Fixed Votes: 0
Labels: replication
Environment:

Centos 7.4.1708


Attachments: PNG File Replication_diagram.png    
Issue Links:
Relates

 Description   

MariaDB slave stops with following errors.

180511 11:07:58 [ERROR] Slave I/O: Unexpected master's heartbeat data: heartbeat is not compatible with local info;the event's data: log_file_name mysql-bin.000009 log_pos 1054262041, Error_code: 1623
180511 11:07:58 [ERROR] Slave I/O: Relay log write failure: could not queue event from master, Error_code: 1595
180511 11:07:58 [Note] Slave I/O thread exiting, read up to log 'mysql-bin.000009', position 1053640539

I am using Master-Master and Master-Slave Replication using MariaDB servers. I am replicating all Databases on Master-Master Replication but there is only one databse replicating on slave servers.Slave servers ignores the query from another database which is not replicating there. All other Binlogs which from not replicating skips by slave servers. We have heavy traffic on Master server. Slaves only for select purpose. Master databse server's system load goes upto 10%.

NOTE: All the slaves are connected using Gigabit Ethernet(Private Network).

We are using 24 cores CPU and 32GB RAM on master and 16 cores CPU and 24GB RAM on each slave.

I am using following version of MariaDB server on my 3 servers
mysql Ver 15.1 Distrib 5.5.56-MariaDB, for Linux (x86_64) using readline 5.1

Operating system
CentOS Linux release 7.4.1708 (Core)

Following is diagram of replication



 Comments   
Comment by Sujatha Sivakumar (Inactive) [ 2021-03-19 ]

Hello Andrei,

Can you please review the fix for MDEV-16146.

Patch: https://github.com/MariaDB/server/commit/7c2ea6b963f8ee3894e25e1f479f9d09b5b766b9

Buildbot: http://buildbot.askmonty.org/buildbot/grid?category=main&branch=bb-10.5-sujatha

At present patch is implemented for 10.5 it needs to be implemented for all GA versions.

Thank you.

Comment by Andrei Elkin [ 2021-04-22 ]

Waiting for update to complete review.

Comment by Sujatha Sivakumar (Inactive) [ 2021-04-26 ]

Hello Andrei,

Thank you for the review comments. I have addressed them in the following patch.

https://github.com/MariaDB/server/commit/724862441d5e1a889a974b5fdb32fd27f409cf6f

BuildBot: http://buildbot.askmonty.org/buildbot/grid?category=main&branch=bb-10.5-sujatha

Please review them.

Thank you.

Comment by Andrei Elkin [ 2021-04-26 ]

Comments were made. Waiting for a new patch.

Comment by Sujatha Sivakumar (Inactive) [ 2021-04-28 ]

Hello Andrei,

Please find the new patch which addresses your latest review comments.

  • Improved commit message
  • Added check for file name length to 'is_valid'
  • s/extra_header/sub_header

https://github.com/MariaDB/server/commit/f2b685c491e0ebe12ac7d5475f33ef65146ca602
BuildBot Test is in progress: http://buildbot.askmonty.org/buildbot/grid?category=main&branch=bb-10.5-sujatha

Thank you.

Comment by Andrei Elkin [ 2021-04-29 ]

Thanks! The patch is free to go. Github suggests a small change to the commit message.

Comment by Sujatha Sivakumar (Inactive) [ 2021-05-01 ]

Fix is implemented in 10.2.38. No merge conflicts were observed upto 10.4.

Fix was cherry-picked to higher versions and tested.

10.3 patch: https://github.com/MariaDB/server/commit/4be2c8baae5cdff607e48fbfb3f76be87aa9fdbd
10.4 patch: https://github.com/MariaDB/server/commit/7d1062ff21fc939fcf62127df867067684bd885c

In 10.5, the changes in 'log_event.cc' have to be moved to 'log_event_server.cc'.
The patch is provided

10.5 patch: https://github.com/MariaDB/server/commit/1d9dbd895e3f358979df0b9c5f6f75a2a4c35c12

Generated at Thu Feb 08 08:26:43 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.