[MDEV-34218] Mariadb Galera cluster fails when replicating from Mysql 5.7 on use of DDL - Jira

Edward Stoever created issue - 2024-05-22 20:27

Edward Stoever made changes - 2024-05-22 20:32

Field	Original Value	New Value
Description	Similar to ~~MDEV-34180~~ CREATE TABLE `tt_mem` ( `mem_id` bigint(20) NOT NULL, `mem_description` varchar(100) DEFAULT NULL, PRIMARY KEY (`mem_id`) ) ENGINE=MEMORY DEFAULT CHARSET=utf8mb4;	Similar to previous Jira issue ~~MDEV-34180~~ HOST: mysql, mysql 5.7.44 binlog_format=ROW HOST: m1, mariadb 10.6 GALERA NODE replicating from HOST mysql, Using_Gtid: No (log file and position) HOST: m2 mariadb 10.6 GALERA NODE HOST: m3 mariadb 10.6 GALERA NODE Run this command on mysql: {NOFORMAT} CREATE TABLE `tt_mem` ( `mem_id` bigint(20) NOT NULL, `mem_description` varchar(100) DEFAULT NULL, PRIMARY KEY (`mem_id`) ) ENGINE=MEMORY DEFAULT CHARSET=utf8mb4; {NOFORMAT} Error on m1: {noformat} 2024-05-22 16:11:07 1 [ERROR] WSREP: Vote 0 (success) on 78cebda7-1876-11ef-896b-8a58fca50d36:2565 is inconsistent with group. Leaving cluster. {noformat} Error on m2 and m3: {noformat} 2024-05-22 16:11:06 2 [ERROR] Error in Log_event::read_log_event(): 'Found invalid event in binary log', data_len: 42, event_type: -94 2024-05-22 16:11:06 2 [ERROR] WSREP: applier could not read binlog event, seqno: 2565, len: 482 {noformat}

Edward Stoever added a comment - 2024-05-22 20:54

A workaround is to replicate from mysql to a stand-alone mariadb with log_slave_updates=ON then from the stand-alone replicate to the Mariadb Galera cluster. This works without error.

Edward Stoever added a comment - 2024-05-22 20:54 A workaround is to replicate from mysql to a stand-alone mariadb with log_slave_updates=ON then from the stand-alone replicate to the Mariadb Galera cluster. This works without error.

JiraAutomate made changes - 2024-05-23 07:33

Fix Version/s		10.4 [ 22408 ]
Fix Version/s		10.5 [ 23123 ]
Fix Version/s		10.6 [ 24028 ]

Edward Stoever made changes - 2024-05-23 15:06

Summary

Mariadb Galera cluster fails when replicating from Mysql 5.7 on use of CREATE TABLE ENGINE=MEMORY command

Mariadb Galera cluster fails when replicating from Mysql 5.7 on use of DML

Edward Stoever made changes - 2024-05-23 15:08

Description

Similar to previous Jira issue ~~MDEV-34180~~

HOST: *mysql*, mysql 5.7.44 binlog_format=ROW
HOST: *m1*, mariadb 10.6 GALERA NODE replicating from HOST mysql, Using_Gtid: No (log file and position)
HOST: *m2* mariadb 10.6 GALERA NODE
HOST: *m3* mariadb 10.6 GALERA NODE

Run this command on mysql:
{NOFORMAT}
CREATE TABLE `tt_mem` (
  `mem_id` bigint(20) NOT NULL,
  `mem_description` varchar(100) DEFAULT NULL,
  PRIMARY KEY (`mem_id`)
) ENGINE=MEMORY DEFAULT CHARSET=utf8mb4;
{NOFORMAT}

Error on m1:
{noformat}
2024-05-22 16:11:07 1 [ERROR] WSREP: Vote 0 (success) on 78cebda7-1876-11ef-896b-8a58fca50d36:2565 is inconsistent with group. Leaving cluster.
{noformat}
Error on m2 and m3:
{noformat}
2024-05-22 16:11:06 2 [ERROR] Error in Log_event::read_log_event(): 'Found invalid event in binary log', data_len: 42, event_type: -94
2024-05-22 16:11:06 2 [ERROR] WSREP: applier could not read binlog event, seqno: 2565, len: 482
{noformat}

Similar to previous Jira issue ~~MDEV-34180~~

HOST: *mysql*, mysql 5.7.44 binlog_format=ROW
HOST: *m1*, mariadb 10.6 GALERA NODE replicating from HOST mysql, Using_Gtid: No (log file and position)
HOST: *m2* mariadb 10.6 GALERA NODE
HOST: *m3* mariadb 10.6 GALERA NODE

Run this command on mysql:
{NOFORMAT}
CREATE TABLE `tt_mem` (
  `mem_id` bigint(20) NOT NULL,
  `mem_description` varchar(100) DEFAULT NULL,
  PRIMARY KEY (`mem_id`)
) ENGINE=MEMORY DEFAULT CHARSET=utf8mb4;
{NOFORMAT}

Error on m1:
{noformat}
2024-05-22 16:11:07 1 [ERROR] WSREP: Vote 0 (success) on 78cebda7-1876-11ef-896b-8a58fca50d36:2565 is inconsistent with group. Leaving cluster.
{noformat}
Error on m2 and m3:
{noformat}
2024-05-22 16:11:06 2 [ERROR] Error in Log_event::read_log_event(): 'Found invalid event in binary log', data_len: 42, event_type: -94
2024-05-22 16:11:06 2 [ERROR] WSREP: applier could not read binlog event, seqno: 2565, len: 482
{noformat}

*OTHER COMMANDS THAT PRODUCE SIMILAR ERRORS*
{noformat}
CREATE TABLE `aa_tab` (
  `col1` int(11) NOT NULL,
  `col2` varchar(100) DEFAULT NULL,
  PRIMARY KEY (`col1`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
{noformat}

{noformat}
rename table aa_tab to dd_tab;
{noformat}

{noformat}
alter table aa_tab add column col3 varchar(10) after col2;
{noformat}

{noformat}
drop table aa_tab;
{noformat}

Julien Fritsch made changes - 2024-05-23 15:26

Fix Version/s	10.4 [ 22408 ]
Fix Version/s	10.5 [ 23123 ]

Julien Fritsch made changes - 2024-05-23 15:26

Affects Version/s

10.6 [ 24028 ]

Julien Fritsch made changes - 2024-05-23 15:26

Assignee

Julius Goryavsky [ sysprg ]

Eirik Øverby added a comment - 2024-05-28 22:09 - edited

I reported this issue.
Note that it also affects 10.11 - which is the version we use.
/Eirik

Eirik Øverby added a comment - 2024-05-28 22:09 - edited I reported this issue. Note that it also affects 10.11 - which is the version we use. /Eirik

Julien Fritsch made changes - 2024-05-31 15:59

Affects Version/s

10.11 [ 27614 ]

Julien Fritsch made changes - 2024-05-31 15:59

Fix Version/s

10.11 [ 27614 ]

Julien Fritsch made changes - 2024-05-31 15:59

Status

Open [ 1 ]

Confirmed [ 10101 ]

Edward Stoever made changes - 2024-06-04 17:40

Description

Similar to previous Jira issue ~~MDEV-34180~~

HOST: *mysql*, mysql 5.7.44 binlog_format=ROW
HOST: *m1*, mariadb 10.6 GALERA NODE replicating from HOST mysql, Using_Gtid: No (log file and position)
HOST: *m2* mariadb 10.6 GALERA NODE
HOST: *m3* mariadb 10.6 GALERA NODE

Run this command on mysql:
{NOFORMAT}
CREATE TABLE `tt_mem` (
  `mem_id` bigint(20) NOT NULL,
  `mem_description` varchar(100) DEFAULT NULL,
  PRIMARY KEY (`mem_id`)
) ENGINE=MEMORY DEFAULT CHARSET=utf8mb4;
{NOFORMAT}

Error on m1:
{noformat}
2024-05-22 16:11:07 1 [ERROR] WSREP: Vote 0 (success) on 78cebda7-1876-11ef-896b-8a58fca50d36:2565 is inconsistent with group. Leaving cluster.
{noformat}
Error on m2 and m3:
{noformat}
2024-05-22 16:11:06 2 [ERROR] Error in Log_event::read_log_event(): 'Found invalid event in binary log', data_len: 42, event_type: -94
2024-05-22 16:11:06 2 [ERROR] WSREP: applier could not read binlog event, seqno: 2565, len: 482
{noformat}

*OTHER COMMANDS THAT PRODUCE SIMILAR ERRORS*
{noformat}
CREATE TABLE `aa_tab` (
  `col1` int(11) NOT NULL,
  `col2` varchar(100) DEFAULT NULL,
  PRIMARY KEY (`col1`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
{noformat}

{noformat}
rename table aa_tab to dd_tab;
{noformat}

{noformat}
alter table aa_tab add column col3 varchar(10) after col2;
{noformat}

{noformat}
drop table aa_tab;
{noformat}

Similar to previous Jira issue ~~MDEV-34180~~

HOST: *mysql*, mysql 5.7.44 binlog_format=ROW
HOST: *m1*, mariadb 10.6 GALERA NODE replicating from HOST mysql, Using_Gtid: No (log file and position)
HOST: *m2* mariadb 10.6 GALERA NODE
HOST: *m3* mariadb 10.6 GALERA NODE

Run this command on mysql:
{NOFORMAT}
CREATE TABLE `tt_mem` (
  `mem_id` bigint(20) NOT NULL,
  `mem_description` varchar(100) DEFAULT NULL,
  PRIMARY KEY (`mem_id`)
) ENGINE=MEMORY DEFAULT CHARSET=utf8mb4;
{NOFORMAT}

Error on m1:
{noformat}
2024-05-22 16:11:07 1 [ERROR] WSREP: Vote 0 (success) on 78cebda7-1876-11ef-896b-8a58fca50d36:2565 is inconsistent with group. Leaving cluster.
{noformat}
Error on m2 and m3:
{noformat}
2024-05-22 16:11:06 2 [ERROR] Error in Log_event::read_log_event(): 'Found invalid event in binary log', data_len: 42, event_type: -94
2024-05-22 16:11:06 2 [ERROR] WSREP: applier could not read binlog event, seqno: 2565, len: 482
{noformat}

*OTHER COMMANDS THAT PRODUCE SIMILAR ERRORS*
{noformat}
CREATE TABLE `aa_tab` (
  `col1` int(11) NOT NULL,
  `col2` varchar(100) DEFAULT NULL,
  PRIMARY KEY (`col1`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
{noformat}

{noformat}
rename table aa_tab to dd_tab;
{noformat}

{noformat}
alter table aa_tab add column col3 varchar(10) after col2;
{noformat}

{noformat}
drop table aa_tab;
{noformat}

{noformat}
grant select on *.* to `edward`@`%`;
{noformat}

Edward Stoever added a comment - 2024-06-04 17:41

I just added GRANT to the list of commands.

Edward Stoever added a comment - 2024-06-04 17:41 I just added GRANT to the list of commands.

Jira Automation (IT) made changes - 2024-07-04 00:11

Zendesk Related Tickets

202508

Kristian Nielsen added a comment - 2024-10-24 14:25

Linking issue that look like they could be the same underlying root cause

Kristian Nielsen added a comment - 2024-10-24 14:25 Linking issue that look like they could be the same underlying root cause

Kristian Nielsen made changes - 2024-10-24 14:25

Link

This issue relates to ~~MDEV-34180~~ [ ~~MDEV-34180~~ ]

Kristian Nielsen added a comment - 2024-10-24 14:31

The "event_type: -94" in the error message on nodes m2 and m3 refers to a GTID_EVENT. This is the event that begins all event groups (InnoDB transactions or DDL/GRANT/etc.), and exists in the MariaDB binlogs but not in the MySQL binlogs. So the node m1 would not have seen an event type -94/GTID_EVENT (It would see a corresponding MySQL BEGIN or GTID event), and presumably m1 generated a GTID_EVENT instead to send to nodes m2/m3. So it is possible that this difference in starting event for the event group triggered the bug on m2/m3.

(Just an observation, I have not analysed in detail what goes wrong in the Galera code).

Kristian Nielsen added a comment - 2024-10-24 14:31 The "event_type: -94" in the error message on nodes m2 and m3 refers to a GTID_EVENT. This is the event that begins all event groups (InnoDB transactions or DDL/GRANT/etc.), and exists in the MariaDB binlogs but not in the MySQL binlogs. So the node m1 would not have seen an event type -94/GTID_EVENT (It would see a corresponding MySQL BEGIN or GTID event), and presumably m1 generated a GTID_EVENT instead to send to nodes m2/m3. So it is possible that this difference in starting event for the event group triggered the bug on m2/m3. (Just an observation, I have not analysed in detail what goes wrong in the Galera code).

Edward Stoever made changes - 2024-12-05 20:28

Summary

Mariadb Galera cluster fails when replicating from Mysql 5.7 on use of DML

Mariadb Galera cluster fails when replicating from Mysql 5.7 on use of DDL

Ralf Gebhardt made changes - 2025-01-08 14:46

Status

Confirmed [ 10101 ]

In Progress [ 3 ]

Jan Lindström added a comment - 2025-01-08 14:47

https://github.com/MariaDB/server/pull/3736

Jan Lindström added a comment - 2025-01-08 14:47 https://github.com/MariaDB/server/pull/3736

Jan Lindström made changes - 2025-01-08 14:50

Assignee

Julius Goryavsky [ sysprg ]

Jan Lindström [ JIRAUSER53125 ]

Jan Lindström made changes - 2025-01-08 14:50

Status

In Progress [ 3 ]

Stalled [ 10000 ]

Jan Lindström made changes - 2025-01-08 14:50

Assignee	Jan Lindström [ JIRAUSER53125 ]	Julius Goryavsky [ sysprg ]
Status	Stalled [ 10000 ]	In Review [ 10002 ]

Jan Lindström made changes - 2025-01-08 17:58

Link

This issue duplicates ~~MDEV-34412~~ [ ~~MDEV-34412~~ ]

Jan Lindström made changes - 2025-01-08 18:00

Link

This issue duplicates ~~MDEV-32366~~ [ ~~MDEV-32366~~ ]

Hemant Dangi added a comment - 2025-01-09 06:03

The issue can be reproduced for mysql 5.7.44, 8.0.39 and 9.0.1.

The patch is tested for mysql 5.7.44, 8.0.39 and 9.0.1

Hemant Dangi added a comment - 2025-01-09 06:03 The issue can be reproduced for mysql 5.7.44, 8.0.39 and 9.0.1. The patch is tested for mysql 5.7.44, 8.0.39 and 9.0.1

Hemant Dangi made changes - 2025-01-09 06:07

Link

This issue duplicates ~~MDEV-34180~~ [ ~~MDEV-34180~~ ]

Hemant Dangi made changes - 2025-01-09 06:07

Link

This issue relates to ~~MDEV-34180~~ [ ~~MDEV-34180~~ ]

Julius Goryavsky added a comment - 2025-01-28 00:17

Thanks, the fix has been merged with the main revision: https://github.com/MariaDB/server/commit/c0b11e75fff9ea513cd6cccfe47837678cedb473

Julius Goryavsky added a comment - 2025-01-28 00:17 Thanks, the fix has been merged with the main revision: https://github.com/MariaDB/server/commit/c0b11e75fff9ea513cd6cccfe47837678cedb473

Julius Goryavsky made changes - 2025-01-28 00:17

Status

In Review [ 10002 ]

Stalled [ 10000 ]

Julius Goryavsky made changes - 2025-01-28 01:13

Component/s		Galera [ 10124 ]
Fix Version/s		10.5.28 [ 29952 ]
Fix Version/s	10.6 [ 24028 ]
Fix Version/s	10.11 [ 27614 ]
Resolution		Fixed [ 1 ]
Status	Stalled [ 10000 ]	Closed [ 6 ]

JiraAutomate made changes - 2025-01-28 01:13

Fix Version/s		10.6.21 [ 29953 ]
Fix Version/s		10.11.11 [ 29954 ]
Fix Version/s		11.4.5 [ 29956 ]
Fix Version/s		11.7.2 [ 29914 ]

MariaDB Server

Mariadb Galera cluster fails when replicating from Mysql 5.7 on use of DDL

Details

Description

Attachments

Issue Links

Activity

People

Dates

Git Integration