[MXS-2264] Schema aware avrorouter ignores table renames Created: 2019-01-15  Updated: 2019-10-01  Resolved: 2019-10-01

Status: Closed
Project: MariaDB MaxScale
Component/s: avrorouter, cdc
Affects Version/s: 2.2.17
Fix Version/s: 2.4.3

Type: Bug Priority: Major
Reporter: Sergiy Kyrychenko Assignee: markus makela
Resolution: Fixed Votes: 0
Labels: None

Sprint: MXS-SPRINT-91

 Description   

We rely on https://github.com/github/gh-ost to make online schema changes in our MariaDB OLTP environment without downtime.

gh-ost like many other tools of this kind relies on atomic table renames in its operation.

It is critical for Maxscale CDC to process table renames similarly to ALTER TABLE statements, generate new/updated AVRO spec and continue converting events.

In current Maxscale version, renaming the table effectively breaks subsequent avro conversion.

To reproduce:

CREATE TABLE `rename_1` (
  `some_id` mediumint(8) unsigned DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4;
 
INSERT INTO `rename_1` (`some_id`)
VALUES
	(1),
	(2);
 
 
CREATE TABLE `rename_2` (
  `some_id` mediumint(8) unsigned DEFAULT NULL,
  `additional_field` mediumint(8) unsigned DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4;
 
INSERT INTO `rename_2` (`some_id`)
VALUES
	(1),
	(2);
 
RENAME TABLE rename_1 TO rename_old, rename_2 TO rename_1;
 
INSERT INTO `rename_1` (`some_id`, `additional_field`)
VALUES
	(3, 1);
 
INSERT INTO `rename_1` (`some_id`, `additional_field`)
VALUES
	(4, 2);

Table contents are:

some_id	additional_field
1	NULL
2	NULL
3	1
4	2

JSON output as follows and last two inserts (some_id =3 and some_id=4) never make it to avro file:

{"namespace": "MaxScaleChangeDataSchema.avro", "type": "record", "name": "ChangeRecord", "fields": [{"name": "domain", "type": "int"}, {"name": "server_id", "type": "int"}, {"name": "sequence", "type": "int"}, {"name": "event_number", "type": "int"}, {"name": "timestamp", "type": "int"}, {"name": "event_type", "type": {"type": "enum", "name": "EVENT_TYPES", "symbols": ["insert", "update_before", "update_after", "delete"]}}, {"name": "some_id", "type": ["null", "int"], "real_type": "mediumint", "length": 8}]}
{"domain": 0, "server_id": 11009, "sequence": 9784476, "event_number": 1, "timestamp": 1547530357, "event_type": "insert", "some_id": 1}
{"domain": 0, "server_id": 11009, "sequence": 9784500, "event_number": 1, "timestamp": 1547530360, "event_type": "insert", "some_id": 2}

Also, performing following insert into brand new table created by same rename does not result in avro spec generation for table `rename_old` so this new table gets completely ignored by Maxscale:

INSERT INTO `rename_old` (`some_id`)
VALUES
	(7);



 Comments   
Comment by Sergiy Kyrychenko [ 2019-02-01 ]

It's been 2 weeks since this issue has been filed. Is it possible to confirm and set proper priority on this?

Comment by markus makela [ 2019-02-01 ]

Based on your test case it doesn't work and it's pretty clear it needs to be fixed. The bug is assigned and will be fixed when other higher severity issues have been fixed.

Generated at Thu Feb 08 04:12:52 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.