Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
2.2.17
-
None
-
MXS-SPRINT-91
Description
We rely on https://github.com/github/gh-ost to make online schema changes in our MariaDB OLTP environment without downtime.
gh-ost like many other tools of this kind relies on atomic table renames in its operation.
It is critical for Maxscale CDC to process table renames similarly to ALTER TABLE statements, generate new/updated AVRO spec and continue converting events.
In current Maxscale version, renaming the table effectively breaks subsequent avro conversion.
To reproduce:
CREATE TABLE `rename_1` (
|
`some_id` mediumint(8) unsigned DEFAULT NULL
|
) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4;
|
|
|
INSERT INTO `rename_1` (`some_id`)
|
VALUES
|
(1),
|
(2);
|
|
|
|
|
CREATE TABLE `rename_2` (
|
`some_id` mediumint(8) unsigned DEFAULT NULL,
|
`additional_field` mediumint(8) unsigned DEFAULT NULL
|
) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4;
|
|
|
INSERT INTO `rename_2` (`some_id`)
|
VALUES
|
(1),
|
(2);
|
|
|
RENAME TABLE rename_1 TO rename_old, rename_2 TO rename_1;
|
|
|
INSERT INTO `rename_1` (`some_id`, `additional_field`)
|
VALUES
|
(3, 1);
|
|
|
INSERT INTO `rename_1` (`some_id`, `additional_field`)
|
VALUES
|
(4, 2);
|
Table contents are:
some_id additional_field
|
1 NULL
|
2 NULL
|
3 1
|
4 2
|
JSON output as follows and last two inserts (some_id =3 and some_id=4) never make it to avro file:
{"namespace": "MaxScaleChangeDataSchema.avro", "type": "record", "name": "ChangeRecord", "fields": [{"name": "domain", "type": "int"}, {"name": "server_id", "type": "int"}, {"name": "sequence", "type": "int"}, {"name": "event_number", "type": "int"}, {"name": "timestamp", "type": "int"}, {"name": "event_type", "type": {"type": "enum", "name": "EVENT_TYPES", "symbols": ["insert", "update_before", "update_after", "delete"]}}, {"name": "some_id", "type": ["null", "int"], "real_type": "mediumint", "length": 8}]}
|
{"domain": 0, "server_id": 11009, "sequence": 9784476, "event_number": 1, "timestamp": 1547530357, "event_type": "insert", "some_id": 1}
|
{"domain": 0, "server_id": 11009, "sequence": 9784500, "event_number": 1, "timestamp": 1547530360, "event_type": "insert", "some_id": 2}
|
Also, performing following insert into brand new table created by same rename does not result in avro spec generation for table `rename_old` so this new table gets completely ignored by Maxscale:
INSERT INTO `rename_old` (`some_id`)
|
VALUES
|
(7);
|