Details
-
Bug
-
Status: Closed (View Workflow)
-
Minor
-
Resolution: Not a Bug
-
22.08.0
-
RHEL 8
Description
Hi,
We're seeing a potential issue with KafkaCDC in a three instance setup of Maxscale, all data appears at the Kafka topic tagged as server_id=1 e.g. we don't see server_id=2 or server_id=3 in the payload. Unsure if it's related to Kafka partition keys or because we're using Galera/galeramon etc..,, however 3 copies of each change appears on the topic all tagged as server_id=1.
Example topic output below;
{"domain":1,"server_id":1,"sequence":11075901205,"event_number":1,"timestamp":1689904677,"event_type":"update_before","table_schema":"Merged_Datastore","table_name":"session",.........} {"domain":1,"server_id":1,"sequence":11075901205,"event_number":2,"timestamp":1689904677,"event_type":"update_after","table_schema":"Merged_Datastore","table_name":"session",.........}Setup details:
The Apache Kafka topic has been setup with 3 partitions and replication factor of 3 across 3 node cluster. We have a 3 node cluster of MariaDB with Galera repplication utilising binary logging & GTIDs. For redundancy we also have an instance of Maxscale on each MariaDB node.
Data is sent to a Kafka topic mds-cpvnf-process-notifier, setup on a container as per below;
$ podman exec -ti kafka-sidecar-ap /opt/kafka/bin/kafka-topics.sh --zookeeper 10.195.241.85:2181 --topic mds-cpvnf-process-notifier --create --partitions 3 --replication-factor 3 --config retention.ms=3600000
The my.cnf's, maxscale.cnf's & galera.cnf's from the 3 nodes is attached.
Thanks.
Attachments
Issue Links
- is blocked by
-
MDEV-30473 Do not allow GET_LOCK() / RELEASE_LOCK() in cluster
- Closed