Uploaded image for project: 'MariaDB MaxScale'
  1. MariaDB MaxScale
  2. MXS-4678

Maxscale 3-node cluster with KafkaCDC sends payload all tagged as `"server_id":1,`.

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Minor
    • Resolution: Not a Bug
    • 22.08.0
    • N/A
    • kafkacdc
    • RHEL 8

    Description

      Hi,

      We're seeing a potential issue with KafkaCDC in a three instance setup of Maxscale, all data appears at the Kafka topic tagged as server_id=1 e.g. we don't see server_id=2 or server_id=3 in the payload. Unsure if it's related to Kafka partition keys or because we're using Galera/galeramon etc..,, however 3 copies of each change appears on the topic all tagged as server_id=1.

      Example topic output below;

      {"domain":1,"server_id":1,"sequence":11075901205,"event_number":1,"timestamp":1689904677,"event_type":"update_before","table_schema":"Merged_Datastore","table_name":"session",.........} {"domain":1,"server_id":1,"sequence":11075901205,"event_number":2,"timestamp":1689904677,"event_type":"update_after","table_schema":"Merged_Datastore","table_name":"session",.........}

      Setup details:
      The Apache Kafka topic has been setup with 3 partitions and replication factor of 3 across 3 node cluster. We have a 3 node cluster of MariaDB with Galera repplication utilising binary logging & GTIDs. For redundancy we also have an instance of Maxscale on each MariaDB node.

      Data is sent to a Kafka topic mds-cpvnf-process-notifier, setup on a container as per below;
      $ podman exec -ti kafka-sidecar-ap /opt/kafka/bin/kafka-topics.sh --zookeeper 10.195.241.85:2181 --topic mds-cpvnf-process-notifier --create --partitions 3 --replication-factor 3 --config retention.ms=3600000

      The my.cnf's, maxscale.cnf's & galera.cnf's from the 3 nodes is attached.

      Thanks.

      Attachments

        1. galera-1.txt
          0.7 kB
        2. galera-2.txt
          0.7 kB
        3. galera-3.txt
          0.7 kB
        4. mariadb-1.txt
          3 kB
        5. mariadb-2.txt
          3 kB
        6. mariadb-3.txt
          3 kB
        7. maxscale-server-1.txt
          5 kB
        8. maxscale-server-2.txt
          3 kB
        9. maxscale-server-3.txt
          3 kB

        Issue Links

          Activity

            People

              markus makela markus makela
              Presnickety Presnickety
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.