Uploaded image for project: 'MariaDB MaxScale'
  1. MariaDB MaxScale
  2. MXS-1686

Direct Kafka Integration

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.13, Icebox
    • Fix Version/s: 2.5.0
    • Component/s: avrorouter, failover
    • Labels:
      None

      Description

      Currently, the CDC events are presented to the user via a particular channel, and a custom script has to pull those events, and then send them on to kafka (if using the kafka integration).

      It would be ideal if maxscale could directly send these events directly to kafka. Necessary information would be the kafka cluster address, the zookeeper address(es), and what tables (via regex, ideally both white and black lists) that should be sent.

      In theory, this could happen on the fly, and completely skip writing these events to disk.

      It would be nice if maxscale could use zookeeper to keep the current position of the last event sent, in order to not send duplicates after a crash.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              markus makela markus makela
              Reporter:
              schatt Drew Schatt
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Git Integration