Uploaded image for project: 'MariaDB MaxScale'
  1. MariaDB MaxScale
  2. MXS-4785

KafkaCDC JSON conversion is taking most of the CPU time

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 6.4.10
    • 24.02.0
    • kafkacdc
    • None
    • MXS-SPRINT-192

    Description

      In MXS-4775 it was found that there are inefficiencies in the KafkaCDC event-to-json conversion pipeline. Here's the visualization of the profiling from MXS-4775.

      perf-replicator.svg

      Here's a quick list of improvements that can be done based on some quick profiling of the code I did locally.

      • The gtid_pos_t::to_string() function uses a std::stringstream to create the GTID string. Using mxb::cat() and std::to_string() might prove to be better. sprintf() could also behave better as it is known that the maximum length of a single GTID value is fixed.
      • The UTF8 checks done by the JSON library show up in profiling. This could be avoided by using json_string_nocheck() but the strings need to be validated to be UTF for this to be reliable. If the strings are known to be UTF8, the checks can be avoided.
      • Most of the time is spent in KafkaEventHandler::commit() where the JSON object is converted into a string. If the string form of the JSON payload is constructed in-place, the overall cost of the JSON conversion would be lower.

      Attachments

        Issue Links

          Activity

            People

              markus makela markus makela
              markus makela markus makela
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.