Uploaded image for project: 'MariaDB MaxScale'
  1. MariaDB MaxScale
  2. MXS-7672

Use quantile sketches in diff

    XMLWordPrintable

Details

    • New Feature
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • None
    • None
    • diff
    • None

    Description

      Currently the diff router uses a sample buffer based on which a histogram is built. The bin count for the histogram depends on the data itself and extreme outlier values may cause massive bin counts to appear. This is because the bin count comes from either the Freedman-Diaconis rule or Sturges's formula and the bin count is (max - min) / bin_width which can be very large.

      Using DDSketch for the histogram implementation has a number of benefits:

      • guaranteed relative error values that are known in advance
      • configurable error (e.g. 0.5%, 1% or 2%)
      • fixed size
      • supports a larger range of values with less memory
      • outliers clamped to predefined maximums

      This also would make it possible to have rotation of the histograms which currently is not possible as the sampling is done at the start and at the end of the comparison, the whole workload might've drifted out of the sampled window.

      Attachments

        Activity

          People

            Unassigned Unassigned
            markus makela markus makela
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Time Tracking

                Estimated:
                Original Estimate - 3d
                3d
                Remaining:
                Remaining Estimate - 3d
                3d
                Logged:
                Time Spent - Not Specified
                Not Specified

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.