Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-6675

TupleBPS extent-count retry loop can run for hours and its error never fires

    XMLWordPrintable

Details

    • Bug
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • 23.02.24
    • None
    • PrimProc
    • None

    Description

      Summary: TupleBPS::run() waits for every column of the table to show the same number of extents, reloading the extent lists between tries:

      while (!goodExtentCount() && retryCounter++ < retryMax) { ... }
      if (retryCounter == retryMax) { error }

      When the counts never agree, the post-increment leaves retryCounter at retryMax + 1, so the error never fires and the query goes on scanning with inconsistent extent lists. The bound is also a count, not a time: each reload asks BRM for every column's extents. On a production 23.02 cluster with a ~200-column table, each reload took ~44 s, so the 1000 retries ran for ~12 hours. The query looked like a "never-completing" aggregation, and KILL had no effect because the loop never checks for cancellation.

      Change (in progress):

      • Track whether the counts agreed instead of comparing the counter.
      • Stop after 1000 retries or 50 s, whichever comes first (the 50 s the existing comment already intends), or as soon as the step is cancelled.
      • On giving up, fail the query with ERR_TUPLE_BPS and name the columns whose extent count differs from the most common one, so the bad partition can be found from the error alone.

      Behavior change: a query over a table whose extent counts stay uneven now fails after ~50 s instead of hanging and then running on inconsistent extents. The table still needs an extent map repair; this only makes the problem visible.

      Not covered: why the extent counts diverged in the first place.

      Status: branch tuplebps-extent-retry (d9ce09215). Compile-checked only.

      Attachments

        Activity

          People

            Unassigned Unassigned
            kyle.hutchinson Kyle Hutchinson
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.