Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-4952

Test failure code minimization

    XMLWordPrintable

Details

    • New Feature
    • Status: Open (View Workflow)
    • Minor
    • Resolution: Unresolved
    • None
    • Icebox
    • None
    • None

    Description

      Here in MCS we use at least two test suites to discover introduction of defects into new or changed code. Both test suites are not optimized for the discovery of defect sources, as one example, tests in MTR suite usually contain dozen(s) of DDL/DML and retrieval clauses and MTR itself does not do any analysis apart from providing diff.

      While these suites and tools are infinitely valuable as they are, they can be enhanced to provide more succinct representation of failure and even provide shorter test cases derived from failing tests. These changes would make life of developers easier.

      The simplest step to make life easier is to annotate MTR's query results with the line range of the test file of query being executed. Results will indicate failures such as completely empty results and also directly point to the queries that diverge. Right now developer has to manually count queries, look at the query results structure and logged results structure to be sure it is wrong one, etc, and then pinpoint problematic query. With the annotation of output we can directly go to the test file and tell "this is the query that has wrong results" or "this query returns empty set and this is why its results are not present".

      More complex way is to track queries executed and use difference in the results of MariaDB and MCS to figure out the least possible set of queries that diverge.

      For example, we may have a CREATE TABLE, some INSERTs and then some SELECTs. Problem in some these SELECTs can be due to interactions between INSERTs and problematic selects and between SELECTs themselves (both are related to extentmap global state). The tool can eliminate some of the queries preceding problematic ones and compare execution on the MariaDB/ISAM and on MCS. Minimal set of queries that diverge on these two engines can be reported to the developer as a special test case he or she can use to better find source of the defect.

      Attachments

        Activity

          People

            maxmether Max Mether
            sergey.zefirov Sergey Zefirov
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.