Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-4889

Manual and automated vacuum cleaning for on-disk data empty records

    XMLWordPrintable

    Details

      Description

      MCS has a notion of an empty value for columnar segment/token files and dictionaries. Empty values corresponds with empty record if treat the table data in row orientation. The empty records are the results of:

      1. DELETE DML statements.
      2. bulk insertion operations that leverage cpimport explicitly or implicitly, e.g. INSERT..SELECT uses cpimport to ingest the data internally.

      In case when the number of empty records is significant it is a disk space and CPU time waste.

      The project will deliver a functionality that allows to:

      • analyze empty values/records percentage in an extent, file, partition, table.
      • manually cleanup empty values to reduce disk space usage.
      • automaticaly cleanup empty values using a background worker.

      Here is the initial scenario that automated background worker should follow to automatically clean-up empty records in a partition:

      • take another partition to cleanup
        • take another extent in the partition choosen; lock it and store its original HWM;
          • open a segment file with the extent with a min(col width) in the partition
          • take a pointer to the last used value in the extent
          • while !eof of the min width extent
            • find another empty value in the extent
            • save block in Version Buffer if not yet
            • replace empty with the pointer
            • update pointer
            • reduce HWM droping empty values

      See FSM diagram for details.

      TBD Exceptions processing

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              toddstoffel Todd Stoffel
              Reporter:
              drrtuy Roman
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.