Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-1997

Performance degradation after running out of disk space

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Cannot Reproduce
    • 1.1.6
    • N/A
    • N/A
    • AWS EC2 m5.xlarge, Amazon Linux 2 AMI.
      MariaDB data is housed on a EBS of type `io1` with 225 IOPS

    Description

      Our instance ran out of disk space because of unneeded binlogs filling up. After purging those and disabling the binlog, the performance of columnstore reads remained sluggish at best for two large tables (having 27 and 66 million rows). Queries that used to take somewhere between 0.5 and 1.5 seconds easily took between 30 and 90 seconds resulting in lots of timeouts on our applications. Rebooting the server didn't help either.

      No issues were reported by `SHOW TABLE STATUS` nor `CHECK TABLE`.

      In the end we created fresh tables and copied data with `INSERT INTO fresh SELECT FROM original` and then a `RENAME original TO old, fresh TO original` and performance was back to normal (fast).

      Most probably, the original tables were damaged somehow when the server ran out of disk space, but we couldn't see this being reported other than having a performance degradation.

      Attachments

        Activity

          People

            Unassigned Unassigned
            Jorick Jorick Schram
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.