Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Cannot Reproduce
-
1.1.6
-
AWS EC2 m5.xlarge, Amazon Linux 2 AMI.
MariaDB data is housed on a EBS of type `io1` with 225 IOPS
Description
Our instance ran out of disk space because of unneeded binlogs filling up. After purging those and disabling the binlog, the performance of columnstore reads remained sluggish at best for two large tables (having 27 and 66 million rows). Queries that used to take somewhere between 0.5 and 1.5 seconds easily took between 30 and 90 seconds resulting in lots of timeouts on our applications. Rebooting the server didn't help either.
No issues were reported by `SHOW TABLE STATUS` nor `CHECK TABLE`.
In the end we created fresh tables and copied data with `INSERT INTO fresh SELECT FROM original` and then a `RENAME original TO old, fresh TO original` and performance was back to normal (fast).
Most probably, the original tables were damaged somehow when the server ran out of disk space, but we couldn't see this being reported other than having a performance degradation.