It's probable that any LOAD DATA INFILE would have a problem if version buffer overflow. This should be looked into. But this has not yet been tested.
Build tested: 6.2.2-1 (#3496)
Could not reproduce the issue with 4GB version buffer, not even doubling the dataset size to 20gb orders.tbl.
I lowered the version buffer to 4MB, instead of 4GB, and I saw the issue again. The following kept coming up when the orders table is queried.
MariaDB [mytest]> select count
from orders;
ERROR 1815 (HY000): Internal error: CAL0006: MCS-2008: The version buffer overflowed. Increase VersionBufferFileSize or limit the rows to be processed.
MariaDB [mytest]> select count from orders;
ERROR 1815 (HY000): Internal error: CAL0006: MCS-2008: The version buffer overflowed. Increase VersionBufferFileSize or limit the rows to be processed.
Regardless of the settings of columnstore_cache_use_import and columnstore_use_import_for_batchinsert, version buffer overflow should result in a rollback and the table should be accessible.
Daniel: Please run a test where
columnstore_cache_inserts=OFF
columnstore_cache_use_import=OFF
columnstore_use_import_for_batchinsert=OFF
Run an LDI with 150000000 rows in an attempt to cause version buffer overflow. Report if the error is recovered from (rollback works).
If it fails to rollback correctly, re-assign this to me.
If it succeeds in the rollback, re-assign to Gagan.
In either case, let me know what happened.
Build tested: Build tests: 6.2.1-1 (#3379)
I confirmed with Mr. Hall that he want to load 15,000,000 rows, not 150,000,000 rows.
I executed the requested test on the same columnstore build which the issue was oringally reported on. It actually took 3 times of the dataset (45,000,000 rows) to cause the version buffer overflow error. When that happened, the subsequent SELECT statement worked fine.
I also repeated the original test (with (45,000,000 rows) on the same build, as well as on the latest from develop-6 (drone build 4287). Both still have the reported issue.