[MCOL-635] Add DML cpimport configurable row limit Created: 2017-03-24  Updated: 2019-07-10  Resolved: 2019-07-10

Status: Closed
Project: MariaDB ColumnStore
Component/s: cpimport, MDB Plugin
Affects Version/s: 1.1.0
Fix Version/s: Icebox

Type: Bug Priority: Major
Reporter: Andrew Hutchings (Inactive) Assignee: Andrew Hutchings (Inactive)
Resolution: Won't Fix Votes: 0
Labels: None

Issue Links:
Relates
relates to MCOL-267 TEXT and BLOB data types are not supp... Closed
Sprint: 2017-7, 2017-8

 Description   

At the moment for INSERT_SELECT and LDI we turn the row format into csv and pipe into cpimport. This piping only has a 1MB csv row buffer. BLOB lengths are doubled due to hexing so at most you can fit a single 512KB BLOB in a row before things error.

The DML commands don't realise something is wrong until the end of the bulk import. The table remains locked afterwards and needs manual unlocking.

I believe the best way to fix this is to have a row length counter which spots this and errors appropriately as well as having a configurable buffers size for the row.

Alternatively it might be possible to make the row buffer grow with the content.


Generated at Thu Feb 08 02:22:34 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.