[MCOL-555] Selectively bulk load based on estimate rows Created: 2017-02-09  Updated: 2021-01-16  Resolved: 2021-01-16

Status: Closed
Project: MariaDB ColumnStore
Component/s: cpimport, MariaDB Server
Affects Version/s: None
Fix Version/s: N/A

Type: Bug Priority: Minor
Reporter: VAROQUI Stephane Assignee: Unassigned
Resolution: Won't Do Votes: 0
Labels: None


 Description   

INSERT INTO SELECT const const is triggering cpimport
This is a valid SQL syntax for a single row INSERT despite INSERT INTO VALUES is mostly used, using



 Comments   
Comment by David Thompson (Inactive) [ 2017-02-09 ]

If we can detect the constant case likely it's better to convert to insert values. Not sure if we can detect row count on the source to make this smarter and trigger on a row count threshold.

Comment by Andrew Hutchings (Inactive) [ 2017-02-09 ]

from the storage engine point of view we get a start_bulk_insert() command for INSERT...SELECT regardless if it is one row or millions, we don't know in advance how many we are getting. This is the point where we trigger cpimport. Then write_row() for all the rows and end_bulk_insert().

We could delay until several rows have been received but it is messy. There are already cpimport delay fixes in 1.1 and we have found a couple of potential places where INSERT...SELECT performance can be improved. For now we are probably better off targeting those.

If a user knows that only one row will be inserted they should turn off infinidb_use_import_for_batchinsert for their session. This will bypass cpimport and use the regular insert code path which will likely be faster for single rows.

Generated at Thu Feb 08 02:21:57 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.