[MCOL-1075] Clarifications for the Bulk Write SDK documentation Created: 2017-12-04 Updated: 2023-10-26 Resolved: 2017-12-06 |
|
| Status: | Closed |
| Project: | MariaDB ColumnStore |
| Component/s: | Documentation |
| Affects Version/s: | 1.1.2 |
| Fix Version/s: | 1.1.3 |
| Type: | Task | Priority: | Trivial |
| Reporter: | Sasha V | Assignee: | David Thompson (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Environment: |
Streaming remotely to a multi-server installation |
||
| Issue Links: |
|
||||||||
| Sprint: | 2017-24 | ||||||||
| Description |
|
5.1.2 createBulkInsert() I expect that the future use variables mode and pm will provide the same functionality as cpimport. Could you please document the corresponding mode and PM used by default in the current Bulk Insert API? 5.2.6 commit() Since the ColumnStoreBulkInsert class can not be re-used upon the failed commit, a guidance on what to do upon the commit failure would be appreciated in this the section. 5.2.11 setBatchSize() Do not implement this function to avoid confusion with the RowsPerBatch value set in the Columnstore.xml It appears to be some inconsistency already, as the default RowsPerBatch in the Columnstore.xml is 10,000, while the SDK buffers up to 100,000 rows by default (despite setting the batchSize to 10,000). |
| Comments |
| Comment by Andrew Hutchings (Inactive) [ 2017-12-04 ] |
|
Thank you for your input. We will take these edits on-board. As far as batch size goes, 10,000 is more optimal for cpimport whereas at least 100,000 is more optimal for the API (there is a trade-off between RAM and network rtt/compression time). This is due to the different ways they handle network communications. In addition we want to have a way for the API to not rely on Columnstore.xml in a future release. This means that you would need a way to set the batch size anyway. We will make sure this is documented. |
| Comment by Andrew Hutchings (Inactive) [ 2017-12-05 ] |
|
Two minor document fixes in pull request |
| Comment by David Thompson (Inactive) [ 2017-12-06 ] |
|
verified the changes are in the generated doc. |