[MCOL-990] provide a resetRow method Created: 2017-10-27  Updated: 2023-10-26  Resolved: 2017-11-01

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: 1.1.0
Fix Version/s: 1.1.1

Type: Task Priority: Critical
Reporter: David Thompson (Inactive) Assignee: David Thompson (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Sprint: 2017-22

 Description   

We should provide a 'reset row' method that allows you to effectively reset / cancel the current row write. This should always succeed and should have the effect that the temporary holding area for the current working row is emptied allowing the prior records in the batch to be committed.

What this would allow you to do is to commit the prior rows in your transaction should you get some sort of error in the middle of processing the current row. Right now if you have an error the only solution is to rollback the entire transaction. This could result in you losing a significant number of prior records in a streaming / micro batched use case (or requires you to stage that locally which makes it more complex.

The following python code should work after this is implemented (in a real case it would probably be in an exception handler of course):

import pymcsapi
driver = pymcsapi.ColumnStoreDriver()
bulk = driver.createBulkInsert('test', 't1', 0,0)
bulk.setColumn(0, 1)
bulk.setColumn(1, 'A')
bulk.writeRow()
bulk.setColumn(0, 2)
bulk.resetRow()
bulk.commit()



 Comments   
Comment by Andrew Hutchings (Inactive) [ 2017-10-31 ]

Pull request for resetRow(), test case and documentation.

Comment by David Thompson (Inactive) [ 2017-11-01 ]

Verified test case and also that can use this in tweepy streaming example.

Generated at Thu Feb 08 02:25:20 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.