[MCOL-1581] CAL0000: VBBM::getBlocks(): version buffer overflow error Created: 2018-07-23  Updated: 2019-07-10  Resolved: 2019-07-10

Status: Closed
Project: MariaDB ColumnStore
Component/s: ?
Affects Version/s: 1.0.9
Fix Version/s: Icebox

Type: Bug Priority: Major
Reporter: Mike Lok Assignee: Unassigned
Resolution: Not a Bug Votes: 0
Labels: None
Environment:

RHEL 7.2

Attachments: File columnstoreSupportReport.tar.gz    

 Description   

Hi all,

I have MariaDB CS 1.0.9 installed in a single node on RHEL7.2.

One of my users recently tried to delete records from a ColumnStore table with the error below returned:

Internal error: CAL0002: Delete Failed: IDB-2008: The version buffer overflowed. Increase VersionBufferFileSize or limit the rows to be processed.

I tried altering the table from ColumnStore to InnoDB but faced a similar error:

{{Jul 18 23:01:44 ods workernode[47824]: 44.562315 |0|0|0| C 30 CAL0000: VBBM::getBlocks(): version buffer overflow. Increase VersionBufferFileSize. Overflow occured in aged blocks. Requested NumBlocks:VbOid:vbFBO:lastFBO = 1:0:27000:27099 lbid locked is 323416308
Jul 18 23:01:44 ods ddlpackageproc[48579]: 44.566626 |0|0|0| E 23 CAL0001: Alter table Failed: WE: Update failed on: systable}}

I get the same error when trying to drop the table.

I've increased the VersionBufferFileSize value in Columnctore.xml to 2GB from 1GB but the problem still persists.

Is there something else I can do?

Thanks.

 Comments   
Comment by Roman [ 2018-07-28 ]

Greetings,
There must be a number of simultaneous DML operations run on this relation. Possible you have a stuck query in the system. To investigate it future you should post an output of support report tool .
Could you rename the table and drop it after or just truncate it?

Comment by Mike Lok [ 2018-07-28 ]

Hi,

I've restarted MCOL system and it's still the same. I doubt there are stuck queries that are causing the issue.

Will try the support report tool you mentioned.

Renaming or dropping the table results in the same error.

Comment by Mike Lok [ 2018-07-30 ]

Hi all,

Uploaded columnstoreSupport output as suggested.

Comment by Richard Stracke [ 2018-07-31 ]

VersionBufferFileSize is still too small.

Example:

Jul 30 03:55:33 ods dmlpackageproc[48616]: 33.153457 |44422|52180|0| D 21 CAL0001: Start SQL statement:  UPDATE DM_MARKETING.DIM_ACCOUNTS AS A#012SET SUBFLAG = 1#012WHERE#012EXISTS (SELECT 'x' FROM DM_MARKETING.DIM_ACCOUNT_SERVICES AS B WHERE B.ACCOUNT_ID = A.ACCOUNT_ID);|dm_marketing|

I see you fill the table shortly before you updated it.

 finished loading table dm_marketing.dim_accounts; 7127866 rows inserted

Worst case scenario only for dim_accounts:

Assuming an average columnwidth from 8 bytes.

73 columns * 8 bytes * 7127866 rows = 4162673744 --> 3,87 GB Versionbuffer minimum.

Alternative:

You can reduce the need of versionbuffer.

1. Extends

Create / Refresh the extends with a simple select on the table, so extends table will be refreshed and extends elimination can work.

2. Proper where filter

Add where condition for better extends elimination.

WHERE#012EXISTS (SELECT 'x' FROM DM_MARKETING.DIM_ACCOUNT_SERVICES AS B WHERE B.ACCOUNT_ID = A.ACCOUNT_ID)


Try to rewrite this into a usual inner join like
 
inner join DIM_ACCOUNT_SERVICES B  
on .ACCOUNT_ID = A.ACCOUNT_ID

3. Don't mix columnstore and other engines , where it is avoidable.

Comment by Mike Lok [ 2018-08-02 ]

Hi Richard,

Thanks for reverting. Will update the VersionBuffer size to at least 4GB. Is there a limit to the VersionBuffer size I can set based on the amount of memory installed in the server?

Thanks again.

Comment by Roman [ 2018-12-27 ]

mikelok I recommend to look at MCOL-1411 for a Richard's version buffer size general sugesstions.

Generated at Thu Feb 08 02:29:49 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.