Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Won't Fix
-
1.2.5
-
None
-
Red Hat Enterprise Linux 7.5
Virtualized, On Premise
Description
Customer reported they started to hit version buffer overflow error after Mar 21 06:03:34.
Once they hit this error, they hit another error with "Create table failed due to WE: Error updating calpontsys.systable: a BRM Begin copy error."
After restarting MCS, cluster got READ_ONLY mode.
Mar 21 06:03:34 nvm002314 workernode[29567]: 34.563294 |0|0|0| C 30 CAL0000: VBBM::getBlocks(): version buffer overflow. Increase VersionBufferFileSize. Overflow occured in aged blocks. Requested NumBlocks:VbOid:vbFBO:lastFBO = 50:0:112800:112899 lbid locked is 1689015296 |
Mar 21 06:03:34 nvm002314 ddlpackageproc[2510]: 34.567453 |0|0|0| E 23 CAL0009: Error in dropping column from systables. |
Mar 21 06:30:34 nvm002314 ddlpackageproc[2510]: 34.574847 |0|0|0| E 23 CAL0009: (2)Create table failed due to WE: Error updating calpontsys.systable: a BRM Begin copy error. |
Mar 21 06:43:03 nvm002314 ddlpackageproc[2510]: 03.714655 |0|0|0| E 23 CAL0009: (2)Create table failed due to WE: Error updating calpontsys.systable: a BRM Begin copy error. |
Mar 21 06:46:46 nvm002314 workernode[29567]: 46.933233 |0|0|0| C 30 CAL0000: VBBM::getBlocks(): version buffer overflow. Increase VersionBufferFileSize. Overflow occured in aged blocks. Requested NumBlocks:VbOid:vbFBO:lastFBO = 336:1:106600:106999 lbid locked is 1689039872 |
Mar 21 06:46:46 nvm002314 dmlpackageproc[2563]: 46.981965 |0|0|0| E 21 CAL0002: Delete Failed: IDB-2008: The version buffer overflowed. Increase VersionBufferFileSize or limit the rows to be processed. |
Mar 21 07:00:22 nvm002314 ddlpackageproc[2510]: 22.692432 |0|0|0| E 23 CAL0009: (2)Create table failed due to WE: Error updating calpontsys.systable: a BRM Begin copy error. |
Mar 21 12:20:57 nvm002314 ddlpackageproc[2510]: 57.912176 |0|0|0| E 23 CAL0009: (2)Create table failed due to WE: Error updating calpontsys.systable: a BRM Begin copy error. |
Mar 21 21:05:11 nvm002314 workernode[29567]: 11.495991 |0|0|0| C 30 CAL0000: VBBM::getBlocks(): version buffer overflow. Increase VersionBufferFileSize. Overflow occured in aged blocks. Requested NumBlocks:VbOid:vbFBO:lastFBO = 1:1:106600:106699 lbid locked is 1689039872 |
Mar 21 21:05:11 nvm002314 dmlpackageproc[2563]: 11.541648 |0|0|0| E 21 CAL0002: Update Failed: IDB-2008: The version buffer overflowed. Increase VersionBufferFileSize or limit the rows to be processed. |
Mar 21 21:16:48 nvm002314 ddlpackageproc[2510]: 48.252063 |0|0|0| E 23 CAL0009: (2)Create table failed due to WE: Error updating calpontsys.systable: a BRM Begin copy error. |
Mar 21 23:20:28 nvm002314 ddlpackageproc[2510]: 28.726261 |0|0|0| E 23 CAL0009: (2)Create table failed due to WE: Error updating calpontsys.systable: a BRM Begin copy error. |
Mar 21 23:25:32 nvm002314 DMLProc[14662]: 32.713136 |0|0|0| C 20 CAL0002: problem with rollback transaction 1498443and DBRM is setting to readonly and table lock is not released: Error encountered while opening version buffer file oid:dbroot = 0:4 and error message:No such file or directory (WriteEngine returns error 1) |
Mar 21 23:25:32 nvm002314 DMLProc[14662]: 32.720120 |0|0|0| C 20 CAL0002: problem with rollback transaction 1500786and DBRM is setting to readonly and table lock is not released: Can't read DBRM isReadWrite [ DBRM is in READ-ONLY mode] |
Mar 21 23:31:39 nvm002314 ddlpackageproc[14526]: 39.432797 |0|0|0| E 23 CAL0009: (2)Create table failed due to WE: Error updating calpontsys.systable: a BRM Begin copy error. |
Mar 21 23:37:34 nvm002314 ddlpackageproc[14526]: 34.238514 |0|0|0| E 23 CAL0009: (2)Create table failed due to WE: Error updating calpontsys.systable: a BRM Begin copy error. |
Mar 21 23:40:38 nvm002314 DMLProc[10355]: 38.175194 |0|0|0| C 20 CAL0002: problem with rollback transaction 1498443and DBRM is setting to readonly and table lock is not released: Error encountered while opening version buffer file oid:dbroot = 0:3 and error message:No such file or directory (WriteEngine returns error 1) |
Mar 21 23:40:38 nvm002314 DMLProc[10355]: 38.184515 |0|0|0| C 20 CAL0002: problem with rollback transaction 1500786and DBRM is setting to readonly and table lock is not released: Can't read DBRM isReadWrite [ DBRM is in READ-ONLY mode] |
Mar 21 23:53:29 nvm002314 ddlpackageproc[10200]: 29.427302 |0|0|0| E 23 CAL0009: (2)Create table failed due to WE: Error updating calpontsys.systable: a BRM Begin copy error. |
Mar 22 00:00:21 nvm002314 ddlpackageproc[10200]: 21.847798 |0|0|0| E 23 CAL0009: (2)Create table failed due to WE: Error updating calpontsys.systable: a BRM Begin copy error. |
Mar 22 00:00:28 nvm002314 workernode[4231]: 28.395074 |0|0|0| C 30 CAL0000: VBBM::getBlocks(): version buffer overflow. Increase VersionBufferFileSize. Overflow occured in aged blocks. Requested NumBlocks:VbOid:vbFBO:lastFBO = 64:1:106600:106699 lbid locked is 1689039872 |
Mar 22 00:00:28 nvm002314 dmlpackageproc[10355]: 28.464383 |0|0|0| E 21 CAL0002: Delete Failed: IDB-2008: The version buffer overflowed. Increase VersionBufferFileSize or limit the rows to be processed. |
Mar 22 01:34:18 nvm002314 DMLProc[6357]: 18.350789 |0|0|0| C 20 CAL0002: problem with rollback transaction 1498443and DBRM is setting to readonly and table lock is not released: Error encountered while opening version buffer file oid:dbroot = 0:2 and error message:No such file or directory (WriteEngine returns error 1) |
Mar 22 01:34:18 nvm002314 DMLProc[6357]: 18.359866 |0|0|0| C 20 CAL0002: problem with rollback transaction 1500786and DBRM is setting to readonly and table lock is not released: Can't read DBRM isReadWrite [ DBRM is in READ-ONLY mode] |
It does look like https://jira.mariadb.org/browse/MCOL-2054, but wanted to make sure it is the different version.
Asked customer to clean up files under /usr/local/mariadb/columnstore/data1/systemFiles/dbrm/DMLLog_, but customer said there were no files with name staring DMLLog_.
After they hit "The version buffer overflowed. Increase VersionBufferFileSize or limit the rows to be processed." error, I recommended to increase it it up to 2GB before it was 1GB. But, it didn't help.