Details
-
Bug
-
Status: Needs Feedback (View Workflow)
-
Critical
-
Resolution: Unresolved
-
23.02.9
-
None
-
2024-2
Description
A client periodically cleans old data using `caldroppartition` whilst running cpimport continuously.
There are multiple minor table lock clashes b/w these two processes. Here DDLProc log.
Aug 21 20:31:04 onl-actixmdb-01 ddlpackageproc[3177276]: 04.665157 |2079|676|0| D 23 CAL0041: Start SQL statement: caldroppartitions|mariadb_actixone_prod|
|
Aug 21 20:31:14 onl-actixmdb-01 DDLProc[3177276]: controllernode[3177276]: 14.680008 |0|0|0| E 29 CAL0000: DBRM: error: SessionManager::rolledback() failed (valid error code) %%10%%
|
Aug 21 20:31:14 onl-actixmdb-01 controllernode[3177276]: 14.680008 |0|0|0| E 29 CAL0000: DBRM: error: SessionManager::rolledback() failed (valid error code) %%10%%
|
Aug 21 20:31:14 onl-actixmdb-01 ddlpackageproc[3177276]: 14.680079 |2079|676|0| D 23 CAL0042: End SQL statement
|
Aug 21 20:31:14 onl-actixmdb-01 DDLProc[3177276]: ddlpackageproc[3177276]: 14.680594 |0|0|0| E 23 MCS-2009: Unable to perform the drop partition operation because cpimport.bin (pm1) with PID 3586566 is currently holding the table lock for session 4294967295.
|
Aug 21 20:31:14 onl-actixmdb-01 ddlpackageproc[3177276]: 14.680594 |0|0|0| E 23 MCS-2009: Unable to perform the drop partition operation because cpimport.bin (pm1) with PID 3586566 is currently holding the table lock for session 4294967295.
|
There are no obvious issues that points that `caldroppartitions` failed after it removes files but before it cleans up EM.
However after some time the table becomes unusable. This doesn't look like VersionBlock issue but merely like their Extent Map silently became outdated.
Aug 22 10:44:46 onl-actixmdb-03 ExeMgr[2699207]: 46.913182 |33556890|0|0| D 16 CAL0041: Start SQL statement: SELECT MIN(date),MAX(date) FROM LTE_ALL_MESH3_HOURLY; |mariadb_actixone_prod|
|
Aug 22 10:45:15 onl-actixmdb-03 PrimProc[2699207]: 15.123049 |0|0|0| C 28 CAL0053: PrimProc could not open file for OID 4020; /var/lib/columnstore/data3/000.dir/000.dir/015.dir/180.dir/033.dir/FILE000.cdf:No such file or directory
|
Aug 22 10:45:15 onl-actixmdb-03 PrimProc[2699207]: 15.124665 |0|0|0| W 28 CAL0000: PrimProc AsyncLoader caught error: MCS-2039: Data file does not exist, please contact your system administrator for more information. %%10%%
|
Aug 22 10:45:20 onl-actixmdb-03 PrimProc[2699207]: 20.123809 |0|0|0| C 28 CAL0053: PrimProc could not open file for OID 4279; /var/lib/columnstore/data3/000.dir/000.dir/016.dir/183.dir/033.dir/FILE000.cdf:No such file or directory
|
Aug 22 10:45:20 onl-actixmdb-03 PrimProc[2699207]: 20.124188 |0|0|0| W 28 CAL0000: PrimProc AsyncLoader caught error: MCS-2039: Data file does not exist, please contact your system administrator for more information. %%10%%
|
Aug 22 10:45:25 onl-actixmdb-03 PrimProc[2699207]: 25.124884 |0|0|0| C 28 CAL0053: PrimProc could not open file for OID 4020; /var/lib/columnstore/data3/000.dir/000.dir/015.dir/180.dir/033.dir/FILE000.cdf:No such file or directory
|
Aug 22 10:45:25 onl-actixmdb-03 PrimProc[2699207]: 25.125243 |0|0|0| W 28 CAL0000: PrimProc AsyncLoader caught error: MCS-2039: Data file does not exist, please contact your system administrator for more information. %%10%%
|
Aug 22 10:45:29 onl-actixmdb-03 cpimport.bin[3043487]: 29.689387 |0|0|0| I 34 CAL0086: Initiating BulkLoad: -L /var/log/mariadb/columnstore/cpimport/ -P pm3-3043487 -T SYSTEM -ub864fa21-ad07-48f9-ad3e-f13858c58c98 mariadb_actixone_prod LTE_ALL_MESH3_HOURLY LTE_ALL_MESH3_HOURLY01_2.tbl
|
Aug 22 10:45:29 onl-actixmdb-03 IDBFile[3043487]: 29.691612 |0|0|0| D 35 CAL0002: IDBFactory::installPlugin: installed filesystem plugin libcloudio.so
|
Aug 22 10:45:30 onl-actixmdb-03 PrimProc[2699207]: 30.126074 |0|0|0| C 28 CAL0053: PrimProc could not open file for OID 4020; /var/lib/columnstore/data3/000.dir/000.dir/015.dir/180.dir/033.dir/FILE000.cdf:No such file or directory
|
Aug 22 10:45:30 onl-actixmdb-03 PrimProc[2699207]: 30.126747 |0|0|0| W 28 CAL0000: MCS-2039: Data file does not exist, please contact your system administrator for more information. %%10%%
|
Aug 22 10:45:30 onl-actixmdb-03 ExeMgr[2699207]: 30.127443 |33556890|0|0| D 16 CAL0042: End SQL statement
|