[MCOL-632] IDB-2044: An internal error occurred Database is not working anymore Created: 2017-03-22  Updated: 2017-05-08  Resolved: 2017-05-08

Status: Closed
Project: MariaDB ColumnStore
Component/s: PrimProc
Affects Version/s: 1.0.7
Fix Version/s: Icebox

Type: Bug Priority: Major
Reporter: Christian2 Assignee: Unassigned
Resolution: Cannot Reproduce Votes: 0
Labels: None
Environment:

Centos 7, Single Server

Attachments: Zip Archive columnstoreSupportReport.columnstore-1.zip    

 Description   

Hello *,

columstore isn't able to proceed a create table. Additional no table will be found anymore.
Any suggestion to solve the issue would be great, its a productive system...

MariaDB [prod]> CREATE TABLE `x_21` (
-> `DB_MANDANT` char(3) DEFAULT NULL,
-> `DB_CORP` char(3) DEFAULT NULL,
-> `DB_LANDID` char(3) DEFAULT NULL,
-> `DB_KDNR` char(9) DEFAULT NULL
-> ) ENGINE=Columnstore DEFAULT CHARSET=utf8;
ERROR 1815 (HY000): Internal error: CAL0009: IDB-2044: An internal error occurred. Check the error log file & contact support.

error.log
Mar 22 22:45:01 kmodekarlsap001 PrimProc[68241]: 01.392952 |0|0|0| C 28 CAL0000: /home/builder/mariadb-columnstore-server/mariadb-columnstore-engine/primitives/primproc/rtscommand.cpp (token column) error on projection for oid 1001 lbid 0: input rids 907, output rids 906#012: Restarted a syscat job 120 times, bailing
Mar 22 22:45:01 kmodekarlsap001 Calpont[68631]: 01.394734 |0|0|0| E 00 CAL0000: /home/builder/mariadb-columnstore-server/mariadb-columnstore-engine/dbcon/joblist/batchprimitiveprocessor-jl.cpp@660: assertion 'in.length() > offset' failed
Mar 22 22:45:01 kmodekarlsap001 joblist[68631]: 01.396323 |2147483653|0|0| C 05 CAL0000: st: 2 TupleBPS::receiveMultiPrimitiveMessages() caught an exception: IDB-2035: An internal error occurred. Check the error log file & contact support.
Mar 22 22:45:01 kmodekarlsap001 ddlpackageproc[68631]: 01.402739 |0|0|0| E 23 CAL0009: IDB-2044: An internal error occurred. Check the error log file & contact support.

/usr/local/mariadb/columnstore/bin/editem -o 1001

Col OID = 1001, NumExtents = 1, width = 8
0 - 8191 (8192) min: notset, max: notset, seqNum: 70025, state: invalid, fbo: 0, DBRoot: 1, part#: 0, seg#: 0, HWM: 1; status: avail

After restart Columnstore:

select * from KD_X limit 1;
ERROR 1815 (HY000): Internal error: IDB-2006: 'sam.kd_x' does not exist in Columnstore

error.log
Mar 22 23:02:25 kmodekarlsap001 controllernode[77277]: 25.015775 |0|0|0| C 29 CAL0000: BRMShmImpl::BRMShmImpl(): retrying on size==0
Mar 22 23:06:28 kmodekarlsap001 ProcessMonitor[77559]: 28.385544 |0|0|0| C 18 CAL0000: *****Calpont Process Restarting: DBRMWorkerNode, old PID = 78105

 Comments   
Comment by David Thompson (Inactive) [ 2017-03-22 ]

Would it be possible for you to run the columnStoreSupport utility and send the results as this will provide more logs and details:
https://mariadb.com/kb/en/mariadb/system-troubleshooting-mariadb-columnstore/#mariadb-columnstore-support-tool

The table ddl is fine and creates on a test instance i have so it's likely something to do with the overall state of the host and software.

Comment by Christian2 [ 2017-03-22 ]

The logs where to big for upload (rotation does not work). I can help with specific parts of it if needed,

Comment by David Thompson (Inactive) [ 2017-03-23 ]

Strange the log rotation is not working but lets focus on the larger problem.

Unless you've overriden the default logging, the debug.log contains all levels, could you share that for the time frame around this?

Also based on the report the system is down, are you also unable to start it? Try doing a shutdownsystem first before startsystem, sometimes that will clear up issues.

Comment by David Thompson (Inactive) [ 2017-05-08 ]

Please re-open if you were able to produce additional logs or uncover more information

Generated at Thu Feb 08 02:22:32 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.