[MCOL-1093] Internal error: CAL0001: Insert Failed: no space for a dictionary insert Created: 2017-12-11  Updated: 2022-11-05  Resolved: 2022-11-05

Status: Closed
Project: MariaDB ColumnStore
Component/s: ?
Affects Version/s: 1.5.3
Fix Version/s: Icebox

Type: Bug Priority: Major
Reporter: hiller1 Assignee: Unassigned
Resolution: Won't Do Votes: 1
Labels: None
Environment:

1.0.11


Attachments: File columnstoreSupportReport.MPP.tar.gz     File columnstoreSupportReport.columnstore-1 (3).tar.gz     Zip Archive editem.zip     File extent_map.txt.gz     Zip Archive information_schema.columnstore_files.zip    

 Description   

Internal error: CAL0001: Insert Failed: no space for a dictionary insert



 Comments   
Comment by Andrew Hutchings (Inactive) [ 2017-12-11 ]

I've not seen this error fire before. In theory it means the extent map is completely full.

Can you please attach the result of running "editem -d"?

Comment by hiller1 [ 2017-12-11 ]

extent_map.txt.gz

Uploaded to attachments

Comment by Andrew Hutchings (Inactive) [ 2017-12-11 ]

I don't see anything immediately obvious wrong. Can you please also send the ColumnStore support report?

https://mariadb.com/kb/en/library/system-troubleshooting-mariadb-columnstore/#mariadb-columnstore-support-report-tool

Comment by hiller1 [ 2017-12-12 ]

columnstoreSupportReport.MPP.tar.gz

Uploaded to attachments

Comment by Andrew Hutchings (Inactive) [ 2017-12-12 ]

Unfortunately this only seems to have collected the PM1 logs. Did you run the multi-node example shown in that link?

Comment by hiller1 [ 2017-12-13 ]

File "columnstoreSupportReport.MPP_multi-node separate.tar.gz" was not uploaded
File exceeds maximum size limit set by the administrator (10.00 MB).

Comment by Andrew Hutchings (Inactive) [ 2017-12-13 ]

Ah! In that case please upload the file to ftp://ftp.mariadb.com/uploads and let us know here when it is done.

Comment by hiller1 [ 2017-12-13 ]

HI,I have uploaded ftp://ftp.mariadb.com/uploads/

the file name is columnstoreSupportReport.MPP_multi-node separate_hiller.tar.gz

thanks!

Comment by Andrew Hutchings (Inactive) [ 2017-12-15 ]

Many thanks for this. It still only contains PM1 instead of the whole cluster but contains a bit more information. This is what we know so far:

  • The problem started before the beginning of the logs (before 6th December) so it is difficult to work out the root cause
  • There are certain processes that are regularly restarting and it is not clear why
  • This is happening on multiple tables

The error is basically triggered when either there is no block space left for the dictionary data (you would need to be in the hundreds of TB range) or when the file size doesn't match the dictionary metadata. I cannot yet see a theoretical reason why getting the file sizes would fail, but this is a possible way to trigger this error.

I am leaning towards the a file size issue due to the process restarts. So, I ask for a couple more things:

1. Did you run the redistribute command at any time?
2. Can you please run a select on information_schema.columnstore_files and send that to us?

Comment by hiller1 [ 2017-12-17 ]

information_schema.columnstore_files.zip

thanks!I have uploaded information_schema.columnstore_files.zip

Comment by hiller1 [ 2017-12-18 ]

I have not executed mcsadmin redistribute before

Comment by Zorion Arrizabalaga [ 2018-01-19 ]

I have the same issue.
I tried mcsadmin redistribute but it didn't help.
My result:
0 success, 0 skipped, 0 failed.
Total time: 2 seconds.

I tried to restart the complete server and it didn't help.
At the end, I renamed my_table to old_my_table and generated another one (with the expected table name) so I'm ok. Now I have the old_my_table (renamed) and the new my_table ok.

I can see that this issue is in a "needs feedback" status. If I can help let me know.

Comment by Andrew Hutchings (Inactive) [ 2018-01-19 ]

Hi zorion,

The reason I asked about 'redistribute' is that it could have been a possible cause. Not a way of fixing the problem. If you have the same error as the description a ColumnStore support report will help us (information on how to do this is in the second comment).

Comment by Zorion Arrizabalaga [ 2018-01-22 ]

Hi Andrew,
Thanks for your reply.

I am not able to reproduce the error, so I won't add further details. I'm sorry I can't help.

In case it may help someone:
Today I reproduced the issue once. I tried to add 100 records to old_my_table. It failed (with no space for dictionary insert).
In order to have a minimal test example I tried it with 1 record and it worked (it DID insert the record and I had no issues).
Now it works even for 100 rows. I am not sure if adding just one record fixed something but the table can be updated with more rows without issues.
I will drop old_my_table since it is has become useless for me.

I'll add details in case I face the issue in the future.
Best regards,
Zorion

Comment by suresh ramagiri [ 2020-10-13 ]

One of our customers, facing this issue at MariaDB Columnstore 1.5 (MariaDB server 10.5.4).

Running an INSERT query throwing error - Insert Failed: no space for a dictionary insert.
They can repeat this problem, executing the same query again.

Locally, I tried to repeat the problem with the DDL they shared of the table used in this INSERT query and running the query, but not happening.

Attaching the columnstore support report - columnstoreSupportReport.columnstore-1 (3).tar.gz with this comment.

Oct 10 11:24:18 x01tgisdb4a dmlpackageproc[16179]: 18.759688 |186984|53128|0| D 21 CAL0001: Start SQL statement: INSERT IGNORE INTO gistgt.local_cust (LOCAL_IDENT_NUM, SRC_APPL_CD, GLOBAL_CUST_KEY,#015#012 ASOF_DT, APPL_LOCATION_CD, LCIN_REL_GRP_KEY, LCIN_RPTNG_CNTRY_CD) VALUES#015#012#011#011#011#011 ( NAME_CONST('DataRow.local_ident_num',_utf8mb4'S0172911Z00' COLLATE 'utf8mb4_bin'), NAME_CONST('lSrcApplcd',_latin1'CPMSDBSSG' COLLATE 'latin1_swedish_ci'), NAME_CONST('lMaxCustKey',_utf8mb4'S0172911Z00-SG20191230' COLLATE 'utf8mb4_bin'), NAME_CONST('p_BusinessDate',TIMESTAMP'2020-06-30 00:00:00'), NAME_CONST('p_BkgCtryCd',_latin1'SG' COLLATE 'latin1_swedish_ci'), NAME_CONST('DataRow.rel_grp_key',_utf8mb4'SGdummy20090131' COLLATE 'utf8mb4_bin'), NAME_CONST('lRelGrpCtryCd',_utf8mb4'SG' COLLATE 'utf8mb4_bin')); |gistgt|
Oct 10 11:24:19 x01tgisdb4a dmlpackageproc[16179]: 19.461080 |186984|53128|0| D 21 CAL0001: End SQL statement
Oct 10 11:24:19 x01tgisdb4a dmlpackageproc[16179]: 19.461253 |0|0|0| E 21 CAL0001: Insert Failed: no space for a dictionary insert columnstoreSupportReport.columnstore-1 (3).tar.gz
Oct 10 11:24:19 x01tgisdb4a dmlpackageproc[16179]: 19.478567 |186984|53128|0| D 21 CAL0001: Start SQL statement: ROLLBACK
Oct 10 11:24:19 x01tgisdb4a dmlpackageproc[16179]: 19.488821 |186984|53128|0| D 21 CAL0001: End SQL statement

Comment by suresh ramagiri [ 2020-10-16 ]

Attaching the output of "editem -d " output - editem.zip, I noted in the comments, requested for this output.

Comment by Todd Stoffel (Inactive) [ 2022-11-05 ]

This item is being closed because it was well passed the expiration date with no activity. If you suspect this was done in error please create a new ticket.

Generated at Thu Feb 08 02:26:08 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.