[MCOL-4451] Problems with INSERT and SELECT operations when the number of rows falls between 100 millions and 1 billion Created: 2020-12-15  Updated: 2021-04-19  Resolved: 2020-12-22

Status: Closed
Project: MariaDB ColumnStore
Component/s: installation
Affects Version/s: 5.4.3
Fix Version/s: 5.4.3

Type: Bug Priority: Major
Reporter: David Hill (Inactive) Assignee: Unassigned
Resolution: Not a Bug Votes: 0
Labels: None
Environment:

1 maxscale
3 CS nodes



 Description   

Customer reported:

Description: During some tests, we found that working with an increasing batch of data (1K, 10K, 100K, 1M, 10M, 100M, 1B rows) the ColumnStore worked as expected till we launched a SQL statement that acts on more than 100M rows. In this case, the operation stopped with the following error:

#1815 - Internal error: InetStreamSocket::readToMagic: Remote is closed

The SQL statement and the corresponding log are in the attached file.

Date/time: 2020-12-04 16:21:06
Signal: 11

/usr/bin/ExeMgr(+0x24ad0)[0x55ab2a6d3ad0]
/lib64/libpthread.so.0(+0xf630)[0x7f48765ee630]
/lib64/libjoblist.so(_ZN7joblist21DistributedEngineComm5SetupEv+0xd9d)[0x7f487621b31d]
/lib64/libjoblist.so(_ZN7joblist21DistributedEngineComm6ListenEN5boost10shared_ptrIN11messageqcpp18MessageQueueClientEEEj+0x2cb)[0x7f487621c85b]
/lib64/libjoblist.so(+0x10bb13)[0x7f487621cb13]
/lib64/libboost_thread-mt.so.1.53.0(+0xd1da)[0x7f48725931da]
/lib64/libpthread.so.0(+0x7ea5)[0x7f48765e6ea5]
/lib64/libc.so.6(clone+0x6d)[0x7f486ff3a96d]



 Comments   
Comment by David Hill (Inactive) [ 2020-12-15 ]

I did recommended using cpimport and I got this followup

We know that UPDATE and DELETE are not optimized in ColumnStore, but it fails the same way even with a SELECT operation only, and this is not a matter of optimization but probably it is due to an anomaly that is triggered by the number of rows involved in the operation. Since we ordinarily deal with billions of rows, for us it is critical to solve this problem also in a simple SELECT statement.

Comment by David Hill (Inactive) [ 2020-12-22 ]

So this one looks like its related to Memory setting on their 3 pm setup

Right on the call of memory overusage, I looked at the config files and did see that the Columnstore memory settings are 2 high.

These are the 2 settings for Columnstore related to memory. These 2 settings should be a max of 75% as shown in following Document. They are at 150%. So these 2 need to be tweaked down to equal 75% between the 2.

<TotalUmMemory>100%</TotalUmMemory>
<NumBlocksPct>50</NumBlocksPct>

https://mariadb.com/kb/en/mariadb-columnstore-performance-related-configuration-settings/#memory-management-numblockspct-and-totalummemory

This is the memory size for InnoDB data. So if you are utilizing InnoDB database, then the combination of the 2 above plus this should be over 85%. This will leave space to temporary storage that is used at times.

innodb_buffer_pool_size 53687091200

Server memory size is 80gb

I will close and can be reopened if something different comes up.

Generated at Thu Feb 08 02:50:26 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.