[MCOL-4859] ExeMgr spawns infinite number of threads Created: 2021-09-06  Updated: 2023-02-06

Status: Open
Project: MariaDB ColumnStore
Component/s: ExeMgr
Affects Version/s: 5.6.2
Fix Version/s: Icebox

Type: Bug Priority: Major
Reporter: andreas eschbacher Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: threads
Environment:

NAME="Red Hat Enterprise Linux"
VERSION="8.2 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.2"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.2 (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8.2:GA"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.2
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.2"
Red Hat Enterprise Linux release 8.2 (Ootpa)
Red Hat Enterprise Linux release 8.2 (Ootpa)

Attachments: PNG File image-2021-09-06-10-01-01-351.png     PNG File image-2021-09-06-10-01-08-977.png    

 Description   

show variables like '%version%'
-> ;
---------------------------------------------------------------------------+

Variable_name Value

---------------------------------------------------------------------------+

in_predicate_conversion_threshold 1000
innodb_version 10.6.4
protocol_version 10
slave_type_conversions  
system_versioning_alter_history ERROR
system_versioning_asof DEFAULT
tls_version TLSv1.1,TLSv1.2,TLSv1.3
version 10.6.4-MariaDB
version_comment MariaDB Server
version_compile_machine x86_64
version_compile_os Linux
version_malloc_library system
version_source_revision 2db692f5b4d6bb31a331dab44544171c455f6aca
version_ssl_library OpenSSL 1.1.1c FIPS 28 May 2019
wsrep_patch_version wsrep_26.22

MariaDB [(none)]> SELECT calgetversion();
-----------------

calgetversion()

-----------------

5.6.2

----------------


1. ) after we start the ETL the number of Threads increase infinite.
2.) when we cancel the execution the number of Threads is still higher than at the beginning

Can you please help us to find a parameter where we can limit the max Threads spawned by ExeMgr.

Thank you.
Br Andreas

 Comments   
Comment by Roman [ 2022-02-22 ]

Greetings Andreas.

This is an interesting case that implicitly points to a potential logical thread leak. Could I have top or ps output that demonstrates the number of threads spawned by MDB/MCS processes so we know what process contributes to the potential leakage? If the main contributor is MDB then could you check if your ETL quits the connection after it finishes?
Could I ask you for DDL(posted here or we can exchange contacts so you can share it secretly) and ingested data set cardinalities to try to reproduce it locally?
JFYI Unfortunately there is no hard limit on the number of threads spawned by MCS. However this it is unexpected behavior and we will solve it if reproduce it.

Comment by andreas eschbacher [ 2023-02-06 ]

Won't proceed here further as the problem with threads seems to be solved by now.

Comment by Roman [ 2023-02-06 ]

Thx for update aeae81. JFYI there problem was caused by MCOL-5384 issue that is resolved and will be shipped with the next community/enterprise server releases.

Generated at Thu Feb 08 02:53:33 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.