Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
6.4.6, 22.08.7
-
None
-
Mariadb ver. 10.6.11/MCS ver. 6.4.6 (compiled from sources)
CentOS 7 x86_64
-
10.0.20
Description
Summary: ExeMgr spawns unlimited number of threads
This problem is not new, it's already been reported as MCOL-4279, MCOL-4997.
Additionally I could find also a post on mariadb-columnstore about it.
Unfortunately all these cases were not solved, because... "not reproducible".
Hopefully now we can fix it, because I'm determined to find the problem, with some help from you guys.
To clarify: I'm very new to Columnstore. This is really my first try to Columnstore, so I cannot say if there are other scenarios or problematic cases. Unfortunately this issue is also a showstopper for me.
My scenario:
Every 15 min I execute couple of "LOAD DATA LOCAL INFILE ..." queries.
Two things might be important here, although I'm not sure if they really have impact on bug or not:
1. file is loaded from remote host (LOCAL)
2. I have 3 parallel processes for loading data, so it might happen that multiple "LAOD DATA" statements are executed at once, although I guarantee that single table is LOAD-ed only by one source at the time.
What I was able to find out till now:
1. Every time I execute LOAD DATA LOCAL, new thread is created inside mariadbd. I can see this clearly in debug.log (dbcon).
2. This new thread opens new the TCP-connection to ExeMgr
3. ExeMgr creates new thread to serve request(-s) from mariadbd.
4. When LOAD DATA is finished, mariadbd destroys the thread it created.
5. But the TCP-connection to ExeMgr is still open (it's not closed).
6. Because connection is still opened, ExeMgr-thread is still waiting in socket-read() mode, expecting to get messages from mariadbd.
At least this is my current understanding of the problem:
tcp-connection is not closed properly before destroying the thread.
Could you give me a hint, what code is responsible for creating mariadb-thread, creating TCP-connection, cleaning up connection and destroying the mariadb-thread?
Looking at the logs I concluded the LOAD DATA statement process is started in
ha_mcs_impl_start_bulk_insert() and ends in ha_mcs_impl_end_bulk_insert()
But still have the problem to figure out the functions responsible for
TCP-connection management during the query.
Best Regards,
Karol