Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-5384

Plugin code leaks network connections towards ExeMgr making EM a threads leak bomb - possible mem leak

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Fixed
    • 6.4.6, 22.08.7
    • 22.08.8, 6.4.7- CS only
    • ExeMgr, MariaDB Server
    • None
    • Mariadb ver. 10.6.11/MCS ver. 6.4.6 (compiled from sources)
      CentOS 7 x86_64
    • 10.0.20

    Description

      Summary: ExeMgr spawns unlimited number of threads

      This problem is not new, it's already been reported as MCOL-4279, MCOL-4997.
      Additionally I could find also a post on mariadb-columnstore about it.
      Unfortunately all these cases were not solved, because... "not reproducible".

      Hopefully now we can fix it, because I'm determined to find the problem, with some help from you guys.

      To clarify: I'm very new to Columnstore. This is really my first try to Columnstore, so I cannot say if there are other scenarios or problematic cases. Unfortunately this issue is also a showstopper for me.

      My scenario:
      Every 15 min I execute couple of "LOAD DATA LOCAL INFILE ..." queries.
      Two things might be important here, although I'm not sure if they really have impact on bug or not:
      1. file is loaded from remote host (LOCAL)
      2. I have 3 parallel processes for loading data, so it might happen that multiple "LAOD DATA" statements are executed at once, although I guarantee that single table is LOAD-ed only by one source at the time.

      What I was able to find out till now:
      1. Every time I execute LOAD DATA LOCAL, new thread is created inside mariadbd. I can see this clearly in debug.log (dbcon).
      2. This new thread opens new the TCP-connection to ExeMgr
      3. ExeMgr creates new thread to serve request(-s) from mariadbd.
      4. When LOAD DATA is finished, mariadbd destroys the thread it created.
      5. But the TCP-connection to ExeMgr is still open (it's not closed).
      6. Because connection is still opened, ExeMgr-thread is still waiting in socket-read() mode, expecting to get messages from mariadbd.

      At least this is my current understanding of the problem:
      tcp-connection is not closed properly before destroying the thread.

      Could you give me a hint, what code is responsible for creating mariadb-thread, creating TCP-connection, cleaning up connection and destroying the mariadb-thread?
      Looking at the logs I concluded the LOAD DATA statement process is started in
      ha_mcs_impl_start_bulk_insert() and ends in ha_mcs_impl_end_bulk_insert()
      But still have the problem to figure out the functions responsible for
      TCP-connection management during the query.

      Best Regards,
      Karol

      Attachments

        Issue Links

          Activity

            People

              drrtuy Roman
              Karol Karol Roslaniec
              Roman Roman
              Kirill Perov Kirill Perov
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.