[MDEV-18857] Memory leak of the server on Windows Created: 2019-03-08  Updated: 2019-05-02  Resolved: 2019-05-02

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - Connect
Affects Version/s: 10.4.3, 10.3.13
Fix Version/s: N/A

Type: Bug Priority: Critical
Reporter: Pat Sav Assignee: Vladislav Vaintroub
Resolution: Not a Bug Votes: 0
Labels: None
Environment:

Under Windows... I have tried Windows server 2016, Windows 7 and Windows 10 pro


Attachments: JPEG File debug snapshot1.jpg     JPEG File debug snapshot2.jpg     JPEG File debug snapshot3.jpg     JPEG File debug snapshot4.jpg     JPEG File debug snapshot5.jpg     JPEG File debug snapshot_new_7.jpg     JPEG File debug snapshot_new_8.jpg     JPEG File debug snapshot_new_9.jpg     JPEG File debug snapshot_new_9b.jpg     Text File umdh_compare.txt    

 Description   

I need a second opinion from a developper team member, I feel that Olivier Bertrand did not understand what was involved or test enough... please, DO NOT transfer this ticket to him.

I have also tried 32 bit and 64 bit of the software, they all have the problem.

To my understanding, ODBC used inside the CONNECT engine should not consume memory when we are done with them... they are only interfaces that we use to communicate with a DB, right ?? I think that the memory leak is inside MariaDB and that it is caused by what is done by the data returned from the ODBC drvier...

To test, I did install a fresh MariaDB version 10.4.3 winx64 from the MSI as my local server and a remote 10.3.13 winx64 from the MSI. I did not change anything from the config / parameters file. Between the 2 server, I have install a MySQL ODBC driver v 8.0.15 64 bit and setup a system DSN.

Everything of the whole story and first experience with a stubborn Olivier is under this ticket

I have to restart my MariaDB service on a daily basis to reset it's memory usage and prevent him from crashing.

Thank you !



 Comments   
Comment by Elena Stepanova [ 2019-03-08 ]

wlad, could you please look into it?

Comment by Pat Sav [ 2019-03-08 ]

I am a programmer, but not a C++ one... not too familiar with all the memory allocation. I have pushed my testing a little bit further. I have installed the debug symbols for the MariaDB server V10.4.3 and with the C++ debugger from Microsoft Visual Studio, I took some snapshot (sorry it's in French). The red arrows that indicate a memory usage increase, this is where I have sent many times in a short amount of time the same SELECT query on a CONNECT table.

Note that the "void" objects are going up, aging and are not released, even after avery long period of time.

Comment by Vladislav Vaintroub [ 2019-03-08 ]

The approach with memory allocation tracking is good, however it does not look like all symbols would be resolved.

You download mariadb-10.4.3-winx64-debugsymbols.zip (if you used 64bit server), or mariadb-10.4.3-win32-debugsymbols.zip (if you used 32bit server) from https://downloads.mariadb.org/mariadb/10.4.3/ ? You need to unpack the zip, and copy ha_connect.pdb next to location of mysqld.exe (e.g C:\Program Files\MariaDB 10.4\bin ) . Then, maybe you can rerun your tests, can be sure all symbols are resolved, and see if the output of the leak detector is the same.

"void" does not help much, since that does not correspond to any known symbol.

Comment by Pat Sav [ 2019-03-08 ]

Hello Vlad.

Yes, I have download the debug symbols from the download link you mentionned, the 64bit ZIP file for my 64bit server. I wasn't sure where to put them, so I have split the content and put them in "lib" and in "bin" folders, next to their programs. I will all put them in the bin folder and add new snapshots.

Comment by Pat Sav [ 2019-03-08 ]

Ok, take two... second snapshots serie.

Each SELECT query on the CONNECT table adds an instance in the "void" section. I took a snapshot of the detail of one of those instance (debug snapshot_new_9b.jpg). You can see the details of the calls made.

Hope it helps, but would be better if you can see it by yourself on your own machine.

Thank you.

Comment by Pat Sav [ 2019-03-19 ]

Hello Vlad,

Did you had time to test the situation on your side ?

Comment by Vladislav Vaintroub [ 2019-03-19 ]

Nope, not yet.

Comment by Pat Sav [ 2019-04-30 ]

Hello, you can close this ticket... seems like it is not important anyway. Two developpers from the team that does not really want to look into it, even if there is a memory leak. It's taking only 10 minutes to setup a faulty environment. The least would have been to rule out if the problem is within MariaDB / Connect engine or from the ODBC driver.

Comment by Vladislav Vaintroub [ 2019-05-02 ]

Pat Sav. The expectation that your ticket will be looked at with highest priority might not always hold. Please have understanding for that, we have many versions to support, and to develop. Eventually, the ticket will be picked up.

Noe to your bug report.. I can not easily reproduce your bug.
My test (easiest, since you do not provide any insight in your queries)

  1. I installed latest MariaDB 10.3.
  2. Installed latest MariaDB ODBC x64
  3. Created system DNS called "mariadb" pointing to the installed instance.
  4. on the same machine, I run 10.4, where I created Connect table like this

    CREATE TABLE `remote_seq_1_to_100` ( seq int) ENGINE=CONNECT CONNECTION='DSN=mariadb' table_type=ODBC tabname='test.seq_1_to_100'

    which spits remote out data of the Sequence engine (https://mariadb.com/kb/en/library/sequence-storage-engine/ )

  5. in an attempt to reproduce the leak, I also created a stored procedure `select_loop` like this

    CREATE PROCEDURE select_loop(N INT)
    BEGIN
    DECLARE x INT;
    SET x = 1;
    WHILE x <= N DO
    SELECT * from remote_seq_1_to_100;
    SET x = x + 1;
    END WHILE;

  6. I run the stored procedure from command line client, suppressing the output, like this

    mysql -uroot test -e "call select_loop(100000)" > NUL

    this executes the remote "SELECT * FROM test.seq_1_to_100" 100000 times via CONNECT engine

  7. The memory usage of both mysqld.exe remains low (working set, as shown in Task Manager, remains under 70MB), the whole time the test took, which is about 15 minutes

I do not see a memory leak in this test. Perhaps, there are is a more complicated setup w.r.t queries and data that you're using, but you'd need to give an idea of that, so it can be reproduced elsewhere.

Comment by Vladislav Vaintroub [ 2019-05-02 ]

Ok, I do however see a memory leak when using MySQL ODBC driver 8.0.

So , MariaDB ODBC driver does not have a leak. MySQL ODBC driver does have a leak, which for you means - do not use buggy MySQL ODBC driver, or, if you have to, report something reproducible to them. Tell them they have a leak in SQLDriverConnect, and use our driver in the meantime.

I will attach umdh (https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/using-umdh-to-find-a-user-mode-memory-leak) output, it seems that their leak is in SQLDriverConnect somewhere, but there are no symbols for driver code itself shown in the output, only our symbols.

On that other hand, that connect engine opens and closes a connection on each query, is quite surprising, to me.

Comment by Vladislav Vaintroub [ 2019-05-02 ]

Closing it is not our bug. Use our driver, there is no leak in it.

Generated at Thu Feb 08 08:47:15 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.