[MDEV-9997] CONNECT: memory leak after connect.alter connect.ini_grant Created: 2016-04-26  Updated: 2016-05-13  Resolved: 2016-05-13

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - Connect
Affects Version/s: 10.0, 10.1, 10.2
Fix Version/s: 10.0.26, 10.1.15

Type: Bug Priority: Major
Reporter: Sergey Vojtovich Assignee: Olivier Bertrand
Resolution: Fixed Votes: 0
Labels: foundation

Issue Links:
Blocks
blocks MDEV-7069 Fix buildbot failures in main server ... Stalled

 Description   

This is failing since quite a while in buildbot: http://buildbot.askmonty.org/buildbot/builders/work-amd64-valgrind/builds/8698/steps/test/logs/stdio (look for PROFILE_Open).

This memory leak is easily reproducible:

BUILD/compile-amd64-valgrind-max &&
cd mysql-test &&
./mtr --no-reorder --valgrind connect.alter connect.ini_grant

Even simpler test basing on previous two:

CREATE TABLE t1 (c INT NOT NULL, d CHAR(10) NOT NULL) ENGINE=CONNECT TABLE_TYPE=fix FILE_NAME='tf1.txt' ENDING=1;
ALTER TABLE t1 ENGINE=ARIA;
DROP TABLE t1;
 
CREATE TABLE t1 (sec CHAR(10) NOT NULL FLAG=1, val CHAR(10) NOT NULL) ENGINE=CONNECT TABLE_TYPE=INI FILE_NAME='t1.EXT';
INSERT INTO t1 VALUES ('sec1','val1');
DROP TABLE t1;

Valgrind warnings:

==28595== 320 bytes in 10 blocks are still reachable in loss record 1 of 3
==28595==    at 0x4C2BC9F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==28595==    by 0xC755E6: PROFILE_Open (inihandl.c:455)
==28595==    by 0xC76C95: PROFILE_GetPrivateProfileString (inihandl.c:1137)
==28595==    by 0xC76D74: GetPrivateProfileString (inihandl.c:1165)
==28595==    by 0xC43728: TDBINI::GetSeclist(_global*) (tabsys.cpp:189)
==28595==    by 0xC438F9: TDBINI::OpenDB(_global*) (tabsys.cpp:260)
==28595==    by 0xBF95DD: CntOpenTable(_global*, TDB*, MODE, char*, char*, bool, ha_connect*) (connect.cc:360)
==28595==    by 0xBE94A0: ha_connect::OpenTable(_global*, bool) (ha_connect.cc:1823)
==28595==    by 0xBEE54F: ha_connect::write_row(unsigned char*) (ha_connect.cc:3251)
==28595==    by 0x877313: handler::ha_write_row(unsigned char*) (handler.cc:6005)
==28595==    by 0x636E54: write_record(THD*, TABLE*, st_copy_info*) (sql_insert.cc:1847)
==28595==    by 0x634960: mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool) (sql_insert.cc:964)
==28595==    by 0x656638: mysql_execute_command(THD*) (sql_parse.cc:3447)
==28595==    by 0x65F289: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6554)
==28595==    by 0x651280: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1308)
==28595==    by 0x65040A: do_command(THD*) (sql_parse.cc:998)

This happens likely due to fact that MRUProfile is never freed:

commit db77e64351310ce76c51f80c00a061cdc777232a
Author: Olivier Bertrand <bertrandop@gmail.com>
Date:   Wed Mar 19 02:25:28 2014 +0100
 
    - Suppress call to PROFILE_End in connect_done_func that causes Signal 11 on Linux
    modified:
      storage/connect/ha_connect.cc
 
diff --git a/storage/connect/ha_connect.cc b/storage/connect/ha_connect.cc
index 09917c8..ec8ed16 100644
--- a/storage/connect/ha_connect.cc
+++ b/storage/connect/ha_connect.cc
@@ -378,7 +378,7 @@ static int connect_done_func(void *p)
 #endif   // LIBXML2_SUPPORT
 
 #if !defined(WIN32)
-  PROFILE_End();
+//PROFILE_End();                Causes signal 11
 #endif   // !WIN32
 
   for (pc= user_connect::to_users; pc; pc= pn) {

But memory leak is in no way good workaround for SIGSEGV.



 Comments   
Comment by Olivier Bertrand [ 2016-04-26 ]

Do you mean that restoring the call to PROFILE_END() solve the problem?

Comment by Sergey Vojtovich [ 2016-04-27 ]

I didn't try. But FWICS PROFILE_End() is the only function that frees MRUProfile. I can try if you like.

Comment by Olivier Bertrand [ 2016-04-27 ]

Yes please. As a matter of fact, I don't know what MRUProfile is and how it is connected to these CONNECT tests. Besides, what make you think that it is the item that is not freed?

Note that because the connect_done_func is only called when the system is shutting down, this memory leak may not be really important. This once more demonstrates the stupidity of these signal raised by some ASSERT MariaDB is full of. ASSERTs, according to all programming experts, is a bad habit that in fact hides the errors, lazily avoid to seach for the cause of them, and prevents to fix them or at least to produce an explicite error message.

Comment by Sergey Vojtovich [ 2016-04-27 ]

I confirm that uncommenting PROFILE_End() in connect_done_func() solves the problem. However I had to also comment out "CurProfile = MRUProfile[i];" and "PROFILE_ReleaseFile();" in PROFILE_End() to make it work properly.

Besides, what make you think that it is the item that is not freed?
"Valgrind warnings" above point to "PROFILE_Open (inihandl.c:455)", which is "MRUProfile[i] = malloc(sizeof(PROFILE));"

Note that because the connect_done_func is only called when the system is shutting down, this memory leak may not be really important.
Strictly speaking connect_done_func() doesn't mean system shutdown, it doesn't even mean MariaDB server shutdown. It means CONNECT engine shutdown. Technically it might be not that important. But it costs us reputation: even if we suppress this leak for valgrind, sooner or later users will find this leak with different memory analysis tool.

This once more demonstrates the stupidity of these signal raised by some ASSERT MariaDB is full of.
How do asserts relate to this bug?

ASSERTs, according to all programming experts, is a bad habit that in fact hides the errors
How is it possible to "hide an error" with assert?

The point of asserts in MariaDB is not "hidding errors" or "avoiding to search for them". Asserts issue additional checks in debug builds, which for performance reason are not issued in release builds.

Comment by Olivier Bertrand [ 2016-04-27 ]

Good and thank you. I shall do this and hopefully fix this bug.

inihandl.c: As a matter of fact, this code is meant to emulate Windows functions not existing on Unix/Linux. It was imported ten years ago in a previous program of mine and I don't know who wrote it. I have never tried to understand in detail what it does, working almost only on Windows.

Signal was the reason why I commented out this line but apparently this did not happen when you restored the previous version so the reason why it raised a signal has probably been fixed (perhaps by your commenting out these two lines) Perhaps this stuff could be just removed on Linux distributions.

About assertion see for example https://www.softwariness.com/articles/assertions-in-cpp/

Comment by Sergey Vojtovich [ 2016-04-28 ]

Thanks for accepting it. Please also note that MRUProfile[0] can be NULL and other array elements can be uninitialized in PROFILE_End(). The change should look something like this:

diff --git a/storage/connect/inihandl.c b/storage/connect/inihandl.c
index 542b807..9ec566c 100644
--- a/storage/connect/inihandl.c
+++ b/storage/connect/inihandl.c
@@ -622,13 +622,14 @@ void PROFILE_End(void)
   if (trace)
     htrc("PROFILE_End: CurProfile=%p N=%d\n", CurProfile, N_CACHED_PROFILES);
 
+  if (!CurProfile)
+    return;
+
   /* Close all opened files and free the cache structure */
   for (i = 0; i < N_CACHED_PROFILES; i++) {
     if (trace)
       htrc("MRU=%s i=%d\n", SVP(MRUProfile[i]->filename), i);
 
-    CurProfile = MRUProfile[i];
-    PROFILE_ReleaseFile();
     free(MRUProfile[i]);
     } // endfor i

Comment by Olivier Bertrand [ 2016-05-13 ]

Thanks to Sergey Vojtovich

Generated at Thu Feb 08 07:38:53 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.