When MySQLdb (python-mysqldb) uses the MariaDB libmysqlclient.so.18 libraries, it seems to leak memory with each failed connection. A simple contrived loop (w/ mariadb stopped, generating client 2003 errors) leads to a few GB RSS for the associated python process in a few seconds when using the MariaDB 10.1 client packages:
#!/usr/bin/env python
|
|
import resource
|
import time
|
|
import MySQLdb
|
|
print("Client version: %s" % MySQLdb.get_client_info())
|
|
max_allowed_rss = 512*1024
|
failed_connects = 0
|
initial_rss = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
|
|
for _ in xrange(50000):
|
try:
|
MySQLdb.connect(user='root', host='127.0.0.1')
|
except MySQLdb.Error:
|
failed_connects += 1
|
|
print("maxrss initial was: %sKB, final is %sKB; %.2fMiB change after %d failed connections." %
|
(
|
initial_rss,
|
resource.getrusage(resource.RUSAGE_SELF).ru_maxrss,
|
(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss - initial_rss) / 1024.0,
|
failed_connects
|
)
|
)
|
The output of running this is as follows:
$ apt-get install libmysqlclient18=5.5.50-0+deb8u1
|
$ python mdev10455_mysqldb_memleak.py
|
Client version: 5.5.50
|
maxrss initial was: 8364KB, final is 10044KB; 1.64MiB change after 50000 failed connections.
|
|
$ apt-get install libmysqlclient18=5.6.30-1~bpo8+1
|
$ python mdev10455_mysqldb_memleak.py
|
Client version: 5.6.30
|
maxrss initial was: 8536KB, final is 10216KB; 1.64MiB change after 50000 failed connections.
|
|
$ apt-get install libmysqlclient18=10.1.16+maria-1~jessie
|
$ python mdev10455_mysqldb_memleak.py
|
Client version: 10.1.16-MariaDB
|
maxrss initial was: 9272KB, final is 437896KB; 418.58MiB change after 50000 failed connections.
|
|
Removing libmariadbclient18 and reverting to the libmysqlclient18 provided by either MySQL 5.5 (5.5.50-0+deb8u1) (debian/jessie main) or MySQL 5.6 (5.6.30-1~bpo8+1 from jessie-backports), the memory leak goes away and memory use stays constant over time.
I see MySQL-python does not call mysql_close() on a failed connection which seems to resolve this behavior with MariaDB - so perhaps this is an incorrect API usage bug. However the MariaDB behavior does seem to differ from the MySQL (5.5.50, 5.6.30) behavior, so I am opening this issue.
Bug
Major
MDEV-10546 Make unit tests run with valgrind
