[MDEV-27077] Large thread-stack size causes near-perpetual loop of memory blocks lost for mysql_install_db and mariadbd Created: 2021-11-18  Updated: 2021-11-18  Resolved: 2021-11-18

Status: Closed
Project: MariaDB Server
Component/s: Server
Affects Version/s: 10.2, 10.3, 10.4, 10.5, 10.6, 10.7, 10.8
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Roel Van de Paar Assignee: Sergei Golubchik
Resolution: Not a Bug Votes: 0
Labels: None


 Description   

Starting mysql_install_db or mariadbd with a large thread-stack size will result in an almost perpetual loop of memory blocks lost in a variety of code places.

$ rm -Rf /test/MD121121-mariadb-10.7.2-linux-x86_64-dbg/data
$ /test/MD121121-mariadb-10.7.2-linux-x86_64-dbg/scripts/mariadb-install-db --no-defaults --force --auth-root-authentication-method=normal  ${MYEXTRA_OPT} --basedir=/test/MD121121-mariadb-10.7.2-linux-x86_64-dbg --datadir=/test/MD121121-mariadb-10.7.2-linux-x86_64-dbg/data --thread-stack=1125899906842624

Leads to:

10.8.0 5566cbadb03856aba9c236b131f544490cd2bee4 (Debug, UBASAN)

Installing MariaDB/MySQL system tables in '/test/MD121121-mariadb-10.7.2-linux-x86_64-dbg/data' ...
2021-11-18 11:55:41 0 [ERROR] Can't create interrupt-thread (error 11, errno: 12)
Warning:   80 bytes lost at 0x55f60d1f1470, allocated by T@0 at 0x55f60b547527, mysys/thr_mutex.c:342, psi/mysql_thread.h:750, sql/log.cc:1057, sql/log.cc:1168, sql/log.cc:6807, sql/log.cc:9174, sql/mysqld.cc:3066
Warning: 2072 bytes lost at 0x55f60d1f37a0, allocated by T@0 at sql/main.cc:36, mysys/array.c:72, mysys/hash.c:98, mysys/thr_mutex.c:175, psi/mysql_thread.h:750, sql/log.cc:1057, sql/log.cc:1168, sql/log.cc:6807
Warning: 2072 bytes lost at 0x55f60d1f2f00, allocated by T@0 at sql/log.cc:9174, mysys/array.c:72, mysys/hash.c:98, mysys/thr_mutex.c:175, psi/mysql_thread.h:750, sql/log.cc:1057, sql/log.cc:1168, sql/log.cc:6807
Warning:  280 bytes lost at 0x55f60d1f12d0, allocated by T@0 at sql/log.cc:9174, mysys/mulalloc.c:51, mysys/thr_mutex.c:159, psi/mysql_thread.h:750, sql/log.cc:1057, sql/log.cc:1168, sql/log.cc:6807, sql/log.cc:9174
Warning: 2072 bytes lost at 0x55f60d1f2660, allocated by T@0 at sql/mysqld.cc:3066, mysys/array.c:72, mysys/hash.c:98, mysys/thr_mutex.c:175, psi/mysql_thread.h:750, csu/libc-start.c:342, ??:0
Warning: 2072 bytes lost at 0x55f60d1f1dc0, allocated by T@0 at mysys/array.c:72, mysys/hash.c:98, mysys/thr_mutex.c:175, psi/mysql_thread.h:750, sql/main.cc:36, csu/libc-start.c:342, ??:0
Warning:  280 bytes lost at 0x55f60d1f0de0, allocated by T@0 at mysys/mulalloc.c:51, mysys/thr_mutex.c:159, psi/mysql_thread.h:750, sql/main.cc:36, csu/libc-start.c:342, ??:0
Warning: 2440 bytes lost at 0x55f60d1ef8d0, allocated by T@0 at mysys/array.c:72, mysys/hash.c:98, sql/sql_connect.cc:496, sql/mysqld.cc:5406, sql/mysqld.cc:5695, sql/main.cc:36, csu/libc-start.c:342, ??:0
Warning: 2440 bytes lost at 0x55f60d1eeec0, allocated by T@0 at mysys/array.c:72, mysys/hash.c:98, sql/sql_connect.cc:489, sql/mysqld.cc:5405, sql/mysqld.cc:5695, sql/main.cc:36, csu/libc-start.c:342, ??:0
Warning: 2440 bytes lost at 0x55f60d1ee4b0, allocated by T@0 at mysys/array.c:72, mysys/hash.c:98, sql/sql_connect.cc:329, sql/mysqld.cc:5404, sql/mysqld.cc:5695, sql/main.cc:36, csu/libc-start.c:342, ??:0
Warning: 8208 bytes lost at 0x55f60d1ec420, allocated by T@0 at mysys/my_alloc.c:69, mysys/my_alloc.c:332, mysys/tree.c:281, myisam/ft_stopwords.c:53, myisam/ft_stopwords.c:118, sql/mysqld.cc:5401, sql/mysqld.cc:5695, sql/main.cc:36
Warning: 2072 bytes lost at 0x152314003110, allocated by T@0 at mysys/array.c:72, mysys/hash.c:98, mysys/thr_mutex.c:175, srv/srv0srv.cc:2006, trx/trx0purge.cc:1294, srv/srv0srv.cc:1814, srv/srv0srv.cc:1970, tpool/task_group.cc:55
Warning: 2072 bytes lost at 0x152314002870, allocated by T@0 at mysys/array.c:72, mysys/hash.c:98, mysys/thr_mutex.c:175, srv/srv0srv.cc:2006, trx/trx0purge.cc:1294, srv/srv0srv.cc:1814, srv/srv0srv.cc:1970, tpool/task_group.cc:55
Warning: 8208 bytes lost at 0x55f60d1ea390, allocated by T@0 at mysys/my_alloc.c:69, mysys/my_alloc.c:332, mysys/tree.c:281, myisam/ft_stopwords.c:53, myisam/ft_stopwords.c:118, sql/mysqld.cc:5401, sql/mysqld.cc:5695, sql/main.cc:36
Warning:  280 bytes lost at 0x1523140026d0, allocated by T@0 at mysys/mulalloc.c:51, mysys/thr_mutex.c:159, srv/srv0srv.cc:2006, trx/trx0purge.cc:1294, srv/srv0srv.cc:1814, srv/srv0srv.cc:1970, tpool/task_group.cc:55, tpool/task.cc:40
Warning: 8208 bytes lost at 0x55f60d1e8300, allocated by T@0 at mysys/my_alloc.c:69, mysys/my_alloc.c:332, mysys/tree.c:281, myisam/ft_stopwords.c:53, myisam/ft_stopwords.c:118, sql/mysqld.cc:5401, sql/mysqld.cc:5695, sql/main.cc:36
Warning:  688 bytes lost at 0x55f60d1e7fd0, allocated by T@0 at myisam/ft_stopwords.c:62, sql/mysqld.cc:5401, sql/mysqld.cc:5695, sql/main.cc:36, csu/libc-start.c:342, ??:0
...etc...

Killing the process can make it go <defunct>.

An ASAN run provides us upfront with:

10.8.0 5566cbadb03856aba9c236b131f544490cd2bee4 (Debug, UBASAN)

/test/10.8_dbg_san/sql/sql_class.cc:2192:44: runtime error: pointer index expression with base 0x7ffe27749470 overflowed to 0xfffc7ffe27749470
2021-11-18 12:05:58 0 [Note] InnoDB: Buffer pool(s) load completed at 211118 12:05:58
    #0 0x5651ac77d982 in THD::store_globals() /test/10.8_dbg_san/sql/sql_class.cc:2192
    #1 0x5651acc7989b in plugin_load /test/10.8_dbg_san/sql/sql_plugin.cc:1846
    #2 0x5651acc7989b in plugin_init(int*, char**, int) /test/10.8_dbg_san/sql/sql_plugin.cc:1769
    #3 0x5651ac192b65 in init_server_components /test/10.8_dbg_san/sql/mysqld.cc:5080
    #4 0x5651ac1a9853 in mysqld_main(int, char**) /test/10.8_dbg_san/sql/mysqld.cc:5695
    #5 0x5651ac178b7a in main /test/10.8_dbg_san/sql/main.cc:34
    #6 0x1492b61160b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
    #7 0x5651ac04c57d in _start (/test/UBASAN_MD151121-mariadb-10.8.0-linux-x86_64-dbg/bin/mariadbd+0x825157d)

Followed by all the memory lost notices and finally summaries for each of them:

10.8.0 5566cbadb03856aba9c236b131f544490cd2bee4 (Debug, UBASAN)

=================================================================
==2140885==ERROR: LeakSanitizer: detected memory leaks
... long list ...



 Comments   
Comment by Sergei Golubchik [ 2021-11-18 ]

With a large thread-stack, you get

2021-11-18 11:55:41 0 [ERROR] Can't create interrupt-thread (error 11, errno: 12)

and the server exits immediately. It doesn't bother to free all allocated memory, because OS will do it anyway, so safemalloc shows spurious "memory leaks".

Generated at Thu Feb 08 09:50:09 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.