[MDEV-24509] Warning: Memory not freed: 56 on SET @@global.wsrep_sst_auth Created: 2021-01-01  Updated: 2021-06-28  Resolved: 2021-01-26

Status: Closed
Project: MariaDB Server
Component/s: Galera SST, wsrep
Affects Version/s: 10.2, 10.3, 10.4, 10.5, 10.6
Fix Version/s: 10.2.37, 10.3.28, 10.4.18, 10.5.9

Type: Bug Priority: Major
Reporter: Roel Van de Paar Assignee: Roel Van de Paar
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-26030 Warning: Memory not freed: 32 on sett... Closed

 Description   

SET @@global.wsrep_sst_auth= 'abcdefghijklmnopqrstuvwxyz';
SHUTDOWN;

Leads to:

10.6.0 9118fd360a3da0bba521caf2a35c424968235ac4 (Optimized)

2021-01-01 17:17:07 0 [Note] InnoDB: Starting shutdown...
2021-01-01 17:17:07 0 [Note] InnoDB: Dumping buffer pool(s) to /test/MD010121-mariadb-10.6.0-linux-x86_64-opt/data/ib_buffer_pool
2021-01-01 17:17:07 0 [Note] InnoDB: Buffer pool(s) dump completed at 210101 17:17:07
2021-01-01 17:17:07 0 [Note] InnoDB: Shutdown completed; log sequence number 42658; transaction id 19
2021-01-01 17:17:07 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2021-01-01 17:17:07 0 [Note] /test/MD010121-mariadb-10.6.0-linux-x86_64-opt/bin/mysqld: Shutdown complete
 
Warning: Memory not freed: 56

Different versions give different numbers of bytes. Debug builds give bit more info:

10.4.18 3454b5cf35a61e8f6cfab376638520dee4a50609 (Debug)

Warning: Memory not freed: 32
Warning:   32 bytes lost at 0x153344011c50, allocated by T@0 at mysys/my_malloc.c:245, sql/wsrep_sst.cc:248, sql/wsrep_sst.cc:284, sql/set_var.cc:209, sql/set_var.cc:837, sql/set_var.cc:740, sql/sql_parse.cc:4994, sql/sql_parse.cc:7958
Memory lost: 32 bytes in 1 chunks
Warning:   32 bytes lost at 0x153344011c50, allocated by T@0 at mysys/my_malloc.c:245, sql/wsrep_sst.cc:248, sql/wsrep_sst.cc:284, sql/set_var.cc:209, sql/set_var.cc:837, sql/set_var.cc:740, sql/sql_parse.cc:4994, sql/sql_parse.cc:7958
Memory lost: 32 bytes in 1 chunks



 Comments   
Comment by Roel Van de Paar [ 2021-01-01 ]

SET @@global.wsrep_sst_auth= '';
SHUTDOWN;

Can also be used

Comment by Jan Lindström (Inactive) [ 2021-01-13 ]

Do I need some special setting as I can't repeat?

Comment by Roel Van de Paar [ 2021-01-14 ]

jplindst I used the CLI for reproduction, not MTR - does that help? And the messages show up as the last few lines of the error log after shutdown.

Also quickly retested and bug is there. Let me know if you want me to test against latest trunk, but was still present on 8 Jan 2021 from what I can see.

Comment by Jan Lindström (Inactive) [ 2021-01-14 ]

I used CLI and mysqladmin and I can't reproduce.

Comment by Roel Van de Paar [ 2021-01-15 ]

Interesting.

OS:

Ubuntu 20.04.1 LTS (Focal Fossa)

Env:

117GB RAM, 32 Tthread CPU, gcc Version 9.3.0  

cmake:

cmake . -DWITH_SSL=bundled -DCMAKE_BUILD_TYPE=Debug -DBUILD_CONFIG=mysql_release -DWITH_TOKUDB=0 -DWITH_JEMALLOC=no -DFEATURE_SET=community -DDEBUG_EXTNAME=OFF -DWITH_EMBEDDED_SERVER=0 -DENABLE_DOWNLOADS=1 -DDOWNLOAD_BOOST=1 -DWITH_BOOST=/tmp/boost_193297 -DENABLED_LOCAL_INFILE=1 -DENABLE_DTRACE=0 -DWITH_PERFSCHEMA_STORAGE_ENGINE=1 -DWITH_ZLIB=bundled -DWITH_ROCKSDB=1 -DWITH_PAM=ON -DFORCE_INSOURCE_BUILD=1 -DCMAKE_C_FLAGS='-Og -march=native -mtune=native' -DCMAKE_CXX_FLAGS='-Og -march=native -mtune=native' -DMYSQL_MAINTAINER_MODE=OFF

Retested with latest 10.6 trunk in debug, using CLI (no mysqladmin needed as SHUTDOWN works), and issue remains:

10.6.0 0dfabf6e409056b02e37d4503d3517725024f90d (Debug)

2021-01-15 14:19:36 0 [Note] InnoDB: Shutdown completed; log sequence number 43218; transaction id 19
2021-01-15 14:19:36 0 [Note] /test/MD150121-mariadb-10.6.0-linux-x86_64-dbg/bin/mysqld: Shutdown complete
Warning: Memory not freed: 56
Warning:   56 bytes lost at 0x14f478012390, allocated by T@0 at 0x55a9c4b123b8, mysys/my_malloc.c:231, sql/wsrep_sst.cc:249, sql/wsrep_sst.cc:285, sql/set_var.cc:208, sql/set_var.cc:859, sql/set_var.cc:746, sql/sql_parse.cc:4919
Memory lost: 56 bytes in 1 chunks
Warning:   56 bytes lost at 0x14f478012390, allocated by T@0 at sql/sql_parse.cc:7901, mysys/my_malloc.c:231, sql/wsrep_sst.cc:249, sql/wsrep_sst.cc:285, sql/set_var.cc:208, sql/set_var.cc:859, sql/set_var.cc:746, sql/sql_parse.cc:4919
Memory lost: 56 bytes in 1 chunks

Though notice the debug message has a different bytes lost size, and line number, now.

CLI command:

/test/MD150121-mariadb-10.6.0-linux-x86_64-dbg/bin/mysql -A -uroot -S/test/MD150121-mariadb-10.6.0-linux-x86_64-dbg/socket.sock --force '--prompt=10.6.0>' --binary-mode test

Start command:

/test/MD150121-mariadb-10.6.0-linux-x86_64-dbg/bin/mysqld --no-defaults --core-file --basedir=/test/MD150121-mariadb-10.6.0-linux-x86_64-dbg --tmpdir=/test/MD150121-mariadb-10.6.0-linux-x86_64-dbg/data --datadir=/test/MD150121-mariadb-10.6.0-linux-x86_64-dbg/data --socket=/test/MD150121-mariadb-10.6.0-linux-x86_64-dbg/socket.sock --port=11710 --log-error=/test/MD150121-mariadb-10.6.0-linux-x86_64-dbg/log/master.err --server-id=100

Comment by Roel Van de Paar [ 2021-01-18 ]

An ASAN run produces similar issues, but generates no additional information

10.6.0 c2ea036b7f4fc4200f168bf53ad3afd208370459 (Debug)

Warning: Memory not freed: 56
Warning:   56 bytes lost at 0x60f000010280, allocated by T@0 at 0x556202a5861e, 0x556202a599af,

10.6.0 c2ea036b7f4fc4200f168bf53ad3afd208370459 (Optimized)

Warning: Memory not freed: 32

The issue also happen when only executing:

SET @@global.wsrep_sst_auth= '';

And then using mysqladmin to shutdown the server. It does not happen if only SHUTDOWN; is issued without setting global.wsrep_sst_auth

Comment by Jan Lindström (Inactive) [ 2021-01-20 ]

Roel Can you try with branch bb-10.6-MDEV-24509 I still can't repeat. Please use -mysqld=-wsrep-debug=1

Comment by Roel Van de Paar [ 2021-01-21 ]

jplindst Thank you & done. Said branch also reproduces the issue.
First testcase

10.6.0 12d3f6c635a5f43609d5d38462db81810edb5358 (Debug)

Warning: Memory not freed: 56
Warning:   56 bytes lost at 0x14875c012390, allocated by T@0 at 0x55b5295ba51c, mysys/my_malloc.c:231, sql/wsrep_sst.cc:249, sql/wsrep_sst.cc:295, sql/set_var.cc:208, sql/set_var.cc:859, sql/set_var.cc:746, sql/sql_parse.cc:4919
Memory lost: 56 bytes in 1 chunks
Warning:   56 bytes lost at 0x14875c012390, allocated by T@0 at sql/sql_parse.cc:7901, mysys/my_malloc.c:231, sql/wsrep_sst.cc:249, sql/wsrep_sst.cc:295, sql/set_var.cc:208, sql/set_var.cc:859, sql/set_var.cc:746, sql/sql_parse.cc:4919
Memory lost: 56 bytes in 1 chunks

Second testcase

10.6.0 12d3f6c635a5f43609d5d38462db81810edb5358 (Debug)

Warning: Memory not freed: 32
Warning:   32 bytes lost at 0x14feb0009160, allocated by T@0 at 0x557c8f63151c, mysys/my_malloc.c:231, sql/wsrep_sst.cc:249, sql/wsrep_sst.cc:295, sql/set_var.cc:208, sql/set_var.cc:859, sql/set_var.cc:746, sql/sql_parse.cc:4919
Memory lost: 32 bytes in 1 chunks
Warning:   32 bytes lost at 0x14feb0009160, allocated by T@0 at sql/sql_parse.cc:7901, mysys/my_malloc.c:231, sql/wsrep_sst.cc:249, sql/wsrep_sst.cc:295, sql/set_var.cc:208, sql/set_var.cc:859, sql/set_var.cc:746, sql/sql_parse.cc:4919
Memory lost: 32 bytes in 1 chunks

Comment by Roel Van de Paar [ 2021-01-21 ]

Also figured out that the size relates to the size of the string used.

10.6.0>SET @@global.wsrep_sst_auth= '0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789';   # Length: 100

10.6.0 12d3f6c635a5f43609d5d38462db81810edb5358 (Debug)

Warning: Memory not freed: 128
Warning:  128 bytes lost at 0x1528340091c0, allocated by T@0 at 0x5633fefc651c, mysys/my_malloc.c:231, sql/wsrep_sst.cc:249, sql/wsrep_sst.cc:295, sql/set_var.cc:208, sql/set_var.cc:859, sql/set_var.cc:746, sql/sql_parse.cc:4919
Memory lost: 128 bytes in 1 chunks
Warning:  128 bytes lost at 0x1528340091c0, allocated by T@0 at sql/sql_parse.cc:7901, mysys/my_malloc.c:231, sql/wsrep_sst.cc:249, sql/wsrep_sst.cc:295, sql/set_var.cc:208, sql/set_var.cc:859, sql/set_var.cc:746, sql/sql_parse.cc:4919
Memory lost: 128 bytes in 1 chunks

Comment by Roel Van de Paar [ 2021-01-22 ]

jplindst Error log attached as requested! Thank you

Comment by Jan Lindström (Inactive) [ 2021-01-25 ]

Thanks, can you test again with bb-10.6-MDEV-24509 branch. I did test case but still can't repeat.

Generated at Thu Feb 08 09:30:32 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.