[#MDEV-30848] Memory leak in MariaDB 10.6

I already reported about "memory leaks" we have seen after upgrade from 10.3 to 10.6 in comments of bugs ~~MDEV-30096~~ and ~~MDEV-29988~~ but it seems there are still another one.
Our servers run with the same applications and the same workload for a long time but since we upgraded MariaDB to 10.6 we see OOM crashes too often and even if we already doubled the RAM sizes of affected VMs.

This machine has configuration where it can eat up max 17GB of memory (if this calculation is right and if there is 100 connections)

:mysql> SELECT @@key_buffer_size / 1048576,

    -> @@query_cache_size / 1048576,

    -> @@innodb_buffer_pool_size / 1048576,

    -> @@innodb_log_buffer_size / 1048576,

    -> @@max_connections,

    -> @@read_buffer_size / 1048576,

    -> @@read_rnd_buffer_size / 1048576,

    -> @@sort_buffer_size / 1048576,

    -> @@join_buffer_size / 1048576,

    -> @@binlog_cache_size / 1048576,

    -> @@thread_stack / 1048576,

    -> @@tmp_table_size / 1048576,

    -> ( @@key_buffer_size

    -> + @@query_cache_size

    -> + @@innodb_buffer_pool_size

    -> + @@innodb_log_buffer_size

    -> + @@max_connections * (

    ->  @@read_buffer_size

    ->  + @@read_rnd_buffer_size

    ->  + @@sort_buffer_size

    ->  + @@join_buffer_size

    ->  + @@binlog_cache_size

    ->  + @@thread_stack

    ->  + @@tmp_table_size )

    -> ) / (1024 * 1024 * 1024) AS MAX_MEMORY_GB\G

*************************** 1. row ***************************

        @@key_buffer_size / 1048576: 128.0000

       @@query_cache_size / 1048576: 1.0000

@@innodb_buffer_pool_size / 1048576: 1024.0000

 @@innodb_log_buffer_size / 1048576: 8.0000

                  @@max_connections: 100

       @@read_buffer_size / 1048576: 1.0000

   @@read_rnd_buffer_size / 1048576: 2.0000

       @@sort_buffer_size / 1048576: 8.0000

       @@join_buffer_size / 1048576: 2.0000

      @@binlog_cache_size / 1048576: 1.0000

           @@thread_stack / 1048576: 0.2852

         @@tmp_table_size / 1048576: 150.0000

                      MAX_MEMORY_GB: 17.1773

But even if there is no more than 5 connections (listed by SHOW PROCESSLIST or tool mytop) MariaDB eats more than 18GB and than was killed by OOM.

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND

 1632 mysql        37  20    0    21G    18G select   3  44:45  62.89% /usr/local/libexec/mariadbd --defaults-extra-file=/usr/local/etc/mysql/my.cnf --basedir=/usr/local --datadir=/var/db/mysql --plugin-dir=/usr/local/lib/mysql/plugin --wsrep_on=1 --wsrep_provider=none --wsrep_on=1 --wsrep_provider=none --log-error=/var/db/mysql/mysql.err --pid-file=/var/run/mysql/mysqld.pid --socket=/var/run/mysql/mysql.sock --port=3306

If I will take the maximum connections seen today = 14 * 164MB per thread + 1161 MB global it should not take more than 3.5GB!
We have graphs in monitoring where we see connections (max 14) and queries per second (49 avg, 330 max)

We see this behavior on other machines with MariaDB 10.6.12 too. The more RAM we add to the VM the more MariaDB uses and crashes often.

What is the right way to identify what in MariaDB eats so much memory and not release it? (There are no long running persistent connections but periodic tasks running from cron)

[MDEV-30848] Memory leak in MariaDB 10.6 Created: 2023-03-14 Updated: 2023-11-28 Resolved: 2023-11-28
Status:	Closed
Project:	MariaDB Server
Component/s:	Server
Affects Version/s:	10.6.12
Fix Version/s:	N/A

[MDEV-30848] Memory leak in MariaDB 10.6 Created: 2023-03-14 Updated: 2023-11-28 Resolved: 2023-11-28