[MDEV-32045] memory accumulation in the docker environment in 10.6.14 Created: 2023-08-30  Updated: 2023-10-27  Resolved: 2023-10-27

Status: Closed
Project: MariaDB Server
Component/s: Docker
Affects Version/s: 10.6.14, 10.11.5
Fix Version/s: N/A

Type: Bug Priority: Critical
Reporter: Yakov Kushnirsky Assignee: Ben Grieser
Resolution: Incomplete Votes: 1
Labels: Memory_leak
Environment:

docker


Attachments: HTML File lab_cmdb0_memory     HTML File lab_cmdb1_memory     HTML File lab_cmdb2_memory     Text File mariadb-1_current_bytes.txt     Text File mariadb-1_engine_inno.txt     Text File mariadb-1_global_stats.txt     Text File mariadb-1_global_vars.txt     Text File mariadb-1_meminfo.txt     Text File mariadb-1_memory_summary.txt     Text File mariadb-2_current_bytes.txt     Text File mariadb-2_engine_inno.txt     Text File mariadb-2_global_stats.txt     Text File mariadb-2_global_vars.txt     Text File mariadb-2_meminfo.txt     Text File mariadb-2_memory_summary.txt     HTML File second_run_cmdb0_memory     HTML File second_run_cmdb1_memory     HTML File second_run_cmdb2_memory     HTML File test_cmdb0_memory     HTML File test_cmdb1_memory     Text File test_cmdb2_memory.txt    

 Description   

Customer reported memory leak in the docker environment in 10.6.14 track and provided numerous performance_schema.memory_summary_global_by_event_name
outputs

According to Customer the issues is beyond existing JIRA reports for 10.6 track



 Comments   
Comment by Daniel Black [ 2023-08-31 ]

while its in Docker, the memory leak isn't going to be docker related, its all in server.

jemalloc is in the container so testing its capabilities of leak detection is one option.

LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2 as an environment variable to the container runtime will enable it along with the desired MALLOC_CONF.

Comment by Oleksandr Byelkin [ 2023-08-31 ]

JFYI Dmitri have yet another 4 melory leak fixed and continue. That memory leak can not be detected by usual mean because if dissconect memory will be freed.

So the question, is memory freed in this case if reconnect/dissconnect all clients?

Comment by Yakov Kushnirsky [ 2023-08-31 ]

Thank you all! I asked Customer in the case 1)if disconnecting all clients will free memory; 2)use jemalloc for leak detection. Will update JIRA again with his answers.
Adding available global and status variables to the JIRA to supplement posted before P_S reports (all for the same datetime - 8/21).

Comment by Michael Widenius [ 2023-09-07 ]

I cannot find any information in the provided stats that show any memory leaks over time.
Please ask the user to provide us with
show global status like "memory_used%";
select * from information_schema.processlist;

Run every 60 seconds to show what memory is growing.

Comment by Ben Grieser [ 2023-09-07 ]

I've attached memory used information provided by the user in CS0574876. I think this is collected in three different runs. The files are prepended with lab, test, and second_run, and were supplied by the user on different dates in that order.

Generated at Thu Feb 08 10:28:25 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.