[MDEV-8965] cluster node (async replication slave) high memory usage Created: 2015-10-19  Updated: 2015-12-14  Resolved: 2015-12-14

Status: Closed
Project: MariaDB Server
Component/s: Galera, Storage Engine - InnoDB, wsrep
Affects Version/s: 10.0.21-galera
Fix Version/s: 10.0.23-galera

Type: Bug Priority: Blocker
Reporter: aftab khan Assignee: Nirbhay Choubey (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Attachments: File massif_cluster_async_slave_node.out     File massif_cluster_normal_node_.out    
Issue Links:
Problem/Incident
is caused by MDEV-9162 MariaDB Galera Cluster memory leak on... Closed

 Description   

We are seeing high memory usage on cluster member running as async replication slave node, whereas other cluster member's memory usage is normal.

– Cluster member asynchronous replication slave node

{{tprod-dba-test-sa-03:~$ top -n5 | grep mysql
32021 mysql 20 0 6161m 5.3g 115m S 3 67.4 562:46.43 massif-amd64-li
32021 mysql 20 0 6161m 5.3g 115m S 1 67.4 562:46.47 massif-amd64-li
32021 mysql 20 0 6161m 5.3g 115m S 10 67.4 562:46.77 massif-amd64-li
32021 mysql 20 0 6161m 5.3g 115m S 5 67.4 562:46.93 massif-amd64-li}}

— Cluster member normal node

{{itprod-dba-test-sa-02:~$ top -n5 | grep mysql
8707 mysql 20 0 2115m 1.4g 137m S 2 18.4 350:05.71 massif-amd64-li
8707 mysql 20 0 2115m 1.4g 137m S 3 18.4 350:05.79 massif-amd64-li
8707 mysql 20 0 2115m 1.4g 137m S 1 18.4 350:05.82 massif-amd64-li
8707 mysql 20 0 2115m 1.4g 137m S 1 18.4 350:05.84 massif-amd64-li
8707 mysql 20 0 2115m 1.4g 137m S 2 18.4 350:05.89 massif-amd64-li}}

I profiled memory usage of 2x nodes cluster using Valgrind massif tool:
https://www.percona.com/blog/2013/01/09/profiling-mysql-memory-usage-with-valgrind-massif/

ms_print outputs are attached:
1. massif_cluster_normal_node_.out
2. massif_cluster_asych_slave_node.out



 Comments   
Comment by aftab khan [ 2015-10-19 ]

memory usage on cluster member (running as async replication slave) steadily increase and then server starts swapping, so after almost 2 weeks or so we have to restart mysql to fix this issue.

Comment by Elena Stepanova [ 2015-10-19 ]

Thanks for the report.

nirbhay_c, at the very first glance it looks like the problem might be related to Galera/wsrep, could you please check it out? If it turns out to be unrelated, please reassign to Jan as InnoDB will be the next suspect.

Comment by Nirbhay Choubey (Inactive) [ 2015-12-14 ]

https://github.com/MariaDB/server/commit/18173ddfc4081407832d9a6703d1b8356b7defe9

Generated at Thu Feb 08 07:31:07 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.