Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
10.4.13
-
None
Description
Galera wsrep layer leaks memory on nodes receiving writesets.
For some reason any issued DDL on master makes such nodes release some of the leaked memory (but doesn't stop the leak).
This is affecting a customer of ours.
It is clearly seen that the leak is in wsrep layer because if node gets evicted (wsrep layer is shut down but mysqld stays up) all memory is released instantly.
Besides, the nodes where the leak appears are receiving NO queries at all.
Leak only affects nodes receiving writesets via wsrep layer - master is NOT affected.
How to reproduce: 3 Centos7 nodes, 1G memory on each, use supplied sysbench script and leave it running for few hours.
sysbench --db-driver=mysql --mysql-host=localhost --mysql-user=root code.lua --tables=16 prepare
|
sysbench --db-driver=mysql --mysql-host=localhost --mysql-user=root code.lua --tables=16 --threads=64 --time=0 run
|
Attachments
Issue Links
- relates to
-
-
- Closed
-
Activity
| Field | Original Value | New Value |
|---|---|---|
| Attachment | my.cnf [ 52336 ] |
| Attachment | code.lua [ 52337 ] |
| Description |
Galera wsrep layer leaks memory on nodes receiving writesets.
For some reason any issued DDL on master makes such nodes release some of the leaked memory (but doesn't stop the leak). This is affecting a customer of ours. It is clearly seen that the leak is in wsrep layer because if node gets evicted (wsrep layer is shut down but mysqld stays up) all memory is released instantly. Besides, the nodes where the leak appears are receiving NO traffic at all. How to reproduce: 3 Centos7 nodes, 1G memory on each, use supplied sysbench script and leave it running for few hours. {code} sysbench --db-driver=mysql --mysql-host=localhost --mysql-user=root code.lua --tables=16 prepare 2:38 sysbench --db-driver=mysql --mysql-host=localhost --mysql-user=root code.lua --tables=16 --threads=64 --time=0 run {code} |
Galera wsrep layer leaks memory on nodes receiving writesets.
For some reason any issued DDL on master makes such nodes release some of the leaked memory (but doesn't stop the leak). This is affecting a customer of ours. It is clearly seen that the leak is in wsrep layer because if node gets evicted (wsrep layer is shut down but mysqld stays up) all memory is released instantly. Besides, the nodes where the leak appears are receiving NO traffic at all. Leak only affects nodes receiving writesets via wsrep layer - master is NOT affected. How to reproduce: 3 Centos7 nodes, 1G memory on each, use supplied sysbench script and leave it running for few hours. {code} sysbench --db-driver=mysql --mysql-host=localhost --mysql-user=root code.lua --tables=16 prepare 2:38 sysbench --db-driver=mysql --mysql-host=localhost --mysql-user=root code.lua --tables=16 --threads=64 --time=0 run {code} |
| Description |
Galera wsrep layer leaks memory on nodes receiving writesets.
For some reason any issued DDL on master makes such nodes release some of the leaked memory (but doesn't stop the leak). This is affecting a customer of ours. It is clearly seen that the leak is in wsrep layer because if node gets evicted (wsrep layer is shut down but mysqld stays up) all memory is released instantly. Besides, the nodes where the leak appears are receiving NO traffic at all. Leak only affects nodes receiving writesets via wsrep layer - master is NOT affected. How to reproduce: 3 Centos7 nodes, 1G memory on each, use supplied sysbench script and leave it running for few hours. {code} sysbench --db-driver=mysql --mysql-host=localhost --mysql-user=root code.lua --tables=16 prepare 2:38 sysbench --db-driver=mysql --mysql-host=localhost --mysql-user=root code.lua --tables=16 --threads=64 --time=0 run {code} |
Galera wsrep layer leaks memory on nodes receiving writesets.
For some reason any issued DDL on master makes such nodes release some of the leaked memory (but doesn't stop the leak). This is affecting a customer of ours. It is clearly seen that the leak is in wsrep layer because if node gets evicted (wsrep layer is shut down but mysqld stays up) all memory is released instantly. Besides, the nodes where the leak appears are receiving NO queries at all. Leak only affects nodes receiving writesets via wsrep layer - master is NOT affected. How to reproduce: 3 Centos7 nodes, 1G memory on each, use supplied sysbench script and leave it running for few hours. {code} sysbench --db-driver=mysql --mysql-host=localhost --mysql-user=root code.lua --tables=16 prepare 2:38 sysbench --db-driver=mysql --mysql-host=localhost --mysql-user=root code.lua --tables=16 --threads=64 --time=0 run {code} |
| Description |
Galera wsrep layer leaks memory on nodes receiving writesets.
For some reason any issued DDL on master makes such nodes release some of the leaked memory (but doesn't stop the leak). This is affecting a customer of ours. It is clearly seen that the leak is in wsrep layer because if node gets evicted (wsrep layer is shut down but mysqld stays up) all memory is released instantly. Besides, the nodes where the leak appears are receiving NO queries at all. Leak only affects nodes receiving writesets via wsrep layer - master is NOT affected. How to reproduce: 3 Centos7 nodes, 1G memory on each, use supplied sysbench script and leave it running for few hours. {code} sysbench --db-driver=mysql --mysql-host=localhost --mysql-user=root code.lua --tables=16 prepare 2:38 sysbench --db-driver=mysql --mysql-host=localhost --mysql-user=root code.lua --tables=16 --threads=64 --time=0 run {code} |
Galera wsrep layer leaks memory on nodes receiving writesets.
For some reason any issued DDL on master makes such nodes release some of the leaked memory (but doesn't stop the leak). This is affecting a customer of ours. It is clearly seen that the leak is in wsrep layer because if node gets evicted (wsrep layer is shut down but mysqld stays up) all memory is released instantly. Besides, the nodes where the leak appears are receiving NO queries at all. Leak only affects nodes receiving writesets via wsrep layer - master is NOT affected. How to reproduce: 3 Centos7 nodes, 1G memory on each, use supplied sysbench script and leave it running for few hours. {code} sysbench --db-driver=mysql --mysql-host=localhost --mysql-user=root code.lua --tables=16 prepare sysbench --db-driver=mysql --mysql-host=localhost --mysql-user=root code.lua --tables=16 --threads=64 --time=0 run {code} |
| Assignee | Jan Lindström [ jplindst ] |
| Fix Version/s | 10.4 [ 22408 ] |
| Labels | need_feedback |
| Labels | need_feedback |
| Assignee | Jan Lindström [ jplindst ] | Julius Goryavsky [ sysprg ] |
| Assignee | Julius Goryavsky [ sysprg ] | Stepan Patryshev [ stepan.patryshev ] |
| Assignee | Stepan Patryshev [ stepan.patryshev ] | Teemu Ollakka [ teemu.ollakka ] |
| Link |
This issue relates to |
| Assignee | Teemu Ollakka [ teemu.ollakka ] | Jan Lindström [ jplindst ] |
| issue.field.resolutiondate | 2020-08-05 12:52:05.0 | 2020-08-05 12:52:05.712 |
| Fix Version/s | 10.4.14 [ 24305 ] | |
| Fix Version/s | 10.5.5 [ 24423 ] | |
| Fix Version/s | 10.4 [ 22408 ] | |
| Resolution | Fixed [ 1 ] | |
| Status | Open [ 1 ] | Closed [ 6 ] |
| Workflow | MariaDB v3 [ 110391 ] | MariaDB v4 [ 158013 ] |
| Zendesk Related Tickets | 103838 |