[MXS-1934] MaxScale encountered internal out of memory when server have ~ 30G free memory Created: 2018-06-20  Updated: 2019-09-05  Resolved: 2019-09-05

Status: Closed
Project: MariaDB MaxScale
Component/s: N/A
Affects Version/s: 2.2.6
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Daniel Snow Assignee: markus makela
Resolution: Cannot Reproduce Votes: 1
Labels: None
Environment:

Debian GNU/Linux 8
kernel 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u2 (2017-06-26) x86_64 GNU/Linux

~# numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 8 9 10 11
node 0 size: 31845 MB
node 0 free: 16528 MB
node 1 cpus: 4 5 6 7 12 13 14 15
node 1 size: 32318 MB
node 1 free: 11758 MB
node distances:
node 0 1
0: 10 21
1: 21 10

CPU: Intel(R) Xeon(R) CPU E5-2623 v4 @ 2.60GHz


 Description   

Hi!

Our installation encountered a strange issue after weeks of a stable operating. MaxScale 2.2.6 just stopped accepting connections with the following error:

maxscale[26935]: [MySQLAuth] Failed to execute auth query: out of memory

However, during the fail server had a quite low workload and about 30G of free RAM.

Seems buggy, or I missed something during the configuration? If anyone need any extra traces or system info, I'll be happy to provide them.

note bene - extended logging was enbled after the failure

Here is my config:

[maxscale]
 
threads=4
 
syslog=0
 
maxlog=1
 
log_to_shm=1
 
log_warning=1
 
log_notice=1
 
 
 
[server0]
 
type=server
 
address=192.168.34.34
 
port=3306
 
protocol=MySQLBackend
 
serversize=0
 
priority=6
 
 
 
[server1]
 
type=server
 
address=192.168.34.35
 
port=3306
 
protocol=MySQLBackend
 
serversize=100
 
priority=1
 
 
 
[server2]
 
type=server
 
address=192.168.34.36
 
port=3306
 
protocol=MySQLBackend
 
serversize=100
 
priority=2
 
 
 
[server3]
 
type=server
 
address=192.168.34.37
 
port=3306
 
protocol=MySQLBackend
 
serversize=100
 
priority=5
 
 
 
[server4]
 
type=server
 
address=192.168.34.38
 
port=3306
 
protocol=MySQLBackend
 
serversize=100
 
priority=3
 
 
 
[MySQL Monitor]
 
type=monitor
 
module=galeramon
 
#module=mmmon
 
servers=server0, server1, server2, server3, server4
 
user=maxscale
 
passwd=maxscale
 
monitor_interval=5000
 
use_priority=true
 
 
 
[Read-Write-Master Service]
 
type=service
 
router=readconnroute
 
servers=server0, server1, server2, server3, server4
 
user=maxscale
 
passwd=maxscale
 
router_options=master
 
 
 
[Read-Write Service]
 
type=service
 
router=readwritesplit
 
servers=server0, server1, server2, server3, server4
 
user=maxscale
 
passwd=maxscale
 
max_slave_connections=100%
 
router_options=slave_selection_criteria=LEAST_CURRENT_OPERATIONS,master_accept_reads=true
 
 
 
[Read-Only Service]
 
type=service
 
router=readconnroute
 
servers=server0, server1, server2, server3, server4
 
user=maxscale
 
passwd=maxscale
 
router_options=slave
 
weightby=serversize
 
 
 
[MaxAdmin Service]
 
type=service
 
router=cli
 
 
 
[Read-Write-Master Listener]
 
type=listener
 
service=Read-Write-Master Service
 
protocol=MySQLClient
 
port=4007
 
 
 
[Read-Write Listener]
 
type=listener
 
service=Read-Write Service
 
protocol=MySQLClient
 
port=4006
 
 
 
[Read-Only Listener]
 
type=listener
 
service=Read-Only Service
 
protocol=MySQLClient
 
port=4005
 
 
 
[MaxAdmin Listener]
 
type=listener
 
service=MaxAdmin Service
 
protocol=maxscaled
 
socket=default



 Comments   
Comment by markus makela [ 2018-06-21 ]

The error appears to be from the SQLite instance used to store the set of database users.

Comment by markus makela [ 2019-04-29 ]

Have you tried a more recent 2.2 version of MaxScale? Have you tried this with the 2.3 version?

Generated at Thu Feb 08 04:10:29 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.