CentOS 6.8 VM 16GB memory.

Please check you system logs for messages from OOM killer. If mysqld process disappears without saying anything at all in the log, it's almost always caused by OOM.

Elena,

Thanks for the quick response. That was the issue. I'm investigating what settings to change to reduce the memory usage. I guess this JIRA can be closed based on that

Thanks for the update. I'm closing it for now, but please comment to re-open if during your investigation you find out that MariaDB might have a memory leak.

Hi,

I'm attaching the output of a tool that does the following every minute:

OUT_FILE=/root/oom.out

/bin/date >> ${OUT_FILE}

/usr/bin/free >> ${OUT_FILE}

ps aux | grep mysqld | grep -v grep >> ${OUT_FILE}

mysql -uroot -p<password> -e "SHOW FULL PROCESSLIST" >> ${OUT_FILE}

mysql -uroot -p<password> -e "SHOW ENGINE INNODB STATUS\G" >> ${OUT_FILE}

The file contains the data in between 2 crashes due to OOM

Reducing the innodb_buffer_pool_size from 12G to 10G helped a little bit but I'm still running out of memory
I see that the process starts using memory to the point that "ps aux" reports a memory footprint of the process larger than the main memory size (16GB for our server)
We have about 221354 tables in the server, most of them being used by 3 warehouse databases (and keeps on growing)
I have modified the /etc/security/limits.conf file to add these lines:

# For MariaDB

*               soft    nofile          1024000

*               hard    nofile          1024000

(I figure that our problem might be related to the number of tables, that's why I mention all those changes)

This server is the master to a rep server that has 32GB in memory (I know that replication runs into issues, but was waiting for the master to stabilize. Don't know if the rep process of the slave might affect the server).

Thanks for the help!

Another clue:

• I have managed to reduce the frequency of OOM crashes by running every 30 mins "FLUSH TABLES"

In this case, maybe you could try reducing table_open_cache which is currently set to a quite big value of 150000.

I have reduced the table_open_cache to 50000 and have been monitoring the system since yesterday.

Is there a way to calculate ahead of time how much memory will MariaDB use as a maximum? I mean, it's kind of hard to know what value to use for some parameters, including table_open_cache, just by trial and error, especially in a production environment. table_open_cache could take days to crawl up until the system runs out of memory. I guess I'm concerned that I report to my management that the issue has been solved and find out later on that I get an OOM because another parameter pushed MariaDB to use more memory.

By the way, we appreciate the time you've spent helping us!

In MDEV-9764 we've now implemented a way to limit MariaDB memory usage per connection. Together with other limits (for global buffers) and a max number of connections limit, this allows you to limit all memory used by MariaDB Server.