PR: https://github.com/MariaDB/server/pull/1556

Hi otto,

In my opinion, your PR could probably be improved by incorporating some of the items from this logrotate configuration in the KB:

https://mariadb.com/kb/en/rotating-logs-on-unix-and-linux/#configuring-logrotate

As you can see in the PR, the KB page is linked for more information. In my opinion those extra lines in the KB example are just bloat and do not bring any real value, just adds complexity.

Hi otto,

OK, good to know. I added those extra lines to the configuration in the KB page, because I found them valuable. It sounds like you disagree.

Personally, I find configuration items like minsize and maxsize extremely valuable. However, I worked in Support for years, and I have had a lot of experiences trying to sift through error logs that were way too big.

In Gentoo Linux we are currently shipping

# Copyright 1999-2019 Gentoo Authors

# Distributed under the terms of the GNU General Public License v2

/var/log/mysql/mysql.err /var/log/mysql/mysql.log /var/log/mysql/mysqld.err {

monthly

create 660 mysql mysql

notifempty

size 5M

sharedscripts

missingok

postrotate

        pidfile=$(my_print_defaults server mysqld mariadb | grep 'pid[_-]file' | tail -n 1 | cut -d = -f 2)

        if [ -s "${pidfile}" ] ; then

                /bin/kill -HUP $(cat "${pidfile}")

        fi

endscript

}

Given Debian logrotate got broken https://github.com/MariaDB/server/commit/7803601dcb8e40746a10a4012b0c8eb08274db3e#diff-49ee12f321170f0cc474278bf311255e (this shouldn't have been changed). Lets get this fixed.

GeoffMontee what do you regard as a manageable log size? Any advice on duration of logs?

whissi Thanks for sharing. I tend to think that monthly like you've done with size limits is better than the daily in #1556. Note whissi Gentoo should be using my_print_defaults --mysqld to server all server options. I've avoided SIGHUP based solutions in favour of minimizing the amount that the server does - MDEV-11610 (mainly due to issues at the time - MDEV-9510).

Hi danblack,

what do you regard as a manageable log size? Any advice on duration of logs?

My thoughts:

• maxsize: In my opinion, somewhere in the 500 MB - 2 GB range is a good maximum size. When you compress the log, those sizes are still small enough that the log can be transferred fairly easily. And when you read the log, it's probably not going to cause your text editor to lock up. When the log gets several GB or larger, then it can become a real pain to deal with. If the average log line is 200 bytes, then a 500 MB log can have around 2.6 million lines, while a 2 GB log can have around 10.7 million lines. Either way, this is still quite a lot of information.
• minsize: Somewhere in the 50 MB - 200 MB range is probably a good minimum size. If the average log line is 200 bytes, then a 50 MB log can have around 260 thousand lines, while a 200 MB log can have around 1 million lines.
• Duration: If maxsize is set, then monthly is probably fine.

danblack Thank you for the hint regarding "--mysqld" but we cannot use this option in Gentoo: In Gentoo we are packaging MySQL servers without client tools. my_print_defaults is either provided by mariadb-connector-c (which supports --mysqld) or mysql-connector-c package. The latter does not support this option. And for completeness, logrotate script is provided by mysql-init-scripts package.

whissi thanks, make sense.

Additional info from GeoffMontee that didn't contain anything private

hey! you're referring to the rotate N option? if both monthly and maxsize are set, then the number of logs rotated may not necessarily be the number of months that pass. for example, if some kind of distributed brute force attack occurs, and log_warnings=2 is set (which is default), then the logs could quickly fill up with "access denied" messages. or if an app doesn't close its connections properly, then it can fill up with "aborted connections" messages as those connections time out. in cases like these, the log could get bigger than maxsize, and it can be rotated earlier than a month. so the N should probably be considered "number of logs", rather than "number of months". i think the optimal number of logs could depend on a lot of factors. for example:

if your logs stay fairly clean, then rotate 10 may allow you to maintain 10 full months of logs, while using very little disk space.
if you use some crappy PHP app that doesn't properly close its connections, and if your server is open to the internet and is constantly getting probed by hackers on port 3306, and if your app constantly triggers warnings then get written to the log, then rotate 10 may give you less than a single month worth of logs

in short: i don't have an answer that will work for everyone, but i do think somewhere around "10" is probably a number that will work fairly well for most people

So thinking maxsize=500MB, rotate=10 for a 5G max size of logs.

This issue is already done and there is a PR waiting to be merged. I don't understand why you started re-engineering a new solution now, that seems like duplicate work to me.

https://github.com/MariaDB/server/pull/1556

Given Debian logrotate got broken https://github.com/MariaDB/server/commit/7803601dcb8e40746a10a4012b0c8eb08274db3e#diff-49ee12f321170f0cc474278bf311255e (this shouldn't have been changed). Lets get this fixed.

What exactly got broken? The commit did not update the logrotate file functionally (just a comment in it).

The PR https://github.com/MariaDB/server/pull/1556 switches from mysql-admin to mariadb-admin.

Ok, I see it was this change:

-          if [ -f `my_print_defaults --mysqld | grep -oP "pid-file=\K[^$]+"` ]; then

+          if [ -f `my_print_defaults --mariadbd | grep -oP "pid-file=\K[^$]+"` ]; then

The whole line should be removed as I suggested in https://github.com/MariaDB/server/pull/1556

I wrote the comments on this issue because #1556 is there. I do plan on using it, just raising the very low values and there was a serg suggested simplification of [ -x mariadb-admin ] in one of the reviews somewhere that I'll find again.

Status: PR still pending at https://github.com/MariaDB/server/pull/1556

mschorm/Faramos Promised by email recently to take a look at some point so we get the RPM side validated/completed.

Status: PR still pending at https://github.com/MariaDB/server/pull/1556

Also still pending feedback from Michal Schorm.

This PR would close half a dozen bugs if merged and it is a pity there is not progress on the RPM side to test/validate this.

https://github.com/ottok/mariadb/pull/6