Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
10.5
-
None
Description
Currently we have in 10.5 master two logrotate files:
$ cat support-files/mysql-log-rotate.sh
|
# This logname can be set in /etc/my.cnf
|
# by setting the variable "log-error"
|
# in the [mysqld] section as follows:
|
#
|
# [mysqld]
|
# log-error=@localstatedir@/mysqld.log
|
#
|
# If the root user has a password you have to create a
|
# /root/.my.cnf configuration file with the following
|
# content:
|
#
|
# [mysqladmin]
|
# password = <secret>
|
# user= root
|
#
|
# where "<secret>" is the password.
|
#
|
# ATTENTION: This /root/.my.cnf should be readable ONLY
|
# for root !
|
|
|
@localstatedir@/mysqld.log {
|
# create 600 mysql mysql
|
notifempty
|
daily
|
rotate 3
|
missingok
|
compress
|
postrotate
|
# just if mysqld is really running
|
if test -x @bindir@/mysqladmin && \
|
@bindir@/mysqladmin ping &>/dev/null
|
then
|
@bindir@/mysqladmin --local flush-error-log \
|
flush-engine-log flush-general-log flush-slow-log
|
fi
|
endscript
|
}
|
|
|
$ cat debian/mariadb-server-10.5.mysql-server.logrotate
|
# - I put everything in one block and added sharedscripts, so that mysql gets
|
# flush-logs'd only once.
|
# Else the binary logs would automatically increase by n times every day.
|
# - The error log is obsolete, messages go to syslog now.
|
/var/log/mysql/mysql.log /var/log/mysql/mysql-slow.log /var/log/mysql/mariadb-slow.log /var/log/mysql/error.log {
|
daily
|
rotate 7
|
missingok
|
create 640 mysql adm
|
compress
|
sharedscripts
|
postrotate
|
test -x /usr/bin/mysqladmin || exit 0
|
if [ -f `my_print_defaults --mysqld | grep -oP "pid-file=\K[^$]+"` ]; then
|
# If this fails, check debian.conf!
|
mysqladmin --defaults-file=/etc/mysql/debian.cnf --local flush-error-log \
|
flush-engine-log flush-general-log flush-slow-log
|
fi
|
endscript
|
}
|
I suggest we in future would have only one single mariadb.logrotate(.sh) file that would be used in all distros, both deb and rpm based. There is after all only one logrotate utility and the mariadbd responds to system signals the same way across all distros, and the mariadb-admin etc tools also work close enough on all distros.
I suggest we use the name 'mariadb' from now on, since the binary is 'mariadbd' from 10.5 onwards. Also I suggest we always use 'logrotate' and not 'log-rotate' in filenames, since the name of the utility is 'logrotate'.
There are currently open 2 PRs on fixing this script:
- https://github.com/MariaDB/server/pull/807 (rpm version fixes)
- https://github.com/MariaDB/server/pull/934 (deb version fixes)
There are also multiple Jira issues:
- https://jira.mariadb.org/browse/MDEV-11884 (Status 'Closed', but wasn't actually fixed by Ondrej in upstream) - deb version, upstreaming of this in PR#934
- https://jira.mariadb.org/browse/MDEV-13050 - deb version, also fixed in PR#934)
- https://jira.mariadb.org/browse/MDEV-16621 - delaycompress and sighup suggestions
- https://jira.mariadb.org/browse/MDEV-16772 - deb version, duplicate pid result fixed in PR#934
- https://jira.mariadb.org/browse/MDEV-19436 - deb version, duplicate of duplicate pid issue
- https://jira.mariadb.org/browse/MDEV-20863 - deb version, duplicate of duplicate pid issue
Also to be considered, that in systemd the logs go to journald and /var/log/mysql stays empty.
Attachments
Issue Links
- includes
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
- is duplicated by
-
-
- Closed
-
Activity
Hi otto,
In my opinion, your PR could probably be improved by incorporating some of the items from this logrotate configuration in the KB:
https://mariadb.com/kb/en/rotating-logs-on-unix-and-linux/#configuring-logrotate
As you can see in the PR, the KB page is linked for more information. In my opinion those extra lines in the KB example are just bloat and do not bring any real value, just adds complexity.
Hi otto,
OK, good to know. I added those extra lines to the configuration in the KB page, because I found them valuable. It sounds like you disagree.
Personally, I find configuration items like minsize and maxsize extremely valuable. However, I worked in Support for years, and I have had a lot of experiences trying to sift through error logs that were way too big.
In Gentoo Linux we are currently shipping
# Copyright 1999-2019 Gentoo Authors
|
# Distributed under the terms of the GNU General Public License v2
|
|
|
/var/log/mysql/mysql.err /var/log/mysql/mysql.log /var/log/mysql/mysqld.err {
|
monthly
|
create 660 mysql mysql
|
notifempty
|
size 5M
|
sharedscripts
|
missingok
|
postrotate
|
pidfile=$(my_print_defaults server mysqld mariadb | grep 'pid[_-]file' | tail -n 1 | cut -d = -f 2)
|
if [ -s "${pidfile}" ] ; then
|
/bin/kill -HUP $(cat "${pidfile}")
|
fi
|
endscript
|
}
|
Given Debian logrotate got broken https://github.com/MariaDB/server/commit/7803601dcb8e40746a10a4012b0c8eb08274db3e#diff-49ee12f321170f0cc474278bf311255e (this shouldn't have been changed). Lets get this fixed.
GeoffMontee what do you regard as a manageable log size? Any advice on duration of logs?
whissi Thanks for sharing. I tend to think that monthly like you've done with size limits is better than the daily in #1556. Note whissi Gentoo should be using my_print_defaults --mysqld to server all server options. I've avoided SIGHUP based solutions in favour of minimizing the amount that the server does - MDEV-11610 (mainly due to issues at the time - MDEV-9510).
Hi danblack,
what do you regard as a manageable log size? Any advice on duration of logs?
My thoughts:
- maxsize: In my opinion, somewhere in the 500 MB - 2 GB range is a good maximum size. When you compress the log, those sizes are still small enough that the log can be transferred fairly easily. And when you read the log, it's probably not going to cause your text editor to lock up. When the log gets several GB or larger, then it can become a real pain to deal with. If the average log line is 200 bytes, then a 500 MB log can have around 2.6 million lines, while a 2 GB log can have around 10.7 million lines. Either way, this is still quite a lot of information.
- minsize: Somewhere in the 50 MB - 200 MB range is probably a good minimum size. If the average log line is 200 bytes, then a 50 MB log can have around 260 thousand lines, while a 200 MB log can have around 1 million lines.
- Duration: If maxsize is set, then monthly is probably fine.
danblack Thank you for the hint regarding "--mysqld" but we cannot use this option in Gentoo: In Gentoo we are packaging MySQL servers without client tools. my_print_defaults is either provided by mariadb-connector-c (which supports --mysqld) or mysql-connector-c package. The latter does not support this option. And for completeness, logrotate script is provided by mysql-init-scripts package.
whissi thanks, make sense.
Additional info from GeoffMontee that didn't contain anything private
hey! you're referring to the rotate N option? if both monthly and maxsize are set, then the number of logs rotated may not necessarily be the number of months that pass. for example, if some kind of distributed brute force attack occurs, and log_warnings=2 is set (which is default), then the logs could quickly fill up with "access denied" messages. or if an app doesn't close its connections properly, then it can fill up with "aborted connections" messages as those connections time out. in cases like these, the log could get bigger than maxsize, and it can be rotated earlier than a month. so the N should probably be considered "number of logs", rather than "number of months". i think the optimal number of logs could depend on a lot of factors. for example:
if your logs stay fairly clean, then rotate 10 may allow you to maintain 10 full months of logs, while using very little disk space.
if you use some crappy PHP app that doesn't properly close its connections, and if your server is open to the internet and is constantly getting probed by hackers on port 3306, and if your app constantly triggers warnings then get written to the log, then rotate 10 may give you less than a single month worth of logs
in short: i don't have an answer that will work for everyone, but i do think somewhere around "10" is probably a number that will work fairly well for most people
So thinking maxsize=500MB, rotate=10 for a 5G max size of logs.
This issue is already done and there is a PR waiting to be merged. I don't understand why you started re-engineering a new solution now, that seems like duplicate work to me.
Given Debian logrotate got broken https://github.com/MariaDB/server/commit/7803601dcb8e40746a10a4012b0c8eb08274db3e#diff-49ee12f321170f0cc474278bf311255e (this shouldn't have been changed). Lets get this fixed.
What exactly got broken? The commit did not update the logrotate file functionally (just a comment in it).
The PR https://github.com/MariaDB/server/pull/1556 switches from mysql-admin to mariadb-admin.
Ok, I see it was this change:
- if [ -f `my_print_defaults --mysqld | grep -oP "pid-file=\K[^$]+"` ]; then
|
+ if [ -f `my_print_defaults --mariadbd | grep -oP "pid-file=\K[^$]+"` ]; then
|
The whole line should be removed as I suggested in https://github.com/MariaDB/server/pull/1556
I wrote the comments on this issue because #1556 is there. I do plan on using it, just raising the very low values and there was a serg suggested simplification of [ -x mariadb-admin ] in one of the reviews somewhere that I'll find again.
Status: PR still pending at https://github.com/MariaDB/server/pull/1556
mschorm/Faramos Promised by email recently to take a look at some point so we get the RPM side validated/completed.
Status: PR still pending at https://github.com/MariaDB/server/pull/1556
Also still pending feedback from Michal Schorm.
This PR would close half a dozen bugs if merged and it is a pity there is not progress on the RPM side to test/validate this.
PR: https://github.com/MariaDB/server/pull/1556