Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-11610

Logrotate to only FLUSH LOCAL ERROR LOGS, ENGINE LOGS, GENERAL LOGS

Details

    Description

      To reduce some of the side effects of log rotation of binary logs, including stalling on large transactions and other errors (MDEV-11550, MDEV-10809, MDEV-10276, ...), perhaps logrotate can be limited to the slow query log, error log, and engine log.

      Avoiding writing to the binlog is also preferable as the slaves will have their on logrotate.

      Requires including these options to mysqladmin and changing ./debian/mariadb-server-10.2.mysql-server.logrotate

      Attachments

        Issue Links

          Activity

            danblack Daniel Black added a comment -

            Any opinion on this otto?

            danblack Daniel Black added a comment - Any opinion on this otto ?
            otto Otto Kekäläinen added a comment - Can you rephrase the question as a diff? To me https://anonscm.debian.org/cgit/pkg-mysql/mariadb-10.0.git/tree/debian/mariadb-server-10.0.mysql-server.logrotate and https://github.com/MariaDB/server/blob/10.2/debian/mariadb-server-10.2.mysql-server.logrotate looks good.
            danblack Daniel Black added a comment -

            Once implemented in mysqladmin would look something like:

            diff --git a/debian/mariadb-server-10.2.mysql-server.logrotate b/debian/mariadb-server-10.2.mysql-server.logrotate
            index 2a9c3cd..c97217b 100644
            --- a/debian/mariadb-server-10.2.mysql-server.logrotate
            +++ b/debian/mariadb-server-10.2.mysql-server.logrotate
            @@ -13,7 +13,7 @@
                       test -x /usr/bin/mysqladmin || exit 0
                       if [ -f `my_print_defaults --mysqld | grep -oP "pid-file=\K[^$]+"` ]; then
                         # If this fails, check debian.conf!
            -            mysqladmin --defaults-file=/etc/mysql/debian.cnf flush-logs
            +            mysqladmin --defaults-file=/etc/mysql/debian.cnf --local flush-logs=error,engine,general
                       fi
                    endscript
             }
            

            Its intentionally not instructing mariadb to rotate the binary logs at the log rotate time. This because of reliability issues (that I'm sure will be worked though) but also this has an effect on large transactions that may be occurring at log rotation time. Binary log are controlled by max_binlog_size and expire_log_days so the lack of a daily/weekly logrotate seems like a good tradeoff.

            danblack Daniel Black added a comment - Once implemented in mysqladmin would look something like: diff --git a/debian/mariadb-server-10.2.mysql-server.logrotate b/debian/mariadb-server-10.2.mysql-server.logrotate index 2a9c3cd..c97217b 100644 --- a/debian/mariadb-server-10.2.mysql-server.logrotate +++ b/debian/mariadb-server-10.2.mysql-server.logrotate @@ -13,7 +13,7 @@ test -x /usr/bin/mysqladmin || exit 0 if [ -f `my_print_defaults --mysqld | grep -oP "pid-file=\K[^$]+"` ]; then # If this fails, check debian.conf! - mysqladmin --defaults-file=/etc/mysql/debian.cnf flush-logs + mysqladmin --defaults-file=/etc/mysql/debian.cnf --local flush-logs=error,engine,general fi endscript } Its intentionally not instructing mariadb to rotate the binary logs at the log rotate time. This because of reliability issues (that I'm sure will be worked though) but also this has an effect on large transactions that may be occurring at log rotation time. Binary log are controlled by max_binlog_size and expire_log_days so the lack of a daily/weekly logrotate seems like a good tradeoff.
            danblack Daniel Black added a comment -

            To be clear, the server side of this implementation by support of various individual FLUSH options (https://mariadb.com/kb/en/mariadb/flush/) is already complete. This just requires the client mysqladmin to implement mapping command line options (as suggested above) to the SQL.

            danblack Daniel Black added a comment - To be clear, the server side of this implementation by support of various individual FLUSH options ( https://mariadb.com/kb/en/mariadb/flush/ ) is already complete. This just requires the client mysqladmin to implement mapping command line options (as suggested above) to the SQL.
            danblack Daniel Black added a comment -

            serg, I'm hoping this isn't too late for 10.0.30 and in light of the large incomplete list of crash bugs references here who suffer segfaults as a result of flush-logs this was an easy fix. All it does is stop flushing binary logs during logrotate.

            Only mysqladmin and associated scripts are changed.

            No server core changed in the attached PR.

            danblack Daniel Black added a comment - serg , I'm hoping this isn't too late for 10.0.30 and in light of the large incomplete list of crash bugs references here who suffer segfaults as a result of flush-logs this was an easy fix. All it does is stop flushing binary logs during logrotate. Only mysqladmin and associated scripts are changed. No server core changed in the attached PR.

            danblack, sure, I think it's a good change even unrelated to other bugs.

            But I don't understand this --local flag. Where did you find mysqladmin that supports it?

            serg Sergei Golubchik added a comment - danblack , sure, I think it's a good change even unrelated to other bugs. But I don't understand this --local flag. Where did you find mysqladmin that supports it?
            danblack Daniel Black added a comment - It didn't support it. I added it. https://github.com/MariaDB/server/pull/323/commits/be82664a331808637755448948c8be7ba14aa258

            People

              svoj Sergey Vojtovich
              danblack Daniel Black
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.