Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-34595

Expiring and purging binlogs not working

Details

    Description

      After upgrading to 11.4 we noticed that binlogs don't expire any more. I tried with a fresh setup on a test system, same result.

      The server was configured with the following settings:

      # mariadbd --print-defaults
      		 
      mariadbd would have been started with the following arguments:
      		 
      --socket=/run/mysqld/mysqld.sock --pid-file=/run/mysqld/mysqld.pid --basedir=/usr --bind-address=127.0.0.1 --server-id=1 --log_bin=/var/log/mysql/mysql-bin.log --max_binlog_size=1M --binlog_expire_logs_seconds=60 --character-set-server=utf8mb4 --character-set-collations=utf8mb4=uca1400_ai_ci

      To create some content for the binary logs I used sysbench

      mariadb -e "CREATE DATABASE test;"
      		 
      sysbench oltp_common --db-driver=mysql --table-size=100000 --mysql-user=root --mysql-db=test prepare
      		 
      sysbench oltp_read_write --db-driver=mysql --table-size=100000 --mysql-user=root --mysql-db=test --threads=1 --report-interval=1 run

      • The log shows nothing except the normal startup lines
      • With version 11.3, binlog files are removed automatically after ~1 minute. With 11.4 they stay
      • Manually purging binary logs via "PURGE BINARY LOGS TO 'mysql-bin.000010';" works as expected in 11.3 but has no effect in 11.4 (also no error message)

      Any suggestions to work around this bug are also welcome as the logs are filling the disk on the affected systems.

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MDEV-34504 PURGE BINARY LOGS not working anymore

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            bnestere Brandon Nesterenko added a comment - - edited

            Hi Delbrueck,

            This appears to be MDEV-34504, which a fix has been pushed and will be included in the next release (currently scheduled for July 25). You can read through the comments on the other JIRA for details, but a few takeaways from the discussion:

            There is a workaround you can use now:

            The fix is to set --slave-connections-needed-for-purge=0.

            A summary of the fix:

            MDEV-34504 PURGE BINARY LOGS not working anymore

            PURGE BINARY LOGS did not always purge binary logs. This commit fixes
            some of the issues and adds notifications if a binary log cannot be
            purged.

            User visible changes:

            'PURGE BINARY LOG TO log_name' and 'PURGE BINARY LOGS BEFORE date'
            worked differently. 'TO' ignored 'slave_connections_needed_for_purge'
            while 'BEFORE' did not. Now both versions ignores the
            'slave_connections_needed_for_purge variable'.
            'PURGE BINARY LOG..' commands now returns 'note' if a binary log cannot
            be deleted like
            Note 1375 Binary log 'master-bin.000004' is not purged because it is
            the current active binlog
            Automatic binary log purges, based on date or size, will write a
            note to the error log if a binary log matching the size or date
            cannot yet be deleted.
            If 'slave_connections_needed_for_purge' is set from a config or
            command line, it is set to 0 if Galera is enabled and 1 otherwise
            (old default). This ensures that automatic binary log purge works
            with Galera as before the addition of
            'slave_connections_needed_for_purge'.
            If the variable is changed to 0, a warning will be printed to the error
            log.

            knielsen, monty FYI

            bnestere Brandon Nesterenko added a comment - - edited Hi Delbrueck , This appears to be MDEV-34504 , which a fix has been pushed and will be included in the next release (currently scheduled for July 25). You can read through the comments on the other JIRA for details, but a few takeaways from the discussion: There is a workaround you can use now: The fix is to set --slave-connections-needed-for-purge=0. A summary of the fix: MDEV-34504 PURGE BINARY LOGS not working anymore PURGE BINARY LOGS did not always purge binary logs. This commit fixes some of the issues and adds notifications if a binary log cannot be purged. User visible changes: 'PURGE BINARY LOG TO log_name' and 'PURGE BINARY LOGS BEFORE date' worked differently. 'TO' ignored 'slave_connections_needed_for_purge' while 'BEFORE' did not. Now both versions ignores the 'slave_connections_needed_for_purge variable'. 'PURGE BINARY LOG..' commands now returns 'note' if a binary log cannot be deleted like Note 1375 Binary log 'master-bin.000004' is not purged because it is the current active binlog Automatic binary log purges, based on date or size, will write a note to the error log if a binary log matching the size or date cannot yet be deleted. If 'slave_connections_needed_for_purge' is set from a config or command line, it is set to 0 if Galera is enabled and 1 otherwise (old default). This ensures that automatic binary log purge works with Galera as before the addition of 'slave_connections_needed_for_purge'. If the variable is changed to 0, a warning will be printed to the error log. knielsen , monty FYI
            Delbrueck Christoph added a comment -

            Hi Brandon,

            thank you, I can confirm that it works with --slave-connections-needed-for-purge=0

            Delbrueck Christoph added a comment - Hi Brandon, thank you, I can confirm that it works with --slave-connections-needed-for-purge=0
            bnestere Brandon Nesterenko added a comment -

            Great! I will close this ticket as a duplicate then.

            bnestere Brandon Nesterenko added a comment - Great! I will close this ticket as a duplicate then.
            bnestere Brandon Nesterenko added a comment -

            Closed as a duplicate of MDEV-34504.

            bnestere Brandon Nesterenko added a comment - Closed as a duplicate of MDEV-34504 .

            People

              Unassigned Unassigned
              Delbrueck Christoph
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.