Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-31404

Implement binlog_space_limit

Details

    Description

      Original request:
      We have a 130GB database, replicated to a backup cluster, and the binlog files consume 400GB of storage for a single day, without any option to reduce that to something more reasonable!

      Implementation guidelines:

      • Add variable to limit max binlog space (max-binlog-total-space and/or binlog_space_limit)
      • Binlog size should be checked at during server start, binlog rotation, FLUSH LOGS, when writing to binary log or when max-binlog-total-size changes value.
      • Add option --slave-connections-needed-for-purge with 1 as default. This is the minimum number of slaves that needs to be connected for binary logs to be deleted.
        The number should normally be the number of minimum expected slaves. The binary logs will not be deleted until we have at least that many slaves attached and no one is using the to-be-deleted binary log.
        For example assuming one is supposed to have 3 slaves connected to the sever. MariaDB will not delete binary logs based on size until all of them are connected at the same time and all of the slaves are requesting data from binary logs after the to-be-deleted one.
      • Add status variable 'Binlog_disk_use' that shows current binary log space usage.

      Attachments

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. MDEV-33116 int MYSQL_BIN_LOG::real_purge_logs_by_size(ulonglong): Assertion `reclaimed_space == found_space' failed

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-33282 Assertion `(longlong) binlog_space_total >= 0' failed in MYSQL_BIN_LOG::real_purge_logs_by_size

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-33319 max_binlog_total_size and slave_connections_needed_for_purge require wrong permissions

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-33320 Assertion `(longlong) binlog_space_total >= 0' failed in MYSQL_BIN_LOG::real_purge_logs_by_size #2

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-34961 BINLOG_DISK_USE status variable is not documented

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is duplicated by

          New Feature - A new feature of the product, which has yet to be developed. MDEV-33029 Add a mechanism to limit total binlog size on disk

          • Major - Major loss of function.
          • Closed
          relates to

          Task - A task that needs to be done. MDEV-11076 Make binlog and relay sizes dynamically configurable by "maximum size"

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-29195 relay_log_purge variable not very useful

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Task - A task that needs to be done. MDEV-29196 Request new system variable: binlog_keep_max

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-34897 Assertion `info->type == READ_CACHE || info->type == WRITE_CACHE' failed in reinit_io_cache on SET ... slave_connections_needed_for_purge

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-35336 PURGE BINARY LOGS not working anymore again

          • Major - Major loss of function.
          • Closed

          Activity

            Transition Time In Source Status Execution Times
            Michael Widenius made transition -
            Open In Progress
            		181d 1h 58m 1
            Michael Widenius made transition -
            In Progress In Testing
            		2d 1h 24m 1
            Elena Stepanova made transition -
            In Testing Stalled
            		69d 25m 1
            Sergei Golubchik made transition -
            Stalled Closed
            		6d 7h 18m 1

            People

              monty Michael Widenius
              ccounotte COUNOTTE CEDRIC
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.