[MDEV-31404] Implement binlog_space_limit Created: 2023-06-05  Updated: 2024-01-31

Status: In Testing
Project: MariaDB Server
Component/s: None
Fix Version/s: 11.4

Type: New Feature Priority: Critical
Reporter: COUNOTTE CEDRIC Assignee: Elena Stepanova
Resolution: Unresolved Votes: 0
Labels: Preview_11.4

Issue Links:
Duplicate
is duplicated by MDEV-33029 Add a mechanism to limit total binlog... Closed
PartOf
Problem/Incident
causes MDEV-33116 int MYSQL_BIN_LOG::real_purge_logs_by... Closed
causes MDEV-33282 Assertion `(longlong) binlog_space_to... Closed
causes MDEV-33319 max_binlog_total_size and slave_conne... Open
causes MDEV-33320 Assertion `(longlong) binlog_space_to... Closed
Relates
relates to MDEV-29195 relay_log_purge variable not very useful Open
relates to MDEV-29196 Request new system variable: binlog_k... Open

 Description   

Original request:
We have a 130GB database, replicated to a backup cluster, and the binlog files consume 400GB of storage for a single day, without any option to reduce that to something more reasonable!

Implementation guidelines:

  • Add variable to limit max binlog space (max-binlog-total-space and/or binlog_space_limit)
  • Binlog size should be checked at during server start, binlog rotation, FLUSH LOGS, when writing to binary log or when max-binlog-total-size changes value.
  • Add option --slave-connections-needed-for-purge with 1 as default. This is the minimum number of slaves
    that needs to be connected for binary logs to be deleted.
  • Add status variable 'Binlog_disk_use' that shows current binary log space usage.


 Comments   
Comment by Daniel Black [ 2023-06-06 ]

binlog_expire_logs_seconds has been there since 10.6.1 as the expire_logs_days did lack granularity less than 1.

Are you using binlog_row_image= NOBLOB or MINIMAL to reduce size?

Also see:

Comment by COUNOTTE CEDRIC [ 2023-06-06 ]

Thanks for your quick reply. I was not aware of the new binlog_expire_logs_seconds when we upgraded to 10.6.x a while back!

I suppose we should first start using MINIMAL, will it cause any issues on the replication slave when changing it dynamically?

EDIT: I did a DROP TABLE xxx on primary cluster, for a table being child of foreign keys and obviously it failed, but for some reason the statement is being replicated and makes the slave constantly exit !? Could it be a side-effect of minimal and how to workaround this?

Comment by Daniel Black [ 2023-06-06 ]

> I suppose we should first start using MINIMAL, will it cause any issues on the replication slave when changing it dynamically?

Not that I'm aware of. Please do a search here around binlog_row_image=MINIMAL.

Comment by Daniel Black [ 2023-06-06 ]

> I did a DROP TABLE xxx on primary cluster, for a table being child of foreign keys and obviously it failed, but for some reason the statement is being replicated and makes the slave constantly exit !? Could it be a side-effect of minimal and how to workaround this?

Unsure. Please do new bug report, with MariaDB versions, and examples of your table structures.

Comment by Michael Widenius [ 2023-12-05 ]

MDEV-31404 Implement binlog_space_limit

binlog_space_limit is a variable in Percona server used to limit the total
size of all binary logs.

This implementation is based on code from Percona server 5.7.

In MariaDB we decided to call the variable max-binlog-total-size to be
similar to max-binlog-size. This makes it easier to find in the output
from 'mariadbd --help --verbose'). MariaDB will also support
binlog_space_limit for compatibility with Percona.

Some internal notes to explain implementation notes:

  • When running MariaDB does not delete binary logs that are either
    used by slaves or have active xid that are not yet committed.

Some implementation notes:

  • max-binlog-total-size is by default 0 (no limit).
  • max-binlog-total-size can be changed without server restart.
  • Binlog file file sizes are checked on startup, or if
    max-binlog-total-size is set to a value > 0, not for every log write.
    The total size of all binary logs is cached and dynamically updated
    when updating the binary log on on binary log rotation.
  • max-binlog-total-size is checked against existing log files during server
    start, binlog rotation, FLUSH LOGS, when writing to binary log
    or when max-binlog-total-size changes value.
  • Option --slave-connections-needed-for-purge with 1 as default added.
    This allows one to ensure that we do not delete binary logs if there
    is less than 'slave-connections-needed-for-purge' connected.
    Without this option max-binlog-total-size would potentially deleted
    binlogs needed by slaves on server startup or when a slave disconnects
    as there are then no connected slaves to protect active binlogs.
  • If binary log is on and max-binlog-total_size <> 0 then the status
    variable 'Binlog_disk_use' shows the current size of all old binary
    logs + the state of the current one.
  • Removed test of strcmp(log_file_name, log_info.log_file_name) in
    purge_logs_before_date() as this is tested in can_purge_logs()
  • To avoid expensive calls of log_in_use() we cache the result for the
    last log that is in use by a slave. Future calls to can_purge_logs()
    for this binary log will return false until a slave stats working on
    a new log.
Comment by Elena Stepanova [ 2023-12-06 ]

ccounotte,
Could you please clarify your side note about DROP TABLE with foreign keys:

I did a DROP TABLE xxx on primary cluster, for a table being child of foreign keys and obviously it failed, but for some reason the statement is being replicated and makes the slave constantly exit !?

In this context, what is the primary cluster and what is the slave? Do you mean that the statement was replicated within a Galera cluster, between nodes? Or is it about traditional replication? Do you have an example of a binary log?

Comment by Kristian Nielsen [ 2023-12-11 ]

Review done: https://lists.mariadb.org/hyperkitty/list/developers@lists.mariadb.org/thread/EXIBNZ6PY4FQNBIUWJXS7SHKBMJWFPQU/

Generated at Thu Feb 08 10:23:37 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.