[MDEV-19371] Implement binlog_expire_logs_seconds for purging of binary logs Created: 2019-04-30  Updated: 2021-12-21  Resolved: 2021-05-10

Status: Closed
Project: MariaDB Server
Component/s: Replication
Fix Version/s: 10.6.1

Type: Task Priority: Critical
Reporter: Muhammad Irfan Assignee: Sujatha Sivakumar (Inactive)
Resolution: Fixed Votes: 3
Labels: replication, upstream-fixed

Issue Links:
Problem/Incident
causes MDEV-27039 Trying to lock mutex ... when the mut... Closed
Sub-Tasks:
Key
Summary
Type
Status
Assignee
MDEV-25648 Document binlog_expire_logs_seconds f... Technical task Closed Ian Gilfillan  

 Description   

Oracle MySQL implemented binary log expiration period in seconds via binlog_expire_logs_seconds variable and expire_logs_days is deprecated in MySQL 8.
It would be nice to have similar option for MariaDB Server. Expiring binary logs within seconds is extremely useful in high volume writes on master and where disk space is limited option.



 Comments   
Comment by Andrei Elkin [ 2019-07-09 ]

ralf.gebhardt@mariadb.com This one looks pretty easy cherry-picking of WL#9237: Add a new variable binlog_expire_logs_seconds

commit a7e1ef858ee82493dd8ad9a76bc9c22fe3b8c05b
Author: Neha Kumari <neha.n.kumari@oracle.com>
Date:   Thu Mar 2 12:30:08 2017 +0530

Comment by Sujatha Sivakumar (Inactive) [ 2021-02-03 ]

Following commits need to be backported.

1)

commit a7e1ef858ee82493dd8ad9a76bc9c22fe3b8c05b
Author: Neha Kumari <neha.n.kumari@oracle.com>
Date:   Thu Mar 2 12:30:08 2017 +0530
 
    WL#9237: Add a new variable binlog_expire_logs_seconds

2)

commit 17bd6e0aee8963275f2b5153c90aad5e3a94b74f
Author: Sven Sandberg <sven.sandberg@oracle.com>
Date:   Tue Apr 10 14:50:04 2018 +0200
 
    BUG#27699608: WARNING FOR CONFLICTING EXPIRE-LOGS-[DAYS|SECONDS] SUPPRESSED WITH SKIP-LOG-BIN

3)

commit 9d591e29af1654b99a670d975ad8b9c95b011747
Author: Neha Kumari <neha.n.kumari@oracle.com>
Date:   Fri Feb 2 09:12:08 2018 +0530
 
    Bug#27252658 :CHANGE DEFAULT VALUE OF BINLOG_EXPIRE_LOGS_SECONDS

Comment by Sujatha Sivakumar (Inactive) [ 2021-02-09 ]

Hello Andrei,

Can you please review the changes for MDEV-19371.

There is only one minor change between upstream patch and its corresponding MariaDB
version, i.e the deprecation warnings are not written to server log, warnings get
printed only when user tries to set the value using mysql client.

Patch: https://github.com/MariaDB/server/commit/7ede75e1d2279e0cce3a27a93cc337d29af0fad7

BuildBot: http://buildbot.askmonty.org/buildbot/grid?category=main&branch=bb-10.6-sujatha

Thank you

Comment by Andrei Elkin [ 2021-04-07 ]

Suggested to extend the work content for the current user to stay with
their whole-days values if they are happy with that.
The existing option is proposed to become of a decimal type, so the fractional part to represent a part of the whole 24 hr and those who need second or so precision could specify that.
At the same time the oracle purge-in-seconds option will be accepted. The two are to show
the same value just in different units.

Comment by Sujatha Sivakumar (Inactive) [ 2021-05-03 ]

Hello Andrei,

I have addressed your review comments. Please review the latest changes.

Patch: https://github.com/MariaDB/server/commit/14ba1d3c29880314ca61ad2feafe53a1d4c8a3da

BuildBot Testing: http://buildbot.askmonty.org/buildbot/grid?category=main&branch=bb-10.6-sujatha

Thank you.

Comment by Andrei Elkin [ 2021-05-05 ]

The patch looks close to be fine. I left few notes on 14ba1d3c298.

Comment by Sujatha Sivakumar (Inactive) [ 2021-05-07 ]

Hello Andrei,

Thank you for the review comments.
I have addressed them in the latest patch.

Please review them.

https://github.com/MariaDB/server/commit/888af81f942e3d537b99f073d9504b4c8a7323b8
buildbot: http://buildbot.askmonty.org/buildbot/grid?category=main&branch=bb-10.6-sujatha

Thank you.

Comment by Sujatha Sivakumar (Inactive) [ 2021-05-10 ]

Patch was approved by Elkin and Brandon.

Comment by Andrei Elkin [ 2021-05-10 ]

For the documentation, the patch 49ff2cbff4
1. changes the type of expire_log_days to double to allow factional part of the whole day.
Backward compatible setting to an integer value is preserved (no need any post-upgrade actions)
2. adopts from the upstream binlog_expire_logs_seconds as an alias to expire_log_days so
the two variables show the same value just in different units and formats.

The changes to --help for expire_log_days can be seen in the patch:

"If non-zero, binary logs will be purged after expire_logs_days "

  • "days; possible purges happen at startup and at binary log rotation",
    + "days; It and binlog_expire_logs_seconds are aliases, such that "
    + "changes in one are converted into the other, presentable as a "
    + "decimal value with 1/1000000 of the day precision; possible "
    + "purges happen at startup and at binary log rotation",

The upstream new binlog_expire_logs_seconds is described as

+ "If non-zero, binary logs will be purged after "
+ "binlog_expire_logs_seconds seconds; It and expire_logs_days are "
+ "aliases, such that changes in one are converted into the other. "
+ "Possible purges happen at startup and at binary log rotation.",

the type is double,
the valid range is from 0 to 99*86400 seconds.
Values from 1 to 11 are rounded to zero 0.000000 of expire_log_days.

Comment by Ian Gilfillan [ 2021-05-11 ]

I suggest not using the word "alias" in the descriptions, as this implies that they are the same, and if a user doesn't look beyond that, they could end up specifying the wrong units. You could replace it with the word "linked", or "forms of aliases".

Comment by Andrei Elkin [ 2021-05-11 ]

Thanks, Ian! Agreed here. sujatha.sivakumar, could you please pick it up to a patch that fixes the valid ranges?

Comment by Sujatha Sivakumar (Inactive) [ 2021-05-11 ]

Thank greenman and Elkin. I will do the changes.

Comment by Sujatha Sivakumar (Inactive) [ 2021-05-11 ]

Hello Elkin

Please find the following post push fixes.

https://github.com/MariaDB/server/commit/2ea32e65a34ba065b3dd63d4c80071404dad1ca8
BuildBot: http://buildbot.askmonty.org/buildbot/grid?category=main&branch=bb-10.6-MDEV-19371

Please review them.

Thank you.

Generated at Thu Feb 08 08:51:10 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.