[MDEV-29196] Request new system variable: binlog_keep_max Created: 2022-07-28 Updated: 2023-06-06 |
|
| Status: | Open |
| Project: | MariaDB Server |
| Component/s: | Replication |
| Fix Version/s: | None |
| Type: | Task | Priority: | Minor |
| Reporter: | Xan Charbonnet | Assignee: | Unassigned |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Issue Links: |
|
||||||||
| Description |
|
I have binary logs that are inconsistent in rate. That is, some days I write a tremendous amount to the log, and some days comparatively very little. I have a partition dedicated to binary logs, and I always want to keep a minimum of 3 days' worth of logs. That part is easily achievable by setting binlog_expire_logs_seconds (or its alias, expire_logs_days). But most of the time, I have plenty of disk space on that partition and would like to store far more than 3 days' worth of logs. Most of the time, setting it to 7 days works, but then every once in a while I run out of space. What I would like to do is set binlog_expire_logs_seconds to 3 days, and then have a new "binlog_keep_max" Boolean variable. Setting that to true would always allow the binary logs to fill up the partition, only deleting logs when the partition is full (but of course refusing to delete logs if they're newer than binlog_expire_logs_seconds). There's no reason NOT to have that partition be full of binary logs. |
| Comments |
| Comment by Andrei Elkin [ 2022-07-29 ] |
|
xan@biblionix.com, could you review the following description that addresses your concern? The idea of garbage collecting by the user instead of automatic purging plus |
| Comment by Xan Charbonnet [ 2022-07-29 ] |
|
Andrei, Thanks for considering this. I'm not sure I'm understanding just how "pull" and "push" notifications would work. The only snag I see there is that at some point, the binlog numbering loops around, and my script would have to be smart in a kind of ugly way about handling that situation. Or it could go by timestamps I guess... Is Maria not in a position to automatically purge such logs when the partition reaches, say, 90% full? Or reaches some threshold size that might be set in a variable? That would be much more straightforward. |
| Comment by Andrei Elkin [ 2022-07-29 ] |
|
> Would that variable contain something along the lines of the first logfile that has passed the binlog_expire_logs_seconds threshold, allowing me to delete that and all older ones? Like that too. And this looks more practical to me. (And yes, that's what I meant as pushing, as you guessed; alternatively one would periodically SELECT/"pull" the same info from the new status var). |
| Comment by Xan Charbonnet [ 2022-07-29 ] |
|
That sounds good, with the possible exception of this part: "do with that file and possible smaller index ones as you want." The earlier binlog files do not necessarily have smaller indexes: when the index number loops around, that assumption does not hold true. My fear is that this will lead to some insidious bugs in scripts people use to handle this. |
| Comment by Andrei Elkin [ 2022-07-29 ] |
|
Well, let's also return the first binlog file in the index. |
| Comment by Xan Charbonnet [ 2022-07-29 ] |
|
Beautiful! |
| Comment by Xan Charbonnet [ 2022-07-29 ] |
|
Or possibly even easier, at least for the user, would be a function that answers whether a particular log file is deletable or not. That way there's no room for error. |
| Comment by Xan Charbonnet [ 2022-08-14 ] |
|
The solution here might end up being related to the solution for MDEV-29195. |