[MDEV-26982] Create a template script to collect statistics about mysqlbinlog output Created: 2021-11-05  Updated: 2023-11-30

Status: Open
Project: MariaDB Server
Component/s: Replication
Fix Version/s: 11.5

Type: New Feature Priority: Major
Reporter: Brandon Nesterenko Assignee: Brandon Nesterenko
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Issue split
split from MDEV-26188 Extend mysqlbinlog with argument --pr... Closed
Relates
relates to MDEV-32674 Extend mariadb-binlog with Scripting ... Open

 Description   

In accordance with the discussion from PR 1855 we should provide a template script for end-users to monitor the process of mysqlbinlog.

To summarize, the usefulness of different metrics used for monitoring progress are very particular to the use case of the end user (e.g. file pos vs GTID, number/percentage of events processed, reporting periodicity, limiting monitoring to specific servers/domains, etc). We therefore do not implement progress monitoring into mysqlbinlog itself, but leave it to end-users to implement themselves in the pipeline. A very simple example that would extract basic file IO information in the pipeline would be:

mysqlbinlog <files> | tee >(grep "Start:|Rotate to|End of log" > io_progress.log ) | mysql

Expanding on this idea, we should provide a template script for end-users which is meant to be extended to meet their specific needs. For example, it could be called mysqlbinlog-record, and invoked as

mysqlbinlog <files> | mysqlbinlog-record | mysql

where it would not modify the output between mysqlbinlog and mysql, but perhaps keep some counters about what has happened (e.g. number of binary logs opened/closed) and overwrite a status file periodically using CSV format.

Potential inspiration for statistics could come from the Maxscale binlog router diagnostics page


Generated at Thu Feb 08 09:49:26 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.