Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-33856

Alternative Replication Lag Representation via Received/Executed Master Binlog Event Timestamps

Details

    Description

      The current definition of Seconds_Behind_Master is both complex and confusing.
      This can be seen in the documentation of the variable at:
      https://mariadb.com/kb/en/delayed-replication/

      The current value is also not very useful as the value can have strange values in the case the master has long pauses between entries.

      The suggested changes are:

      • Parallel execution uses the time AFTER execution while 'normal' replication used the value BEFORE to calculate Seconds_Behind_Master. This should be changed to always use AFTER.
      • Change Seconds_Behind_Master to have a more well defined meaning: 'The slave data is snapshot XXX seconds behind the data on the master. This can be calculated by using the formula: Newest event completion time from master (when writing to the relay log) - Completion time for last committed event from the master. If the slave is 'idle' then the Seconds_Behind_Master should be 0.

      As an example with SQL_DELAY=5 hours:

      • IO thread reads an event that was completed at 2023:01:01 06:01:01 on the master
      • The SQL thread commits later an event that was originally committed at 2023:01:01 01:01:01

      In this case the Seconds_Behind_master will be 5 hours.

      The benefit of this idea is that 'Seconds_Behind_Master' will be well defined in all of the following cases:

      • Without and without delayed replication
      • When the master has been down and is just coming up.
      • If the master has been 'idle' for some time.
      • When the slave is stopped and later restarted (Seconds_Behind_Master will be up to date as soon as IO thread is up to date). We could have make the value 'correct' from start by setting Newest event completion time from master as 'master current time' when connecting.

      Note too that the final commit also adds in the table information_schema.slave_status (MDEV-33526), as an alias for SHOW ALL SLAVES STATUS. So these fields can now be queried by SELECT statements.

      Attachments

        Issue Links

          Activity

            A summary of the commit:

            The following new variables replaces 'seconds_behind_master':

            • Master_last_event_time Timestamp of the last committed event read from the master by the IO thread.
            • Slave_last_event_time Master timestamp of the last event committed on the slave.
            • Master_Slave_time_diff The difference of the above two timestamps.

            In addition SLAVE STATUS is now part of information_schema.slave_status. One can now query this table instead of using SHOW SLAVE STATUS or SHOW ALL SLAVES STATUS.
            One benefit of information_schema.slave_status is that it is now easy to just extract some of the fields instead of having to always get all fields as one gets with SHOW SLAVE STATUS.

            monty Michael Widenius added a comment - A summary of the commit: The following new variables replaces 'seconds_behind_master': Master_last_event_time Timestamp of the last committed event read from the master by the IO thread. Slave_last_event_time Master timestamp of the last event committed on the slave. Master_Slave_time_diff The difference of the above two timestamps. In addition SLAVE STATUS is now part of information_schema.slave_status. One can now query this table instead of using SHOW SLAVE STATUS or SHOW ALL SLAVES STATUS. One benefit of information_schema.slave_status is that it is now easy to just extract some of the fields instead of having to always get all fields as one gets with SHOW SLAVE STATUS.
            knielsen Kristian Nielsen added a comment - Review: https://lists.mariadb.org/hyperkitty/list/developers@lists.mariadb.org/thread/LQZOIASN4VSOFQJXVDOOO5NCYOELEXOS/

            Hi knielsen!

            I took over for Monty in making the edits for your review. Can you review the changes?

            + e1ee2ed0522...071dcfaeca5 bb-11.6-MDEV-33856-bnestere -> bb-11.6-MDEV-33856-bnestere
            

            bnestere Brandon Nesterenko added a comment - Hi knielsen ! I took over for Monty in making the edits for your review. Can you review the changes? + e1ee2ed0522...071dcfaeca5 bb-11.6-MDEV-33856-bnestere -> bb-11.6-MDEV-33856-bnestere

            I had a quick look, looks ok, thanks.

            - Kristian.

            knielsen Kristian Nielsen added a comment - I had a quick look, looks ok, thanks. - Kristian.

            Pushed on Monty's behalf into 11.6 as 25b5c63905682a0e38d8ed86170d98be6ddb8cf8

            bnestere Brandon Nesterenko added a comment - Pushed on Monty's behalf into 11.6 as 25b5c63905682a0e38d8ed86170d98be6ddb8cf8

            People

              monty Michael Widenius
              monty Michael Widenius
              Votes:
              3 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.