Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-225

Replace with dummy events an event that is not understood by a slave to which it should be sent

    XMLWordPrintable

Details

    • Task
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • None
    • None
    • None

    Description

      When new features are added we may want to add new binary log events. But
      adding new events presents problems for replicating against older slaves that
      do not understand the new event.

      This is needed for eg. MDEV-181. The event that annotates row events with
      their original query (introduced in 5.2 or 5.3 I forget which) is another
      example.

      In MySQL 5.6+, there is a flag to mark an event as ignorable - a 5.6+ slave
      will ignore such an event, even if it does not know of it. But this does not
      work for 5.5 or earlier versions (of either MariaDB or MySQL).

      MariaDB 5.5 slave can tolerate that master just omits sending the event,
      leaving a "hole" in the binlog stream. However, MariaDB <5.5 or any MySQL will
      get confused on such holes, and will fail shortly after (because the
      accumulated size of events received no longer match the master's binlog). In
      fact, this happens when enabling row annotation events in MariaDB master and
      using a slave that does not understand row annotation events. This bug is
      fixed for mariadb 5.5 slave but remains for old mariadb slave or for mysql
      slave.

      The proposed solution here is that the master will for such older slaves not
      just omit the event. Instead it will replace the event with a dummy event of
      the same size - an empty query event (just a comment). The old slave will
      execute the event (with no effect) and avoid becoming confused or failing on
      unknown event.

      If the master detects that the slave is new enough to understand the original
      event (ie. understands the ignorable event flag), it will not substitute the
      dummy event.


      As dummy event we will use an empty query holding just a comment, like this:

      "# Dummy event replacing event type %u that slave cannot handle."

      The comment will be padded with spaces as needed, or truncated if the original
      event is shorter (the whole idea is to supply the slave with a dummy event of
      the same length, as old slaves do not tolerate holes in the binlog stream).

      The query event will have no status variables, so it is just the fixed headers
      + one null byte (required for empty db) + the comment. This gives a minimum
      event length of 34 bytes with a query consisting of a single character '#'.

      For shorter events, we can instead replace with a user_var event. The variable
      will be called @`!dummyvar` (or a shorter prefix of this that fits the length
      of the original event, possibly just @`!`). This allows to go down to an event
      length of just 25 bytes, 19 bytes of fixed header plus 6 bytes of data. I do
      not think there exists events shorter than this, but if there does, they can
      not be replaced with dummy events using this mechanism.


      To help the master know what events the slave does and does not understand, we
      will extend the slave to announce this.

      The slave will, when connecting, set a user variable
      @mariadb_slave_capability. The master will read the value of this variable
      and use it to determine the capabilities of the slave. This is more robust and
      flexible than relying on version numbers, since ideally any version master
      should be able to replicate against any version slave. When we release the
      code in a master, it is hard to predict what slave capabilities every possible
      version of future releases will have - especially with multiple forks/branches
      of MySQL being developed individually. And announcing the slave capabilities
      explicitly allows the slave code full flexibility in working correctly against
      older masters.

      /* MySQL or old MariaDB slave with no announced capability. */
      #define MARIA_SLAVE_CAPABILITY_UNKNOWN 0
      /* MariaDB >= 5.3, which understands ANNOTATE_ROWS_EVENT. */
      #define MARIA_SLAVE_CAPABILITY_ANNOTATE 1
      /*
        MariaDB >= 5.5. This version has the capability to tolerate events omitted
        from the binlog stream without breaking replication (MySQL slaves fail
        because they mis-compute the offsets into the master's binlog).
      */
      #define MARIA_SLAVE_CAPABILITY_TOLERATE_HOLES 2
      /* MariaDB > 5.5, which knows about binlog_checkpoint_log_event. */
      #define MARIA_SLAVE_CAPABILITY_BINLOG_CHECKPOINT 3
      /*
        MariaDB server which understands MySQL 5.6 ignorable events. This server
        can tolerate receiving any event with the LOG_EVENT_IGNORABLE_F flag set.
      */
      #define MARIA_SLAVE_CAPABILITY_IGNORABLE 4

      We can additionally push into 5.3 and 5.5 the functionality of the slave of
      setting @mariadb_slave_capability - this is a safe change, and will help
      mariadb masters >5.5 to work optimally against older MariaDB slaves.

      The capability is a simple integer, as capabilities are added in new releases
      but not removed.

      The master can use the capability number to know when it must not send a
      certain event to a certain slave. If it must not send, it can check if the
      capability is at least MARIA_SLAVE_CAPABILITY_TOLERATE_HOLES. If so, master
      can just skip the event and not send it to the slave - the slave is able to
      deal with the resulting hole in the binlog stream. But if not - master will
      replace the event with a dummy event, as implemented in this task, allowing
      the slave to ignore the event and replicate correctly.

      Attachments

        Issue Links

          Activity

            People

              knielsen Kristian Nielsen
              knielsen Kristian Nielsen
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.