Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-5262

Missing retry after temp error in parallel replication

    XMLWordPrintable

    Details

      Description

      If a transaction fails with temporary error (like a deadlock), replication
      must retry the transaction.

      But when using @@slave-parallel-threads > 0, this retry does not happen.

      It isn't trivial to correctly do such retry in the parallel case. MySQL 5.6
      multi-threaded slave, for example, does not do any retries.

      The main problem is where to get the events from to retry. Keeping all events
      from each transaction in-memory is not necessarily possible (eg. DELETE FROM
      table in row-based replication).

      The best design I could come up with so far is:

      • When an event is queued for replication in a worker thread, remember the
        relay log position of the end of the event, and the start position of the
        initial GTID event of the event group.
      • In case of temporary error that needs retry, temporarily set
        thd->wait_for_commit_ptr=NULL (to avoid notifying following transactions
        too early), rollback, restore thd->wait_for_commit_ptr.
      • Open a new file handle for the relay log file of the starting GTID of the
        event group, and seek to that GTID's position. Start reading and executing
        events from that relay log until we reach and successfully execute the
        event that caused the retry, then switch back to executing events queued in
        memory.
      • When retrying, we need to handle local rotate events (to be able to switch
        to a new relay log). Master does not switch logs within an event group, but
        we still need to handle and rollback in the case where master crashed in
        the middle of writing an event group (we get a rotate+format description
        before the end of the event group).
      • Implement reference counting for relay log files. Whenever an event is
        queued for a worker, increment the reference count for the containing relay
        log file. When an event group has completed, decrement the reference count
        for the relay log files of each event contained in the event group (so
        remember each relay log file used in the event group, and the count of
        events in each of them; an event group can span multiple relay log files).
      • Change automatic relay log purge to use the reference counting, so that we
        do not purge relay log files while a transaction may still need them for
        retry.
      • Implement the proper logic in the worker threads so that they are able to
        execute events either from the queued list or from re-reading from the
        relay logs. Also need to handle that the retry again fails, and a second
        retry is necessary.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              knielsen Kristian Nielsen
              Reporter:
              knielsen Kristian Nielsen
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: