Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-32570

Fragment ROW replication events larger than max_packet_size

    XMLWordPrintable

Details

    Description

      When ROW based replication is used and the transaction size, converted to base64 encoding, exceeds max_packet_size, replication breaks:

      Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'log event entry exceeded max_allowed_packet; Increase max_allowed_packet on master; the first event 'mariadb-bin.109824' at 4, the last event read from 'mariadb-bin.109825' at 15113554, the last byte read from 'mariadb-bin.109825' at 15113573.'
      

      Unfortunately, max_packet_size maximum allowed value is 1G.

      A blob over 500M large will cause the binlogged event to be larger than 1G so that it will crash replication, with NO workaround other than manually extracting the transaction from master's binlog and apply to slave.

      This ticket involves fragmenting row events larger than max_packet_size into multiple events. Additionally, Update_rows_log_events can be split into two series of events: 1) a series of deletes that cover the OLD rows of the update, and then 2) a series of inserts that cover the NEW rows of the update.

      Original solution kept for reference:

      My proposal: before binlogging the event on master, when the computed base64-encoded size is greater than max_packet_size, just convert format to STATEMENT for this event only.

      Attachments

        Issue Links

          Activity

            People

              bnestere Brandon Nesterenko
              rpizzi Rick Pizzi (Inactive)
              Votes:
              2 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.