[MDEV-32570] revert ROW replication to STATEMENT for events larger than max_packet_size Created: 2023-10-25  Updated: 2023-10-26

Status: Open
Project: MariaDB Server
Component/s: Replication, Server
Fix Version/s: 11.5

Type: Task Priority: Major
Reporter: Rick Pizzi Assignee: Andrei Elkin
Resolution: Unresolved Votes: 2
Labels: None


 Description   

When ROW based replication is used and the transaction size, converted to base64 encoding, exceeds max_packet_size, replication breaks:

Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'log event entry exceeded max_allowed_packet; Increase max_allowed_packet on master; the first event 'mariadb-bin.109824' at 4, the last event read from 'mariadb-bin.109825' at 15113554, the last byte read from 'mariadb-bin.109825' at 15113573.'

Unfortunately, max_packet_size maximum allowed value is 1G.

A blob over 500M large will cause the binlogged event to be larger than 1G so that it will crash replication, with NO workaround other than manually extracting the transaction from master's binlog and apply to slave.

My proposal: before binlogging the event on master, when the computed base64-encoded size is greater than max_packet_size, just convert format to STATEMENT for this event only.

 Comments   
Comment by Sergei Golubchik [ 2023-10-25 ]

falling back to STATEMENT is rather risky. Wouldn't it be much safer to fall back to binlog_row_image=NOBLOB or MINIMAL ?
It'll still be possible to hit max_packet_size limit, but it'll be much less likely. Also, switching to compressed events might help.

Comment by Rick Pizzi [ 2023-10-26 ]

In my opinion the proper thing to do would be to fail the query on master if binlogging is enabled and it can't be binlogged without breaking replication.
Since that may be a bit extreme, but at the same time we don't want to break replication (as manual fixing is VERY annoying and sometimes complicated),
maybe using MINIMAL would help (although it will not guarantee we do not break replication, only switching to STATEMENT will)

Comment by Rick Pizzi [ 2023-10-26 ]

In any case it should be possible, before writing the event to the binlog, to compute the actual size of the encoded event, then switch to the mode that offers the most complete info while fitting within max_allowed_packet. And if it can't fit even in MINIMAL, move to STATEMENT, or fail altogether.

Generated at Thu Feb 08 10:32:21 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.