Seconds_Behind_Master spikes to millions of seconds

Hey everyone

Rarely, we have a weird issue happening. We have a master <-> master setup. The writes are going to only one of them. Every now and then, the replication lag on the master where we send writes jumps to millions of seconds (ex: 2879736). Then, after 1s, is back to 0. After all, there's no lag, as the writes and the binlog from the other master should be ignored, as this is the server where we write to.
Both servers have the same timezone and ntp running.
Replication is set with GTIDs, and parallel threads.

Sample config:
slave_net_timeout=2
gtid-strict-mode=1
gtid-domain-id=1
server_id = 101
sync_binlog = 1
binlog_format = ROW
log-slave-updates
replicate-same-server-id = 0
log_bin=/var/log/mysql/binlog/mysql-bin.log
log_bin_index=/var/log/mysql/binlog/mysql-bin.log.index
relay_log=/var/log/mysql/binlog/mysql-relay-bin
relay_log_index=/var/log/mysql/binlog/mysql-relay-bin.index
expire_logs_days=5
max_binlog_size=1G
slave_parallel_threads=10

We have a bunch of monitoring scripts looking at that value and triggering actions and alerts and we want to know what can cause this and if there's anything we can do to avoid this in the future.

Thank you