When a Kafka consumer is restarted, a common problem that occurs is that the Kafka producer re-sends records that the consumer had already received before the restart. One way to prevent this problem is to design the Kafka producer to keep track of the last record delivered to the consumer, so that if the consumer is restarted, the producer does not send that record again.

Some users would like cdc_kafka_producer.py to have a mechanism like this.