[MXS-4095] Maxscale CDC to Kafka job is failing for few tables Created: 2022-04-14  Updated: 2022-05-18  Resolved: 2022-05-04

Status: Closed
Project: MariaDB MaxScale
Component/s: cdc
Affects Version/s: 6.2.3
Fix Version/s: 6.3.1

Type: Bug Priority: Major
Reporter: Pon Suresh Pandian (Inactive) Assignee: markus makela
Resolution: Fixed Votes: 0
Labels: None
Environment:

Ubuntu 20.04

Sprint: MXS-SPRINT-156

 Description   

Hi Team,

Maxscale for CDC of multiple tables into Kafka its running for most of the tables but is not starting for few tables. We are getting the following error in the log, when trying to start CDC Job

From Maxscale Logs :

2022-04-14 16:38:52   notice : Service 'avro-converter-PI2' started (4/4)
 
2022-04-14 16:38:52   notice : Started replicating from [10.8.2x.xx]:3306 at GTID '0-202-74961217'
 
2022-04-14 16:38:52   notice : Started replicating from [10.8.2x.xx]:3306 at GTID '0-197-76367692'
 
2022-04-14 16:38:53   notice : Read 191 user@host entries from 'server2' for service 'Binlog-Service-PI2'.
 
2022-04-14 16:38:53   notice : Read 165 user@host entries from 'server1' for service 'Binlog-Service-PI1'.
 
2022-04-14 16:42:11   error  : Sync marker mismatch.
 
2022-04-14 16:42:11   error  : [avrorouter] Reading Avro file failed with error 'MAXAVRO_ERR_IO'.
 
2022-04-14 16:42:11   error  : Write to Client DCB ::ffff:10.8.2x.xx in state DCB::State::POLLING failed: 104, Connection reset by peer
 
2022-04-14 16:42:11   error  : Write to Client DCB ::ffff:10.8.2x.xx in state DCB::State::POLLING failed: 104, Connection reset by peer
 
2022-04-14 16:42:11   error  : Sync marker mismatch.
 
2022-04-14 16:42:11   error  : [avrorouter] Reading Avro file failed with error 'MAXAVRO_ERR_IO'.
 
2022-04-14 16:42:11   error  : Sync marker mismatch.
 
2022-04-14 16:42:11   error  : [avrorouter] Reading Avro file failed with error 'MAXAVRO_ERR_IO'.
 
2022-04-14 16:42:12   error  : Sync marker mismatch.
 
2022-04-14 16:42:12   error  : [avrorouter] Reading Avro file failed with error 'MAXAVRO_ERR_IO'.
 
2022-04-14 16:42:12   error  : Write to Client DCB ::ffff:10.8.2x.xx in state DCB::State::POLLING failed: 104, Connection reset by peer
 
2022-04-14 16:42:12   error  : Sync marker mismatch.
 
2022-04-14 16:42:12   error  : [avrorouter] Reading Avro file failed with error 'MAXAVRO_ERR_IO'.
 
2022-04-14 16:42:12   error  : Write to Client DCB ::ffff:10.8.2x.xx in state DCB::State::POLLING failed: 104, Connection reset by peer
 
2022-04-14 16:42:12   error  : Write to Client DCB ::ffff:10.8.2x.xx in state DCB::State::POLLING failed: 104, Connection reset by peer
 
2022-04-14 16:42:14   error  : Sync marker mismatch.
 
2022-04-14 16:42:14   error  : [avrorouter] Reading Avro file failed with error 'MAXAVRO_ERR_IO'.
 
2022-04-14 16:42:16   error  : Sync marker mismatch.
 
2022-04-14 16:42:16   error  : [avrorouter] Reading Avro file failed with error 'MAXAVRO_ERR_IO'.
 
2022-04-14 16:42:18   error  : Write to Client DCB ::ffff:10.8.2x.xx in state DCB::State::POLLING failed: 104, Connection reset by peer
 
2022-04-14 16:42:20   error  : Sync marker mismatch.
 
2022-04-14 16:42:20   error  : [avrorouter] Reading Avro file failed with error 'MAXAVRO_ERR_IO'.



 Comments   
Comment by markus makela [ 2022-04-21 ]

Seems like this might be related to the actual data in the tables. We'll need an example table and some data to reproduce this.

Comment by markus makela [ 2022-04-25 ]

The Write to Client DCB ::ffff:10.8.2x.xx in state DCB::State::POLLING failed: 104, Connection reset by peer errors seem to be caused by a bug in the error detection code in the cdc.py example script. The Sync marker mismatch errors are still caused by something unknown.

Generated at Thu Feb 08 04:26:10 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.