Ubuntu 14.04

This is possibly fixed by commit 7d7d8a0560023f84bcc292d0b74b98a00fb5c910.

I would suggest trying out this version of the cdc_kafka script: https://github.com/mariadb-corporation/MaxScale/tree/2.0/server/modules/protocol/examples

That is the version I am using. https://gist.github.com/Geesu/4b3a9316afe082c49b57e1b8d4a5a376

I'm using the 2.0.5 build you created for me in another issue (MXS-1191).

Ah, then this is a new bug. Does the cdc.py script work correctly without the Kafka producer part?

Yes, and it works correctly if I specify a table that doesn't have as many changes.

I'm actually quite surprised it hasn't come up before, it feels like this script could easily be a bottleneck for other clients trying to push to kafka as well.

This script is an example to show how CDC api can be used by inside a Kafka producer. Users are expected to write their own Kafka producers that uses CDC api and manage the scalability side of Kafka producers.

That's definitely not the impression from your docs/marketing.

geesu I think adding some debug output to the script should solve some of the issues. If you can alter the code from this:

   # JSONDecoder will return a ValueError if a partial JSON object is read

   except ValueError as err:

      pass

to this:

   # JSONDecoder will return a ValueError if a partial JSON object is read

   except ValueError as err:

      print(err)

      pass

you should start receiving errors in the standard output for invalid JSON. When it hangs, you should receive the same error over and over again. If this happens, please post the error (without any sensitive data). I suspect that the string to JSON conversion might be hitting some problems that cause it to constantly try and convert the same string into valid JSON.

Due to changes in the cdc.py script and how the avrorouter sends the data, I think the cdc_kafka_producer can be simplified even more. Please try this simplified version of the cdc_kafka_producer.py script and see if it solves the problems: https://gist.github.com/markus456/1e3f77693c7211df803e551729a2d417

I was originally using your script that doesn't do any JSON parsing (the one added to 2.0.5), so adding those exception handlers wouldn't help.

I just tried out the new script and it still wouldn't work, but it did work properly for a smaller table with fewer changes.

OK, then we can rule out the JSON parsing and deduce that something else is indeed going on with tables with large amounts of changes.

By a large amount of changes do you mean that the rate of changes is high or the amount of data in each change is large? If it's possible for you to give an example definition of a large table, we could try and see if we can reproduce it on our side.

Does the cdc.py script work alone without the kafka part even on the table with the large amount of changes?

If you replace the following lines (42 and 43 in the new script) with a call to print(data) does the script work?

      producer.send(topic=opts.kafka_topic, value=data)

      producer.flush()

If the script works without the actual Kafka part, I think the problem might be either in how we use the kafka python library or in the library itself.

Thanks - I think I have it figured out. Here is the exception it is throwing:

'ascii' codec can't decode byte 0xc3 in position 153: ordinal not in range(128)

I've attached the binary data it was trying to decode. last_data.bin

And the code that produced it: https://gist.github.com/Geesu/8cc7158dfdd135ee701d7893bfc15dfe

If I remove the encode() it works properly. Is that ok?

Yes, that should be OK. I don't think the encode call is necessary and it could probably be removed as the trailing newline can be removed with the following code:

data = buf[:-1]

Closing as fixed since a solution was found. 2.1.4 will be the first release with this fix.