[MXS-3897] MaxScale crashes when executing CDC process to kafka Created: 2021-12-03  Updated: 2022-02-14  Resolved: 2021-12-22

Status: Closed
Project: MariaDB MaxScale
Component/s: cdc
Affects Version/s: 2.4.18, 2.5.17, 6.2.0
Fix Version/s: 2.5.18, 6.2.1

Type: Bug Priority: Major
Reporter: Pon Suresh Pandian (Inactive) Assignee: Niclas Antti
Resolution: Fixed Votes: 0
Labels: None
Environment:

Ubuntu - 20.04.2 LTS

Attachments: File maxscale.cnf     Text File maxscale.log     HTML File syslog    

 Description   

Hi Team,

We are configuring maxscale for cdc to kafka but when we start cdc.py process , maxscale stops after sometime with the below errors:

Note :
------
Maxscale and DB both are on the same server as we are doing it in dev environment before proceeding to production.

We are invoking the commands similar to the following for cdc:

  1. cdc.py --user=myuser --password=mypasswd --host=127.0.0.1 --port=4001 test.mytable | cdc_kafka_producer.py --kafka-broker 127.0.0.1:9092 --kafka-topic test.mytable

Here I have attached the maxscale config & log file please check it.

Error :

2021-12-02 14:52:24   notice : [CDC] (avro-listener); avro-converter: new connection from [::ffff:127.0.0.1]
 
alert  : MaxScale 6.1.4 received fatal signal 6. Commit ID: d358b90fc6c45c1875b5e32133b19b80bddf117d System name: Linux Release string: Ubuntu 20.04.2 LTS
 
 
 
 
 
2021-12-02 14:53:30   alert  : MaxScale 6.1.4 received fatal signal 6. Commit ID: d358b90fc6c45c1875b5e32133b19b80bddf117d System name: Linux Release string: Ubuntu 20.04.2 LTS
 
2021-12-02 14:53:30   alert  : Statement currently being classified: none/unknown
 
nm: /lib/x86_64-linux-gnu/libc.so.6: no symbols
 
nm: /lib/x86_64-linux-gnu/libc.so.6: no symbols
 
alert  :   /lib/x86_64-linux-gnu/libc.so.6(epoll_wait+0x5e): ??:0
 
  /usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker15poll_waiteventsEv+0x155): maxutils/maxbase/src/worker.cc:797
 
  /usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker3runEPNS_9SemaphoreE+0x53): maxutils/maxbase/src/worker.cc:562
 
  /usr/bin/maxscale(main+0x195e): server/core/gateway.cc:2213
 
  /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3): ??:0
 
  /usr/bin/maxscale(_start+0x2e): ??:?
 
 
 
 
 
2021-12-02 14:53:30   alert  : 
  /lib/x86_64-linux-gnu/libc.so.6(epoll_wait+0x5e): ??:0
 
  /usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker15poll_waiteventsEv+0x155): maxutils/maxbase/src/worker.cc:797
 
  /usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker3runEPNS_9SemaphoreE+0x53): maxutils/maxbase/src/worker.cc:562
 
  /usr/bin/maxscale(main+0x195e): server/core/gateway.cc:2213
 
  /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3): ??:0
 
  /usr/bin/maxscale(_start+0x2e): ??:?
 
alert  : Writing core dump.



 Comments   
Comment by Niclas Antti [ 2021-12-21 ]

There is a bug in the avro code when the buffer size is set higher than 32k.
In the avro config, remove the line
block_size=1024K
The block size will then be defaulted to 16k and avro will behave correctly . The bug fix will be in the next release.

Generated at Thu Feb 08 04:24:44 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.