[MXS-1576] Maxscale crashes when starting if .avro and .avsc files are present Created: 2017-12-12  Updated: 2018-01-16  Resolved: 2018-01-16

Status: Closed
Project: MariaDB MaxScale
Component/s: avrorouter
Affects Version/s: 2.1.11
Fix Version/s: 2.1.14

Type: Bug Priority: Major
Reporter: Drew Schatt Assignee: markus makela
Resolution: Fixed Votes: 0
Labels: None
Environment:

Centos 7.4, x86_64

Sprint: 2017-49, MXS-SPRINT-50

 Description   

Started maxscale. Set up binlog/avro routing. Started slave, it runs through all the binlogs and creates corresponding .avro and .avsc files.

Killed maxscale, on restarting, we get:

[root@nf-mdb-maxscale01 dschatt]# /bin/maxscale

MariaDB Corporation MaxScale 2.1.11 Thu Dec 7 23:08:58 2017
------------------------------------------------------
[root@nf-mdb-maxscale01 dschatt]# maxscale: /home/vagrant/MaxScale/server/modules/routing/avrorouter/avro_rbr.c:630: process_row_event_data: Assertion `ptr < end' failed.
Fatal: MaxScale 2.1.11 received fatal signal 6. Attempting backtrace.
Commit ID: source-build System name: Linux Release string: NAME="CentOS Linux"

/bin/maxscale[0x4043a0]
/lib64/libpthread.so.0(+0xf5e0)[0x7f01baa385e0]
/lib64/libc.so.6(gsignal+0x37)[0x7f01b91c11f7]
/lib64/libc.so.6(abort+0x148)[0x7f01b91c28e8]
/lib64/libc.so.6(+0x2e266)[0x7f01b91ba266]
/lib64/libc.so.6(+0x2e312)[0x7f01b91ba312]
/usr/lib64/maxscale/libavrorouter.so(process_row_event_data+0x11a7)[0x7f01b5620eb1]
/usr/lib64/maxscale/libavrorouter.so(handle_row_event+0x56a)[0x7f01b561f514]
/usr/lib64/maxscale/libavrorouter.so(avro_read_all_events+0xa50)[0x7f01b562363e]
/usr/lib64/maxscale/libavrorouter.so(converter_func+0xbd)[0x7f01b5616231]
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x466a4)[0x7f01bb1666a4]
/lib64/libpthread.so.0(+0x7e25)[0x7f01baa30e25]
/lib64/libc.so.6(clone+0x6d)[0x7f01b928434d]

Writing core dump

 Comments   
Comment by markus makela [ 2017-12-12 ]

Removing the avro-conversion.ini found in /var/lib/maxscale might fix it if the conversion file position tracking is wrong.

Comment by Drew Schatt [ 2017-12-16 ]

Removing that file does indeed let maxscale start up, but it seems to lose the position for the avro files if it's removed. So while it's a temporary workaround, it doesn't seem like it is a production level fix.

Comment by markus makela [ 2018-01-03 ]

How do you kill MaxScale? Stopping the process via systemctl should stop the conversion process at a well-defined point. If this is how you stop MaxScale and it still loses the position, there is something wrong with the code that aborts the conversion process.

Comment by markus makela [ 2018-01-15 ]

A guaranteed way to stop the avrorouter at a well-defined point is to issue the following MaxAdmin command: maxadmin call command avrorouter convert <service-name> stop

The <service-name> should be the name of the avrorouter service. The command stops the conversion process which makes sure that no data is being converted when MaxScale is stopped.

If possible, please try and see if the aforementioned steps result in a successful startup. If it is successful, I suspect that the way the internal task threads in MaxScale are stopped does not allow enough time for the tasks to complete.

Comment by markus makela [ 2018-01-16 ]

schatt Any updates on this?

Comment by markus makela [ 2018-01-16 ]

Added the new command, if the problem persists, reopen this issue.

Generated at Thu Feb 08 04:07:49 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.