[MXS-2007] Fatal: MaxScale 2.2.13 received fatal signal 11 (Aurora Monitor) Created: 2018-08-11  Updated: 2018-08-14  Resolved: 2018-08-14

Status: Closed
Project: MariaDB MaxScale
Component/s: Monitor
Affects Version/s: 2.2.13
Fix Version/s: 2.2.14

Type: Bug Priority: Major
Reporter: Jibu Chacko Assignee: markus makela
Resolution: Fixed Votes: 0
Labels: None
Environment:

Maxscale Version : 2.2.13
Ubuntu : 16.04

Sprint: MXS-SPRINT-64

 Description   

I am not abe to start Maxscale after adding Aurora Monitor

Maxscale Version : 2.2.13
Ubuntu : 16.04

Log :

MariaDB MaxScale /var/log/maxscale/maxscale.log Sat Aug 11 05:33:38 2018

----------------------------------------------------------------------------

2018-08-11 05:33:38 notice : Working directory: /var/log/maxscale

2018-08-11 05:33:38 notice : The collection of SQLite memory allocation statistics turned off.

2018-08-11 05:33:38 notice : Threading mode of SQLite set to Multi-thread.

2018-08-11 05:33:38 notice : MariaDB MaxScale 2.2.13 started (Commit: ec178fd8da484dd29560379a53fcecd11fd4f1c5)

2018-08-11 05:33:38 notice : MaxScale is running in process 10036

2018-08-11 05:33:38 notice : Configuration file: /etc/maxscale.cnf

2018-08-11 05:33:38 notice : Log directory: /var/log/maxscale

2018-08-11 05:33:38 notice : Data directory: /var/lib/maxscale

2018-08-11 05:33:38 notice : Module directory: /usr/lib/x86_64-linux-gnu/maxscale

2018-08-11 05:33:38 notice : Service cache: /var/cache/maxscale

2018-08-11 05:33:38 notice : Loading /etc/maxscale.cnf.

2018-08-11 05:33:38 notice : /etc/maxscale.cnf.d does not exist, not reading.

2018-08-11 05:33:38 notice : [cli] Initialise CLI router module

2018-08-11 05:33:38 notice : Loaded module cli: V1.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libcli.so

2018-08-11 05:33:38 notice : [readwritesplit] Initializing statement-based read/write split router module.

2018-08-11 05:33:38 notice : Loaded module readwritesplit: V1.1.0 from /usr/lib/x86_64-linux-gnu/maxscale/libreadwritesplit.so

2018-08-11 05:33:38 notice : Loaded module auroramon: V1.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libauroramon.so

2018-08-11 05:33:38 warning: Protocol module 'mysqlbackend' has been deprecated, use 'mariadbbackend' instead.

2018-08-11 05:33:38 notice : Loaded module mariadbbackend: V2.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbbackend.so

2018-08-11 05:33:38 notice : Loaded module MySQLBackendAuth: V1.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmysqlbackendauth.so

2018-08-11 05:33:38 notice : Loaded module maxscaled: V2.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmaxscaled.so

2018-08-11 05:33:38 notice : Loaded module MaxAdminAuth: V2.1.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmaxadminauth.so

2018-08-11 05:33:38 notice : Loaded module MariaDBClient: V1.1.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbclient.so

2018-08-11 05:33:38 notice : Loaded module MySQLAuth: V1.1.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmysqlauth.so

2018-08-11 05:33:38 notice : Monitor 'Aurora-Monitor' is missing the 'journal_max_age' parameter, using default value of 28800 seconds.

2018-08-11 05:33:38 notice : Monitor 'Aurora-Monitor' is missing the 'script_timeout' parameter, using default value of 90 seconds.

2018-08-11 05:33:38 notice : No query classifier specified, using default 'qc_sqlite'.

2018-08-11 05:33:38 notice : Loaded module qc_sqlite: V1.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libqc_sqlite.so

2018-08-11 05:33:38 warning: Removing stale journal file for monitor 'Aurora-Monitor'.

2018-08-11 05:33:38 notice : Using encrypted passwords. Encryption key: '/var/lib/maxscale/.secrets'.

2018-08-11 05:33:38 notice : Starting a total of 2 services...

2018-08-11 05:33:38 alert : Fatal: MaxScale 2.2.13 received fatal signal 11. Attempting backtrace.

2018-08-11 05:33:38 alert : Commit ID: ec178fd8da484dd29560379a53fcecd11fd4f1c5 System name: Linux Release string: Ubuntu 16.04.5 LTS

2018-08-11 05:33:38 alert : /usr/bin/maxscale() [0x4083fd]: ??:0

2018-08-11 05:33:38 alert : /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390) [0x7f4098925390]: ??:?

2018-08-11 05:33:38 alert : /usr/lib/x86_64-linux-gnu/maxscale/libauroramon.so(update_server_status+0xe1) [0x7f4093cf18e1]: /home/vagrant/MaxScale/server/modules/monitor/auroramon/auroramon.c:74

2018-08-11 05:33:38 alert : /usr/lib/x86_64-linux-gnu/maxscale/libauroramon.so(+0x1a7c) [0x7f4093cf1a7c]: /home/vagrant/MaxScale/server/modules/monitor/auroramon/auroramon.c:134

2018-08-11 05:33:38 alert : /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7f409891b6ba]: ??:?

2018-08-11 05:33:38 alert : /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f409708641d]: ??:0

MariaDB MaxScale /var/log/maxscale/maxscale.log Sat Aug 11 05:33:38 2018

----------------------------------------------------------------------------

2018-08-11 05:33:38 error : No data read from child process pipe.

2018-08-11 05:33:38 MariaDB MaxScale is shut down.

 Comments   
Comment by markus makela [ 2018-08-11 ]

To confirm whether this is crashing because of what I think it is, can you see if the following SQL returns any rows?

SELECT @@aurora_server_id, server_id FROM
 
    information_schema.replica_host_status 
    WHERE session_id = 'MASTER_SESSION_ID';

Comment by Jibu Chacko [ 2018-08-11 ] 

mysql> SELECT @@aurora_server_id, server_id FROM     information_schema.replica_host_status      WHERE session_id = 'MASTER_SESSION_ID';
 
+---------------------+---------------------+
 
| @@aurora_server_id  | server_id           |
 
+---------------------+---------------------+
 
| myserver-name| myserver-name |
 
+---------------------+---------------------+
 
1 row in set (0.00 sec)

Comment by markus makela [ 2018-08-11 ]

Does it return a row for all the servers configured for the monitor? I think that an empty row is what is causing the crash. In any case, this will be fixed in the next release.

Comment by Jibu Chacko [ 2018-08-11 ]

I resolved one issue by

apt install binutils

Now I have this error

Aug 11 21:21:36 ip-172-18-4-196 maxscale[3395]: Commit ID: ec178fd8da484dd29560379a53fcecd11fd4f1c5 System name: Linux Release string: Ubuntu 18.04.1 LTS
Aug 11 21:21:36 ip-172-18-4-196 maxscale[3395]: /usr/bin/maxscale(+0x83ab) [0x5585fc3b13ab]: /home/vagrant/MaxScale/server/core/gateway.cc:504
Aug 11 21:21:36 ip-172-18-4-196 maxscale[3395]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890) [0x7fbcd0ca6890]: ??:?
Aug 11 21:21:36 ip-172-18-4-196 maxscale[3395]: /usr/lib/x86_64-linux-gnu/maxscale/libauroramon.so(update_server_status+0x112) [0x7fbccba96882]: /home/vagra
Aug 11 21:21:36 ip-172-18-4-196 maxscale[3395]: /usr/lib/x86_64-linux-gnu/maxscale/libauroramon.so(+0x19f4) [0x7fbccba969f4]: /home/vagrant/MaxScale/server/
Aug 11 21:21:36 ip-172-18-4-196 maxscale[3395]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fbcd0c9b6db]: ??:?
Aug 11 21:21:36 ip-172-18-4-196 maxscale[3391]: nm: /lib/x86_64-linux-gnu/libc.so.6: no symbols
Aug 11 21:21:36 ip-172-18-4-196 maxscale[3395]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fbccf27f88f]: ??:0
Aug 11 21:21:36 ip-172-18-4-196 maxscale[3391]: Writing core dump
Aug 11 21:21:36 ip-172-18-4-196 maxscale[3391]: No data read from child process pipe.
Aug 11 21:21:36 ip-172-18-4-196 maxscale[3391]: * Error: No data read from child process pipe.
Aug 11 21:21:36 ip-172-18-4-196 systemd[1]: maxscale.service: Control process exited, code=exited status=6
Aug 11 21:21:36 ip-172-18-4-196 systemd[1]: maxscale.service: Failed with result 'exit-code'.
Aug 11 21:21:36 ip-172-18-4-196 systemd[1]: Failed to start MariaDB MaxScale Database Proxy.

Comment by Jibu Chacko [ 2018-08-11 ]

@markus

Yes... The Read only server returns an empty set

mysql> SELECT @@aurora_server_id, server_id FROM     information_schema.replica_host_status      WHERE session_id = 'MASTER_SESSION_ID';
 
Empty set (0.01 sec)

Comment by markus makela [ 2018-08-11 ]

OK, then I think I can quite confidently say it's fixed in the next release. I'll see if I can build debug packages that you could use to confirm the fix works.

Comment by Jibu Chacko [ 2018-08-12 ]

That would be really helpful. I really am in need of a working solution. Can you please build the debug packages?

Comment by markus makela [ 2018-08-12 ]

I've built debug packages that contain a fix: http://max-tst-01.mariadb.com/ci-repository/2.2-markusjm-aug12/mariadb-maxscale/

You can use them to test whether the fix works.

Comment by Jibu Chacko [ 2018-08-12 ]

It works!!! Thank you

Generated at Thu Feb 08 04:11:01 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.