[MXS-1851] Using backend protocol as client protocol causes a crash Created: 2018-05-04  Updated: 2019-05-28  Resolved: 2019-05-28

Status: Closed
Project: MariaDB MaxScale
Component/s: Core
Affects Version/s: 2.2.5
Fix Version/s: 2.3.8

Type: Bug Priority: Minor
Reporter: Venkatarao Beru Assignee: markus makela
Resolution: Fixed Votes: 0
Labels: None
Environment:

software platform


Issue Links:
Duplicate
is duplicated by MXS-2104 Aurora Monitor maxscale crash Closed

 Description   

We are trying to setup MaxScale 2.2.5 so we untar the software and created /etc/maxscale.cnf file. We got the below errors while starting up maxscale.

$ ./bin/maxscale --user=maxscale --basedir=/mysql/maxscale --config=/etc/maxscale.cnf -d
Info : MaxScale will be run in the terminal process.
Configuration file : /etc/maxscale.cnf
Log directory : /mysql/maxscale/var/log/maxscale
Data directory : /mysql/maxscale/var/lib/maxscale
Module directory : /mysql/maxscale/lib64/maxscale
Service cache : /mysql/maxscale/var/cache/maxscale

Fatal: MaxScale 2.2.5 received fatal signal 11. Attempting backtrace.
Commit ID: d9e69413b4d18da24aa04e68e3ce9208159a267a System name: Linux Release string: Red Hat Enterprise Linux Server release 7.4 (Maipo)

./bin/maxscale[0x407901]
/lib64/libpthread.so.0(+0xf5e0)[0x7f41367d65e0]

Writing core dump
Segmentation fault

Could you please let us know, is it a bug? or if not please provide us a solution to fix this.

Thanks,
Venkat Beru



 Comments   
Comment by markus makela [ 2018-05-04 ]

Yes, this appears to be a bug. Can you confirm that this was with the tarball, not the RPM package?

Comment by Venkatarao Beru [ 2018-05-04 ]

Yes. We installed with the following tar ball

maxscale-2.2.5.rhel.7.tar.gz

Comment by markus makela [ 2018-05-04 ]

Would it be possible for you to confirm that the 2.2.4 tarball works? This would tell us whether 2.2.5 introduced some changes that triggered this.

Another thing I would like to ask you to verify is whether this happens only with the tarball. To do this, install MaxScale from the RPM package and start it up with the same configuration file.

Also, if possible, please upload your configuration file with all sensitive data (passwords, server hostnames etc.) removed.

Comment by Venkatarao Beru [ 2018-05-04 ]

Thank you Makela for your quick responses and updates.

Please find requested information.

We have not tried to install 2.2.4 tarball. Let me try to install this.

Yes. This happened with "maxscale-2.2.5.rhel.7.tar.gz" tar ball s/w. Let me install with RPM package and will get back to you.

Please find my configuration file

####################################################################
 
#######################################################################
########################## Maxscale Configuration #####################
#######################################################################
 
[MaxScale]
threads=auto
admin_auth=http
 
[SplitterService]
type=service
router=readwritesplit
servers=SERVER1,SERVER2
user=maxscale
passwd=**************
master_accept_reads=false
master_failure_mode=error_on_write
slave_selection_criteria=LEAST_CURRENT_OPERATIONS
 
[SlaveReadOnly]
type=service
servers=SERVER2
router=readconnroute
user=maxscale
passwd=**************
 
[SplitterListener]
type=listener
service=SplitterService
protocol=MariaDBBackend
port=3333
#socket=/tmp/maxscale
 
[SERVER1]
type=server
address=SERVER1
port=3307
protocol=MariaDBBackend
 
[SERVER2]
type=server
address=SERVER2
port=3307
protocol=MariaDBBackend
 
[CLI]
type=service
router=cli
 
[CLIListener]
type=listener
service=CLI
protocol=maxscaled
address=localhost
port=6603
 
[MaxAdminService]
type=service
router=cli
 
[MaxAdminListener]
type=listener
service=MaxAdminService
protocol=maxscaled
address=localhost
port=4444
#socket=default
 
[MyMonitor]
type=monitor
module=mariadbmon
servers=SERVER1,SERVER2
user=maxscale
passwd=**************
 
###################################################################3

We have installed mariadb on SERVER1 and SERVER2 and configured bi directional (MASTER↔MASTER) replication for time being. Once maxscale successfully configured then will setup MASTER→SLAVE replication.

Please find mariadb versions on SERVER1 and SERVER2

SERVER1
========

-bash-4.2$ mysql --version
mysql Ver 15.1 Distrib 10.2.12-MariaDB, for Linux (x86_64) using readline 5.1

SERVER2
========

-bash-4.2$ mysql --version
mysql Ver 15.1 Distrib 10.2.12-MariaDB, for Linux (x86_64) using readline 5.1

 

Thanks,
Venkat Beru

Comment by Venkatarao Beru [ 2018-05-04 ]

As you said, untar the tar ball "maxscale-2.2.4.rhel.7.tar.gz" and got the below error

$ pwd
/mysql/mariadb/maxscale-2.2.4.rhel.7

[maxscale@dbcct3vmyd01006 maxscale-2.2.4.rhel.7]$ ./bin/maxscale --user=maxscale --basedir=/mysql/mariadb/maxscale-2.2.4.rhel.7 --config=/etc/maxscale.cnf -d
./bin/maxscale: error while loading shared libraries: libmaxscale-common.so.1.0.0: cannot open shared object file: No such file or directory

Please let me know, is there any library issue or any RPMs needs to be installed at o/s level i.e., RHEL 7

$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.4 (Maipo)

Thanks,
Venkat Beru

Comment by Venkatarao Beru [ 2018-05-04 ]

We tried maxscale 2.2.5 setup by RPM but it has also same issue. Please find details

  1. rpm -qa |grep maxscale
    maxscale-2.2.5-1.x86_64

[maxscale@dbcct3vmyd01006 maxscale]$ which maxscale
/usr/bin/maxscale
[maxscale@dbcct3vmyd01006 maxscale]$ /usr/bin/maxscale --user=maxscale -d
Info : MaxScale will be run in the terminal process.
Configuration file : /etc/maxscale.cnf
Log directory : /var/log/maxscale
Data directory : /var/lib/maxscale
Module directory : /usr/lib64/maxscale
Service cache : /var/cache/maxscale

Fatal: MaxScale 2.2.5 received fatal signal 11. Attempting backtrace.
Commit ID: d9e69413b4d18da24aa04e68e3ce9208159a267a System name: Linux Release string: Red Hat Enterprise Linux Server release 7.4 (Maipo)

/usr/bin/maxscale[0x407921]
/lib64/libpthread.so.0(+0xf5e0)[0x7f361c0e05e0]

Writing core dump
Segmentation fault

      1. /var/log/maxscale/maxscale.log file contents

MariaDB MaxScale /var/log/maxscale/maxscale.log Fri May 4 08:34:30 2018
----------------------------------------------------------------------------
2018-05-04 08:34:30 notice : The collection of SQLite memory allocation statistics turned off.
2018-05-04 08:34:30 notice : Threading mode of SQLite set to Multi-thread.
2018-05-04 08:34:30 notice : MariaDB MaxScale 2.2.5 started
2018-05-04 08:34:30 notice : MaxScale is running in process 5678
2018-05-04 08:34:30 notice : Configuration file: /etc/maxscale.cnf
2018-05-04 08:34:30 notice : Log directory: /var/log/maxscale
2018-05-04 08:34:30 notice : Data directory: /var/lib/maxscale
2018-05-04 08:34:30 notice : Module directory: /usr/lib64/maxscale
2018-05-04 08:34:30 notice : Service cache: /var/cache/maxscale
2018-05-04 08:34:30 notice : Loading /etc/maxscale.cnf.
2018-05-04 08:34:30 notice : /etc/maxscale.cnf.d does not exist, not reading.
2018-05-04 08:34:30 notice : [mariadbmon] Initialise the MariaDB Monitor module.
2018-05-04 08:34:30 notice : Loaded module mariadbmon: V1.5.0 from /usr/lib64/maxscale/libmariadbmon.so
2018-05-04 08:34:30 notice : [cli] Initialise CLI router module
2018-05-04 08:34:30 notice : Loaded module cli: V1.0.0 from /usr/lib64/maxscale/libcli.so
2018-05-04 08:34:30 notice : [readconnroute] Initialise readconnroute router module.
2018-05-04 08:34:30 notice : Loaded module readconnroute: V1.1.0 from /usr/lib64/maxscale/libreadconnroute.so
2018-05-04 08:34:30 notice : [readwritesplit] Initializing statement-based read/write split router module.
2018-05-04 08:34:30 notice : Loaded module readwritesplit: V1.1.0 from /usr/lib64/maxscale/libreadwritesplit.so
2018-05-04 08:34:30 notice : Loaded module MariaDBBackend: V2.0.0 from /usr/lib64/maxscale/libmariadbbackend.so
2018-05-04 08:34:30 notice : Loaded module MySQLBackendAuth: V1.0.0 from /usr/lib64/maxscale/libmysqlbackendauth.so
2018-05-04 08:34:30 notice : Monitor 'monitor_interval' is missing the 'MyMonitor' parameter, using default value of 2000 milliseconds.
2018-05-04 08:34:30 notice : Monitor 'MyMonitor' is missing the 'journal_max_age' parameter, using default value of 28800 seconds.
2018-05-04 08:34:30 notice : Monitor 'MyMonitor' is missing the 'script_timeout' parameter, using default value of 90 seconds.
2018-05-04 08:34:30 notice : Loaded module maxscaled: V2.0.0 from /usr/lib64/maxscale/libmaxscaled.so
2018-05-04 08:34:30 notice : Loaded module MaxAdminAuth: V2.1.0 from /usr/lib64/maxscale/libmaxadminauth.so
2018-05-04 08:34:30 notice : No query classifier specified, using default 'qc_sqlite'.
2018-05-04 08:34:30 notice : Loaded module qc_sqlite: V1.0.0 from /usr/lib64/maxscale/libqc_sqlite.so
2018-05-04 08:34:30 warning: Removing stale journal file for monitor 'MyMonitor'.
2018-05-04 08:34:30 notice : Using encrypted passwords. Encryption key: '/var/lib/maxscale/.secrets'.
2018-05-04 08:34:30 notice : Starting a total of 4 services...
2018-05-04 08:34:30 alert : Fatal: MaxScale 2.2.5 received fatal signal 11. Attempting backtrace.
2018-05-04 08:34:30 alert : Commit ID: d9e69413b4d18da24aa04e68e3ce9208159a267a System name: Linux Release string: Red Hat Enterprise Linux Server release 7.4 (Maipo)
2018-05-04 08:34:30 alert : /usr/bin/maxscale() [0x407921]: ??:0
2018-05-04 08:34:30 alert : /lib64/libpthread.so.0(+0xf5e0) [0x7f361c0e05e0]: sigaction.c:?

Please suggest us to overcome this setup issue.

Thanks,
Venkat Beru

Comment by Todd Stoffel (Inactive) [ 2018-05-04 ]

[SplitterListener]
type=listener
service=SplitterService
protocol=MariaDBBackend
port=3333
#socket=/tmp/maxscale

The protocol should be MariaDBClient

Comment by Venkatarao Beru [ 2018-05-05 ]

Thank you Todd for finding the issue. After chaging the protocol, Maxscale started. Currently RPM has been installed on the server. Let me remove RPM then install TAR ball again then will check Maxscale can be started without having any issues. Please find maxscale status with RPM installation

MaxScale> show service "Splitter Service"
Invalid argument: Splitter-Service
MaxScale> show service "SplitterService"
Service: SplitterService
Router: readwritesplit
State: Started

use_sql_variables_in: all
slave_selection_criteria: LEAST_CURRENT_OPERATIONS
master_failure_mode: error_on_write
max_slave_replication_lag: -1
retry_failed_reads: true
strict_multi_stmt: false
strict_sp_calls: false
disable_sescmd_history: true
max_sescmd_history: 0
master_accept_reads: false

Number of router sessions: 0
Current no. of router sessions: 1
Number of queries forwarded: 0
Number of queries forwarded to master: 0 (0.00%)
Number of queries forwarded to slave: 0 (0.00%)
Number of queries forwarded to all: 0 (0.00%)
Started: Fri May 4 21:27:22 2018
Root user access: Disabled
Backend databases:
[SERVER1]:3307 Protocol: MariaDBBackend Name: SERVER1
[SERVER2]:3307 Protocol: MariaDBBackend Name: SERVER2
Total connections: 1
Currently connected: 1

Regards,
Venkat Beru

Generated at Thu Feb 08 04:09:52 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.