Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Cannot Reproduce
-
2.4.2, 2.4.3
-
None
-
CentOS7, 8 core Xeon, 8GB RAM.
Maxscale 2.4.3 (updated today). MXS CDC Connector 2.4.3.
Description
Currently seeing Maxscale hang, fail and then restart via systemd watchdog whenever a CDC request is made. This is both via a local cdc.py request and from an external server running mxs_adapter.
The service had been running for around 6 hours without failure, handling circa 1k new rows into Columnstore every 10 seconds successfully.
Nothing changed in the config of mxs_adapter or maxscale between running state and failed state.
When failure occured, an update from 2.4.2 to 2.4.3 was performed today.
Additionally, following advice in https://jira.mariadb.org/browse/MXS-964 the router_options entry was added to the avro-router.
On failure Maxscale service outputs this...
Nov 06 16:35:05 maxscale1 systemd[1]: maxscale.service watchdog timeout (limit 1min)! |
Nov 06 16:35:05 maxscale1 maxscale[63821]: Fatal: MaxScale 2.4.3 received fatal signal 6. Commit ID: b33ef98f6c26b71e3cc9ea44b398776d51b35664 System name: Linux Release string: NAME="CentOS Linux" |
Nov 06 16:35:05 maxscale1 maxscale[63821]: |
/lib64/libc.so.6(epoll_wait+0x33): :? |
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker15poll_waiteventsEv+0xd0): maxutils/maxbase/src/worker.cc:795 |
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker3runEPNS_9SemaphoreE+0x53): maxutils/maxbase/src/worker.cc:559 |
/usr/bin/maxscale(main+0x2a76): server/core/gateway.cc:2265 |
/lib64/libc.so.6(__libc_start_main+0xf5): ??:? |
/usr/bin/maxscale(): ??:?
|
Nov 06 16:35:05 maxscale1 systemd[1]: maxscale.service: main process exited, code=killed, status=6/ABRT |
Nov 06 16:35:05 maxscale1 systemd[1]: Unit maxscale.service entered failed state. |
Nov 06 16:35:05 maxscale1 systemd[1]: maxscale.service failed. |
And here is the relevant section from Maxscale config file:
[replication-listener]
|
type=listener
|
service=replication
|
protocol=MariaDBClient
|
port=3311 |
 |
[replication]
|
type=service
|
router=binlogrouter
|
master_id=50 |
server_id=50 |
binlogdir=/home/binlogs
|
filestem=mysql-bin
|
user=username
|
password=password
|
 |
[avro-router]
|
type=service
|
router=avrorouter
|
source=replication
|
router_options=disable_sescmd_history=true |
match=/databasename\.tablename/
|
binlogdir=/home/binlogs
|
avrodir=/home/binlogs
|
filestem=mysql-bin
|
start_index=17 |
 |
[avro-listener]
|
type=listener
|
service=avro-router
|
protocol=CDC
|
port=4001 |