Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
2.5.8
-
None
Description
Customer hit the same fatal as https://jira.mariadb.org/browse/MXS-3425.
A crash appears to have happened when the persistent connections are being removed:
/lib64/libpthread.so.0(+0xdbcf): sem_wait.c:?
|
/lib64/libpthread.so.0(sem_wait+0x2b): ??:0
|
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN8maxscale13RoutingWorker20execute_concurrentlyESt8functionIFvvEE+0x62): maxutils/maxbase/include/maxbase/semaphore.hh:146
|
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNK6Server30cleanup_persistent_connectionsEv+0x31): /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/std_function.h:275
|
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNK6Server15json_attributesEv+0x184): include/maxscale/target.hh:316
|
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNK6Server12to_json_dataEPKc+0x73): server/core/server.cc:842
|
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN13ServerManager29server_to_json_data_relationsEPK6ServerPKc+0x1c8): server/core/servermanager.cc:223
|
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x1585a5): server/core/servermanager.cc:186
|
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN13ServerManager19server_list_to_jsonEPKc+0xb4): server/core/servermanager.cc:47
|
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x133cf8): server/core/resource.cc:606
|
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNK8Resource4callERK11HttpRequest+0x14): server/core/resource.cc:151
|
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x137eb2): server/core/resource.cc:1313
|
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x139a10): server/core/resource.cc:1343
|
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker14handle_messageERNS_12MessageQueueERKNS_19MessageQueueMessageE+0x159): maxutils/maxbase/src/worker.cc:490
|
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase12MessageQueue18handle_poll_eventsEPNS_6WorkerEj+0x138): maxutils/maxbase/src/messagequeue.cc:307
|
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker15poll_waiteventsEv+0x1be): maxutils/maxbase/src/worker.cc:879
|
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker3runEPNS_9SemaphoreE+0x53): maxutils/maxbase/src/worker.cc:574
|
/usr/bin/maxscale(main+0x1dbb): maxutils/maxbase/include/maxbase/log.h:168
|
/lib64/libc.so.6(__libc_start_main+0xf5): ??:?
|
/usr/bin/maxscale(): ??:?
|
Original log:
Mar 12 08:00:49 ibs-reportdbproxy-dc1 maxscale[15759]: (192725) (RW-Service-report) > Processing LOAD DATA LOCAL INFILE: 434809100 bytes sent.
|
Mar 12 08:01:00 ibs-reportdbproxy-dc1 systemd: maxscale.service watchdog timeout (limit 1min)!
|
Mar 12 08:01:00 ibs-reportdbproxy-dc1 maxscale[15759]: MaxScale 2.5.8 received fatal signal 6. Commit ID: 8a610d4dad58a710669ab137be91f3c9376c3e9a System name: Linux Release string: NAME="CentOS Linux"
|
Mar 12 08:01:00 ibs-reportdbproxy-dc1 maxscale[15759]: Statement currently being classified: none/unknown
|
Mar 12 08:01:01 ibs-reportdbproxy-dc1 systemd: Started Session 1007 of user root.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) (RW-Service-report) > Processing LOAD DATA LOCAL INFILE: 771521600 bytes sent.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) [readwritesplit] (RW-Service-report) Route query to master: ibs-reportdb-dc1 <
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) (RW-Service-report) > Processing LOAD DATA LOCAL INFILE: 771525700 bytes sent.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) [readwritesplit] (RW-Service-report) Route query to master: ibs-reportdb-dc1 <
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) (RW-Service-report) > Processing LOAD DATA LOCAL INFILE: 771529800 bytes sent.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) [readwritesplit] (RW-Service-report) Route query to master: ibs-reportdb-dc1 <
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) (RW-Service-report) > Processing LOAD DATA LOCAL INFILE: 771533900 bytes sent.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) [readwritesplit] (RW-Service-report) Route query to master: ibs-reportdb-dc1 <
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) (RW-Service-report) > Processing LOAD DATA LOCAL INFILE: 771538000 bytes sent.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) [readwritesplit] (RW-Service-report) Route query to master: ibs-reportdb-dc1 <
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) (RW-Service-report) > Processing LOAD DATA LOCAL INFILE: 771542100 bytes sent.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) [readwritesplit] (RW-Service-report) Route query to master: ibs-reportdb-dc1 <
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) (RW-Service-report) > Processing LOAD DATA LOCAL INFILE: 771546200 bytes sent.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192725) [readwritesplit] (RW-Service-report) Route query to master: ibs-reportdb-dc1 <
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192678) [readwritesplit] (RW-Service-report) Route query to master: ibs-reportdb-dc1 <
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192711) (RW-Service-report) > Processing LOAD DATA LOCAL INFILE: 707368900 bytes sent.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]:
|
#012 /lib64/libpthread.so.0(+0xdbcf): sem_wait.c:?
|
#012 /lib64/libpthread.so.0(sem_wait+0x2b): ??:0
|
#012 /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN8maxscale13RoutingWorker20execute_concurrentlyESt8functionIFvvEE+0x62): maxutils/maxbase/include/maxbase/semaphore.hh:146
|
#012 /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNK6Server30cleanup_persistent_connectionsEv+0x31): /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/std_function.h:275
|
#012 /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNK6Server15json_attributesEv+0x184): include/maxscale/target.hh:316
|
#012 /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNK6Server12to_json_dataEPKc+0x73): server/core/server.cc:842
|
#012 /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN13ServerManager29server_to_json_data_relationsEPK6ServerPKc+0x1c8): server/core/servermanager.cc:223
|
#012 /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x1585a5): server/core/servermanager.cc:186
|
#012 /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN13ServerManager19server_list_to_jsonEPKc+0xb4): server/core/servermanager.cc:47
|
#012 /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x133cf8): server/core/resource.cc:606
|
#012 /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNK8Resource4callERK11HttpRequest+0x14): server/core/resource.cc:151
|
#012 /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x137eb2): server/core/resource.cc:1313
|
#012 /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x139a10): server/core/resource.cc:1343
|
#012 /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker14handle_messageERNS_12MessageQueueERKNS_19MessageQueueMessageE+0x159): maxutils/maxbase/src/worker.cc:490
|
#012 /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase12MessageQueue18handle_poll_eventsEPNS_6WorkerEj+0x138): maxutils/maxbase/src/messagequeue.cc:307
|
#012 /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker15poll_waiteventsEv+0x1be): maxutils/maxbase/src/worker.cc:879
|
#012 /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker3runEPNS_9SemaphoreE+0x53): maxutils/maxbase/src/worker.cc:574
|
#012 /usr/bin/maxscale(main+0x1dbb): maxutils/maxbase/include/maxbase/log.h:168
|
#012 /lib64/libc.so.6(__libc_start_main+0xf5): ??:?
|
#012 /usr/bin/maxscale(): ??:?
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192711) [readwritesplit] (RW-Service-report) Route query to master: ibs-reportdb-dc1 <
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) [readwritesplit] (RW-Service-report) Route query to master: ibs-reportdb-dc1 <
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) (RW-Service-report) > Processing LOAD DATA LOCAL INFILE: 771550300 bytes sent.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) [readwritesplit] (RW-Service-report) Route query to master: ibs-reportdb-dc1 <
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192678) (RW-Service-report) > Processing LOAD DATA LOCAL INFILE: 908703500 bytes sent.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192678) [readwritesplit] (RW-Service-report) Route query to master: ibs-reportdb-dc1 <
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192725) (RW-Service-report) > Processing LOAD DATA LOCAL INFILE: 616529300 bytes sent.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192725) [readwritesplit] (RW-Service-report) Route query to master: ibs-reportdb-dc1 <
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) (RW-Service-report) > Processing LOAD DATA LOCAL INFILE: 771554400 bytes sent.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 maxscale[15759]: (192700) [readwritesplit] (RW-Service-report) Route query to master: ibs-reportdb-dc1 <
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 systemd: maxscale.service: main process exited, code=killed, status=6/ABRT
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 systemd: Unit maxscale.service entered failed state.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 systemd: maxscale.service failed.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 systemd: maxscale.service holdoff time over, scheduling restart.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 systemd: Stopped MariaDB MaxScale Database Proxy.
|
Mar 12 08:01:02 ibs-reportdbproxy-dc1 systemd: Starting MariaDB MaxScale Database Proxy...
|
Mar 12 08:01:03 ibs-reportdbproxy-dc1 maxscale[32441]: The 'log_debug' option has no effect in release mode.
|
Mar 12 08:01:03 ibs-reportdbproxy-dc1 maxscale[32441]: Found static and runtime configurations for [maxscale], ignoring static configuration. Move the runtime changes into the static configuration file and remove the generated file in '/var/lib/maxscale/maxscale.cnf.d' to remove this warning.
|
Mar 12 08:01:03 ibs-reportdbproxy-dc1 maxscale[32441]: Missing slashes (/) around a regular expression is deprecated: 'exclude=select,show'.
|
Mar 12 08:01:03 ibs-reportdbproxy-dc1 maxscale[32441]: Missing quotes (") around a quoted string is deprecated: 'separator=,'.
|
Mar 12 08:01:03 ibs-reportdbproxy-dc1 systemd: Started MariaDB MaxScale Database Proxy.
|
Mar 12 08:01:19 ibs-reportdbproxy-dc1 journal: Suppressed 592865 messages from /system.slice/maxscale.service
|
Mar 12 08:01:19 ibs-reportdbproxy-dc1 maxscale[32441]: (17) Low water mark hit for 'report'@'::ffff:10.1.12.107', accepting new data
|
|