Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Incomplete
-
22.08.3
-
production
Description
The maxscale process failed with the following backtrace.
This is the performance issue. The one maxscale is capable up to 5k connections. Over the 5k connection, the response is quite slow .. and after few minutes the maxscale crash.
2023-11-21 11:57:17 alert : MaxScale 22.08.4 received fatal signal 6. Commit ID: 9be47263ae2ff2c7bdf4c3d1334fb86a7de53b77 System name: Li
nux Release string: undefined
2023-11-21 11:57:17 alert : Statement currently being classified: none/unknown
2023-11-21 11:57:17 notice : For a more detailed stacktrace, install GDB and add 'debug=gdb-stacktrace' under the [maxscale] section.
nm: /lib/x86_64-linux-gnu/libc.so.6: no symbols
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11174): ??:?
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11278): ??:?
/usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN8maxscale13RoutingWorker20execute_concurrentlyERKSt8functionIFvvEE+0x52)
: maxutils/maxbase/include/maxbase/semaphore.hh:146
/usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN8maxscale12WorkerGlobalISt13unordered_mapIjmSt4hashIjESt8equal_toIjESaIS
t4pairIKjmEEEE6assignERKSA_0xdb): /usr/include/c+/10/bits/std_function.h:244
/usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(0x2ae40d): /usr/include/c+/10/bits/hashtable.h:421
/usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker14handle_messageERNS_12MessageQueueERKNS_19MessageQueueMessageE+0x79): maxutils/maxbase/src/worker.cc:738
/usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase17EventMessageQueue18handle_poll_eventsEPNS_6WorkerEjNS_8Pollable7ContextE+0x73): maxutils/maxbase/src/messagequeue.cc:220
/usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker14deliver_eventsEmNSt6chrono10time_pointINS1_3_V212steady_clockENS1_8durationIlSt5ratioILl1ELl1000000000EEEEEEPNS_8PollableEjNSA_7ContextE+0x8d): maxutils/maxbase/src/worker.cc:1043
/usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker15poll_waiteventsEv+0x344): maxutils/maxbase/src/worker.cc:1168
/usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker3runEPNS_9SemaphoreE+0x4f): maxutils/maxbase/src/worker.cc:822
/usr/bin/maxscale(main+0x26fd): server/core/gateway.cc:2243
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea): ??:0
/usr/bin/maxscale(_start+0x2a): ??:?
alert : Writing core dump.
The current maxscale configuration
[maxscale]
|
threads=auto
|
|
|
[db1]
|
type=server
|
address=db-1
|
port=3306
|
protocol=MariaDBBackend
|
|
|
[db2]
|
type=server
|
address=db-2
|
port=3306
|
protocol=MariaDBBackend
|
|
|
[db3]
|
type=server
|
address=db-3
|
port=3306
|
protocol=MariaDBBackend
|
|
|
[Read-Write-Service]
|
type=service
|
router=readwritesplit
|
servers=db1,db2,db3
|
user=maxscale
|
password=<redacted>
|
connection_keepalive=300s
|
master_accept_reads=true
|
max_slave_replication_lag=2s
|
filters=Hint
|
|
|
[Read-Write-Listener]
|
type=listener
|
service=Read-Write-Service
|
protocol=MariaDBClient
|
port=3306
|
Attachments
Activity
You can also try to install MaxScale 23.08 which has a built-in functionality for dumping stacktraces of all threads.
I'll close this as Incomplete due to no feedback with the latest version. If this occurs with the latest version again, please report back and we'll reopen it.
Signal 6 typically means that MaxScale was killed by the systemd watchdog.
You could try to replace the domain names (db-1 etc) with their real IPs, as domain name lookup may take a while and have been known to cause problems.
Also follow the instructions from the log
2023-11-21 11:57:17 notice : For a more detailed stacktrace, install GDB and add 'debug=gdb-stacktrace' under the [maxscale] section.
as that will provide a more informative stack-trace.