Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
2.5.13
-
None
-
VM Virtual Box -6.1 , RHEL 7.7
-
MXS-SPRINT-135
Description
One of our customers reported that at VM instance using OS RHEL 7, changing the CPU's from 2 to 4 and trying to login the GUI, causes the maxscale to crash.
I can locally able to reproduce the issue. Tried this at our latest release maxscale - 2.5.13
Here is the procedure which needs to be followed:
1. Run MaxScale on a 2 CPU VM.
1) Stop MaxScale
2) Set threads=auto in /etc/maxscale.cnf file.
3) Remove the /var/lib/maxscale/maxscale.cnf.d/maxscale.cnf file if it exists.
4) Run MaxScale
2. Check the MaxScale parameters and values.
$ maxctrl show maxscale
=> "threads" = 2
3. Change the value of the MaxScale parameter with the alter maxscale command.
e.g $ maxctrl alter maxscale passive false
At this time, the file /var/lib/maxscale/maxscale.cnf.d/maxscale.cnf is created.
This file records threads=2.
4. Stop the VM, change the VM's CPU to 4, then start the VM.
5. Check the MaxScale parameters and values.
$ grep threads /etc/maxscale.cnf
=> threads=auto
$ grep threads /var/lib/maxscale/maxscale.cnf.d/maxscale.cnf
=> threads=2
$ maxctrl show maxscale | grep threads
=> "threads" = 2
※ /var/lib/maxscale/maxscale.cnf.d/maxscale.cnf settings take precedence over /etc/maxscale.cnf
6. Log in to the MaxScale Web GUI admin page.
e.g. http://<IP>:8989
Upon login, the MaxScale process terminated with error messages and process core dump.
If MaxScale is run as a Linux service, no process core dump is generated and the process restarts.
7. Temporary workaround
1) $ vi /var/lib/maxscale/maxscale.cnf.d/maxscale.cnf
Change threads=2 to threads=auto or threads=4.
2) Restart MaxScale
3) $ maxctrl show maxscale | grep threads
=> "threads" = 4
4) If you log in to Web GUI admin page(e.g. http://<IP>:8989), it works normally.
Logs -
-
-
- Error in `/usr/bin/maxscale': free(): invalid next size (fast): 0x00000000016c5470 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x816b9)[0x7fc9b4c7f6b9]
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_Z24mxs_rworker_list_to_jsonPKc+0x9d)[0x7fc9b791b96d]
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x136378)[0x7fc9b790c378]
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNK8Resource4callERK11HttpRequest+0x14)[0x7fc9b790ec84]
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x13a6b2)[0x7fc9b79106b2]
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x13c210)[0x7fc9b7912210]
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker14handle_messageERNS_12MessageQueueERKNS_19MessageQueueMessageE+0x159)[0x7fc9b795fa39]
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase12MessageQueue18handle_poll_eventsEPNS_6WorkerEj+0x138)[0x7fc9b7962d48]
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker15poll_waiteventsEv+0x1be)[0x7fc9b7960a4e]
/usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker3runEPNS_9SemaphoreE+0x53)[0x7fc9b7960c63]
/usr/bin/maxscale(main+0x1f7a)[0x40939a]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7fc9b4c20545]
/usr/bin/maxscale[0x409ca2]
======= Memory map: ========
00400000-00415000 r-xp 00000000 fd:00 12950925 /usr/bin/maxscale
00614000-00615000 r--p 00014000 fd:00 12950925 /usr/bin/maxscale
00615000-00616000 rw-p 00015000 fd:00 12950925 /usr/bin/maxscale
00616000-00618000 rw-p 00000000 00:00 0
01683000-016c5000 rw-p 00000000 00:00 0 [heap]
016c5000-036e9000 rw-p 00000000 00:00 0 [heap]
- Error in `/usr/bin/maxscale': free(): invalid next size (fast): 0x00000000016c5470 ***
-