Uploaded image for project: 'MariaDB MaxScale'
  1. MariaDB MaxScale
  2. MXS-3650

Maxscale crashes while logging to GUI

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.5.13
    • Fix Version/s: 2.5.14
    • Component/s: Core
    • Labels:
      None
    • Environment:
      VM Virtual Box -6.1 , RHEL 7.7
    • Sprint:
      MXS-SPRINT-135

      Description

      One of our customers reported that at VM instance using OS RHEL 7, changing the CPU's from 2 to 4 and trying to login the GUI, causes the maxscale to crash.

      I can locally able to reproduce the issue. Tried this at our latest release maxscale - 2.5.13
      Here is the procedure which needs to be followed:

      1. Run MaxScale on a 2 CPU VM.
      1) Stop MaxScale
      2) Set threads=auto in /etc/maxscale.cnf file.
      3) Remove the /var/lib/maxscale/maxscale.cnf.d/maxscale.cnf file if it exists.
      4) Run MaxScale

      2. Check the MaxScale parameters and values.
      $ maxctrl show maxscale
      => "threads" = 2

      3. Change the value of the MaxScale parameter with the alter maxscale command.
      e.g $ maxctrl alter maxscale passive false
      At this time, the file /var/lib/maxscale/maxscale.cnf.d/maxscale.cnf is created.
      This file records threads=2.

      4. Stop the VM, change the VM's CPU to 4, then start the VM.

      5. Check the MaxScale parameters and values.
      $ grep threads /etc/maxscale.cnf
      => threads=auto
      $ grep threads /var/lib/maxscale/maxscale.cnf.d/maxscale.cnf
      => threads=2
      $ maxctrl show maxscale | grep threads
      => "threads" = 2
      ※ /var/lib/maxscale/maxscale.cnf.d/maxscale.cnf settings take precedence over /etc/maxscale.cnf

      6. Log in to the MaxScale Web GUI admin page.
      e.g. http://<IP>:8989
      Upon login, the MaxScale process terminated with error messages and process core dump.
      If MaxScale is run as a Linux service, no process core dump is generated and the process restarts.

      7. Temporary workaround
      1) $ vi /var/lib/maxscale/maxscale.cnf.d/maxscale.cnf
      Change threads=2 to threads=auto or threads=4.
      2) Restart MaxScale
      3) $ maxctrl show maxscale | grep threads
      => "threads" = 4
      4) If you log in to Web GUI admin page(e.g. http://<IP>:8989), it works normally.

      Logs -

          • Error in `/usr/bin/maxscale': free(): invalid next size (fast): 0x00000000016c5470 ***
            ======= Backtrace: =========
            /lib64/libc.so.6(+0x816b9)[0x7fc9b4c7f6b9]
            /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_Z24mxs_rworker_list_to_jsonPKc+0x9d)[0x7fc9b791b96d]
            /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x136378)[0x7fc9b790c378]
            /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNK8Resource4callERK11HttpRequest+0x14)[0x7fc9b790ec84]
            /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x13a6b2)[0x7fc9b79106b2]
            /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x13c210)[0x7fc9b7912210]
            /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker14handle_messageERNS_12MessageQueueERKNS_19MessageQueueMessageE+0x159)[0x7fc9b795fa39]
            /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase12MessageQueue18handle_poll_eventsEPNS_6WorkerEj+0x138)[0x7fc9b7962d48]
            /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker15poll_waiteventsEv+0x1be)[0x7fc9b7960a4e]
            /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker3runEPNS_9SemaphoreE+0x53)[0x7fc9b7960c63]
            /usr/bin/maxscale(main+0x1f7a)[0x40939a]
            /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fc9b4c20545]
            /usr/bin/maxscale[0x409ca2]
            ======= Memory map: ========
            00400000-00415000 r-xp 00000000 fd:00 12950925 /usr/bin/maxscale
            00614000-00615000 r--p 00014000 fd:00 12950925 /usr/bin/maxscale
            00615000-00616000 rw-p 00015000 fd:00 12950925 /usr/bin/maxscale
            00616000-00618000 rw-p 00000000 00:00 0
            01683000-016c5000 rw-p 00000000 00:00 0 [heap]
            016c5000-036e9000 rw-p 00000000 00:00 0 [heap]

        Attachments

          Activity

            People

            Assignee:
            johan.wikman Johan Wikman
            Reporter:
            suresh.ramagiri@mariadb.com suresh ramagiri
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Git Integration