Uploaded image for project: 'MariaDB MaxScale'
  1. MariaDB MaxScale
  2. MXS-3774

Maxscale crash during xpand scale up

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 6.1.1
    • Fix Version/s: 6.1.2
    • Component/s: xpandmon
    • Labels:
      None

      Description

      During xpand scale up, Maxscale sometimes crashes.
      example log

      2021-09-17 11:13:56   notice : [xpandmon] Xpand-Monitor: Monitoring Xpand cluster state using node 10.11.202.88:3306.
      alert  : MaxScale 6.1.1 received fatal signal 11. Commit ID: 9b4abbc9bd86e6f9b6c3fc581214eb8583d959e4 System name: Linux Release string: CentOS Linux release 8.4.2105
       
       
      2021-09-17 11:13:56   alert  : (@@Xpand-Monitor:node-4); MaxScale 6.1.1 received fatal signal 11. Commit ID: 9b4abbc9bd86e6f9b6c3fc581214eb8583d959e4 System name: Linux Release string: CentOS Linux release 8.4.2105
      2021-09-17 11:13:56   alert  : (@@Xpand-Monitor:node-4); Statement currently being classified: none/unknown
      alert  :   /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNK8maxscale11WorkerLocalISt10shared_ptrINS_10SSLContextEENS_15CopyConstructorIS3_EEE15get_local_valueEv+0x2d): include/maxscale/indexedstorage.hh:90
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN6Server14post_configureEv+0x37a): include/maxscale/workerlocal.hh:219
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN6Server6createEPKcRKN8maxscale16ConfigParametersE+0x8c): server/core/server.cc:433
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN13ServerManager13create_serverEPKcRKN8maxscale16ConfigParametersE+0x5c): /usr/include/c++/8/bits/unique_ptr.h:150
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_Z30runtime_create_volatile_serverRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES6_i+0x16c): server/core/config_runtime.cc:1731
        /usr/lib64/maxscale/libxpandmon.so(_ZN12XpandMonitor13refresh_nodesEP8st_mysql+0x6b6): server/modules/monitor/xpandmon/xpandmonitor.cc:676
        /usr/lib64/maxscale/libxpandmon.so(_ZN12XpandMonitor4tickEv+0xc6): server/modules/monitor/xpandmon/xpandmonitor.cc:403
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN8maxscale13MonitorWorker12run_one_tickEv+0x15): server/core/monitor.cc:2046
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN8maxscale13MonitorWorker17call_run_one_tickEN7maxbase6Worker4Call8action_tE+0x56): server/core/monitor.cc:2029
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker4tickEv+0x128): maxutils/maxbase/include/maxbase/worker.hh:891
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase11WorkerTimer6handleEPNS_6WorkerEj+0x57): maxutils/maxbase/src/worker.cc:243
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker15poll_waiteventsEv+0x1e3): maxutils/maxbase/src/worker.cc:865
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker3runEPNS_9SemaphoreE+0x4f): maxutils/maxbase/src/worker.cc:562
        /lib64/libstdc++.so.6(+0xc2ba3): ??:?
        /lib64/libpthread.so.0(+0x814a): pthread_create.c:?
        /lib64/libc.so.6(clone+0x43): :?
       
       
      alert  : Writing core dump.
      2021-09-17 11:13:57   alert  : (@@Xpand-Monitor:node-4); 
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZNK8maxscale11WorkerLocalISt10shared_ptrINS_10SSLContextEENS_15CopyConstructorIS3_EEE15get_local_valueEv+0x2d): include/maxscale/indexedstorage.hh:90
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN6Server14post_configureEv+0x37a): include/maxscale/workerlocal.hh:219
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN6Server6createEPKcRKN8maxscale16ConfigParametersE+0x8c): server/core/server.cc:433
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN13ServerManager13create_serverEPKcRKN8maxscale16ConfigParametersE+0x5c): /usr/include/c++/8/bits/unique_ptr.h:150
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_Z30runtime_create_volatile_serverRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES6_i+0x16c): server/core/config_runtime.cc:1731
        /usr/lib64/maxscale/libxpandmon.so(_ZN12XpandMonitor13refresh_nodesEP8st_mysql+0x6b6): server/modules/monitor/xpandmon/xpandmonitor.cc:676
        /usr/lib64/maxscale/libxpandmon.so(_ZN12XpandMonitor4tickEv+0xc6): server/modules/monitor/xpandmon/xpandmonitor.cc:403
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN8maxscale13MonitorWorker12run_one_tickEv+0x15): server/core/monitor.cc:2046
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN8maxscale13MonitorWorker17call_run_one_tickEN7maxbase6Worker4Call8action_tE+0x56): server/core/monitor.cc:2029
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker4tickEv+0x128): maxutils/maxbase/include/maxbase/worker.hh:891
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase11WorkerTimer6handleEPNS_6WorkerEj+0x57): maxutils/maxbase/src/worker.cc:243
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker15poll_waiteventsEv+0x1e3): maxutils/maxbase/src/worker.cc:865
        /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker3runEPNS_9SemaphoreE+0x4f): maxutils/maxbase/src/worker.cc:562
        /lib64/libstdc++.so.6(+0xc2ba3): ??:?
        /lib64/libpthread.so.0(+0x814a): pthread_create.c:?
        /lib64/libc.so.6(clone+0x43): :?
      error sending signal child exited to process: os: process already finished
      failed to execute 'maxscale': process exited with code: 11
      

      "node-4" here is the new node that has just been added

      This seems to be a regression from 2.5.14 where this issue was not observed
      This was seen in SkySQL

        Attachments

          Activity

            People

            Assignee:
            johan.wikman Johan Wikman
            Reporter:
            petko.vasilev Petko Vasilev
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.