Uploaded image for project: 'MariaDB MaxScale'
  1. MariaDB MaxScale
  2. MXS-4563

when create cache filter MaxScale 23.02.1 received fatal signal 11

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Cannot Reproduce
    • None
    • N/A
    • cache
    • None

    Description

      MaxScale crashing after creating cache filter and restarting. But, everytime I try to login in GUI it crashes.
      also maxctrl list filters does the same thing
      root@... ~ # maxctrl list filters
      Error: socket hang up. If MaxScale is configured to use HTTPS, use the --secure option.

      MariaDB MaxScale  /var/log/maxscale/maxscale.log  Fri Mar 24 07:52:09 2023
      ----------------------------------------------------------------------------
      2023-03-24 07:52:09   notice : Module 'mariadbmon' loaded from '/usr/lib/x86_64-linux-gnu/maxscale/libmariadbmon.so'.
      2023-03-24 07:52:09   notice : Module 'masking' loaded from '/usr/lib/x86_64-linux-gnu/maxscale/libmasking.so'.
      2023-03-24 07:52:09   notice : Module 'readconnroute' loaded from '/usr/lib/x86_64-linux-gnu/maxscale/libreadconnroute.so'.
      2023-03-24 07:52:09   notice : Module 'readwritesplit' loaded from '/usr/lib/x86_64-linux-gnu/maxscale/libreadwritesplit.so'.
      2023-03-24 07:52:09   notice : Module 'cache' loaded from '/usr/lib/x86_64-linux-gnu/maxscale/libcache.so'.
      2023-03-24 07:52:09   notice : The logging of info messages has been enabled.
      2023-03-24 07:52:09   notice : Using up to 37.74GiB of memory for query classifier cache
      2023-03-24 07:52:09   notice : syslog logging is disabled.
      2023-03-24 07:52:09   notice : maxlog logging is enabled.
      2023-03-24 07:52:09   notice : Host: 'PapaBase' OS: Linux@5.10.0-18-amd64, #1 SMP Debian 5.10.140-1 (2022-09-02), x86_64 with 12 processor cores (12.00 available).
      2023-03-24 07:52:09   notice : Total main memory: 251.6GiB (251.6GiB usable).
      2023-03-24 07:52:09   notice : MariaDB MaxScale 23.02.1 started (Commit: 5de96ecc39da80a992fc8f362d22a030a844bef5)
      2023-03-24 07:52:09   notice : MaxScale is running in process 3384064
      2023-03-24 07:52:09   notice : Configuration file: /etc/maxscale.cnf
      2023-03-24 07:52:09   notice : Log directory: /var/log/maxscale
      2023-03-24 07:52:09   notice : Data directory: /var/lib/maxscale
      2023-03-24 07:52:09   notice : Module directory: /usr/lib/x86_64-linux-gnu/maxscale
      2023-03-24 07:52:09   notice : Service cache: /var/cache/maxscale
      2023-03-24 07:52:09   notice : Working directory: /var/log/maxscale
      2023-03-24 07:52:09   notice : Module 'qc_sqlite' loaded from '/usr/lib/x86_64-linux-gnu/maxscale/libqc_sqlite.so'.
      2023-03-24 07:52:09   info   : qc_sqlite loaded.
      2023-03-24 07:52:09   notice : Query classification results are cached and reused. Memory used per thread: 3.15GiB
      2023-03-24 07:52:09   notice : Password encryption key file '/var/lib/maxscale/.secrets' not found, using configured passwords as plaintext.
      2023-03-24 07:52:09   notice : The systemd watchdog is Enabled. Internal timeout = 30s
      2023-03-24 07:52:09   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 140525978312192.
      2023-03-24 07:52:09   info   : No 'auto_tune' parameters specified, no auto tuning will be performed.
      2023-03-24 07:52:09   notice : [masking] Masking filter [Obfuscates-card] created.
      2023-03-24 07:52:09   notice : [masking] The parameter 'treat_string_arg_as_field' is enabled for Obfuscates-card, disabling the query classifier cache.
      2023-03-24 07:52:09   notice : Query classifier cache disabled.
      2023-03-24 07:52:09   notice : [cache] Creating thread specific cache.
      2023-03-24 07:52:09   notice : [cache] Created cache per thread.
      2023-03-24 07:52:09   notice : Using HS256 for JWT signatures
      2023-03-24 07:52:09   notice : Started REST API on [0.0.0.0]:8989
      2023-03-24 07:52:09   notice : 'server1' sent version string '10.5.15-MariaDB-0+deb11u1'. Detected type: 'MariaDB', version: 10.5.15.
      2023-03-24 07:52:09   notice : Server 'server1' charset: utf8mb4_general_ci
      2023-03-24 07:52:09   notice : Starting a total of 2 services...
      2023-03-24 07:52:09   notice : (Read-Write-Listener); Listening for connections at [::]:4006
      2023-03-24 07:52:09   notice : Service 'Read-Write-Service' started (1/2)
      2023-03-24 07:52:09   notice : (Read-Only-Listener); Listening for connections at [::]:4008
      2023-03-24 07:52:09   notice : Service 'Read-Only-Service' started (2/2)
      2023-03-24 07:52:09   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 140525558560512.
      2023-03-24 07:52:09   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:52:09   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 140525541775104.
      2023-03-24 07:52:09   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:52:09   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 140525524989696.
      2023-03-24 07:52:09   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:52:09   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 140525508204288.
      2023-03-24 07:52:09   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:52:09   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 140525281732352.
      2023-03-24 07:52:09   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:52:09   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 140525264946944.
      2023-03-24 07:52:09   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:52:09   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 140525248161536.
      2023-03-24 07:52:09   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:52:09   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 140524954580736.
      2023-03-24 07:52:09   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:52:09   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 140524937795328.
      2023-03-24 07:52:09   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:52:09   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 140524921009920.
      2023-03-24 07:52:09   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:52:09   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 140524904224512.
      2023-03-24 07:52:09   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:52:09   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 140524677752576.
      2023-03-24 07:52:09   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:52:09   notice : MaxScale started with 12 worker threads.
      2023-03-24 07:52:10   notice : Read 10 user@host entries from 'server1' for service 'Read-Write-Service'.
      2023-03-24 07:52:10   notice : Read 10 user@host entries from 'server1' for service 'Read-Only-Service'.
      2023-03-24 07:52:11   info   : Variables have changed on 'server1', next check in 10 seconds: 'session_track_system_variables = autocommit,character_set_client,character_set_connection,character_set_results,time_zone', 'wait_timeout = 28800'
      2023-03-24 07:54:39   info   : Accept authentication from 'admin', using password. Request: /v1/filters
      MaxScale 23.02.1 received fatal signal 11. Commit ID: 5de96ecc39da80a992fc8f362d22a030a844bef5, System name: Linux, Release string: undefined, Thread: maxscale
       
      2023-03-24 07:54:39   notice : For a more detailed stacktrace, install GDB and add 'debug=gdb-stacktrace' under the [maxscale] section.
      nm: /lib/x86_64-linux-gnu/libc.so.6: no symbols
        /usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZNK8maxscale13RoutingWorker5indexEv+0): server/core/routingworker.cc:835
        /usr/lib/x86_64-linux-gnu/maxscale/libcache.so(_ZN7CachePT12worker_cacheEv+0x4d): server/modules/filter/cache/cachept.cc:209
        /usr/lib/x86_64-linux-gnu/maxscale/libcache.so(_ZNK7CachePT9all_rulesEv+0x18): server/modules/filter/cache/cachept.hh:86
        /usr/lib/x86_64-linux-gnu/maxscale/libcache.so(_ZNK5Cache11do_get_infoEj+0x5a): /usr/include/c++/10/bits/shared_ptr_base.h:1325
        /usr/lib/x86_64-linux-gnu/maxscale/libcache.so(_ZNK7CachePT8get_infoEj+0x1b): server/modules/filter/cache/cachept.cc:106
        /usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZNK9FilterDef9json_dataEPKc+0x14c): server/core/filter.cc:239
        /usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN9FilterDef19filter_list_to_jsonEPKc+0x5b): server/core/filter.cc:276
        /usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(+0x2aa909): server/core/resource.cc:745
        /usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZNK8Resource4callERK11HttpRequest+0x16): server/core/resource.cc:153
        /usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(+0x2b0fa5): server/core/resource.cc:1717
        /usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker14handle_messageERNS_12MessageQueueERKNS_19MessageQueueMessageE+0x79): maxutils/maxbase/src/worker.cc:778
        /usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase17EventMessageQueue18handle_poll_eventsEPNS_6WorkerEjNS_8Pollable7ContextE+0x73): maxutils/maxbase/src/messagequeue.cc:222
        /usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker14deliver_eventsEmNSt6chrono10time_pointINS1_3_V212steady_clockENS1_8durationIlSt5ratioILl1ELl1000000000EEEEEEPNS_8PollableEjNSA_7ContextE+0x8d): maxutils/maxbase/src/worker.cc:1100
        /usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker15poll_waiteventsEv+0x33c): maxutils/maxbase/src/worker.cc:1221
        /usr/lib/x86_64-linux-gnu/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase6Worker3runEPNS_9SemaphoreE+0x55): maxutils/maxbase/src/worker.cc:865
        /usr/bin/maxscale(main+0x289c): server/core/gateway.cc:2133
        /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea): ??:0
        /usr/bin/maxscale(_start+0x2a): ??:?
      Writing core dump.
       
       
      MariaDB MaxScale  /var/log/maxscale/maxscale.log  Fri Mar 24 07:54:41 2023
      ----------------------------------------------------------------------------
      2023-03-24 07:54:41   notice : Module 'mariadbmon' loaded from '/usr/lib/x86_64-linux-gnu/maxscale/libmariadbmon.so'.
      2023-03-24 07:54:41   notice : Module 'masking' loaded from '/usr/lib/x86_64-linux-gnu/maxscale/libmasking.so'.
      2023-03-24 07:54:41   notice : Module 'readconnroute' loaded from '/usr/lib/x86_64-linux-gnu/maxscale/libreadconnroute.so'.
      2023-03-24 07:54:41   notice : Module 'readwritesplit' loaded from '/usr/lib/x86_64-linux-gnu/maxscale/libreadwritesplit.so'.
      2023-03-24 07:54:41   notice : Module 'cache' loaded from '/usr/lib/x86_64-linux-gnu/maxscale/libcache.so'.
      2023-03-24 07:54:41   notice : The logging of info messages has been enabled.
      2023-03-24 07:54:41   notice : Using up to 37.74GiB of memory for query classifier cache
      2023-03-24 07:54:41   notice : syslog logging is disabled.
      2023-03-24 07:54:41   notice : maxlog logging is enabled.
      2023-03-24 07:54:41   notice : Host: 'PapaBase' OS: Linux@5.10.0-18-amd64, #1 SMP Debian 5.10.140-1 (2022-09-02), x86_64 with 12 processor cores (12.00 available).
      2023-03-24 07:54:41   notice : Total main memory: 251.6GiB (251.6GiB usable).
      2023-03-24 07:54:41   notice : MariaDB MaxScale 23.02.1 started (Commit: 5de96ecc39da80a992fc8f362d22a030a844bef5)
      2023-03-24 07:54:41   notice : MaxScale is running in process 3384649
      2023-03-24 07:54:41   notice : Configuration file: /etc/maxscale.cnf
      2023-03-24 07:54:41   notice : Log directory: /var/log/maxscale
      2023-03-24 07:54:41   notice : Data directory: /var/lib/maxscale
      2023-03-24 07:54:41   notice : Module directory: /usr/lib/x86_64-linux-gnu/maxscale
      2023-03-24 07:54:41   notice : Service cache: /var/cache/maxscale
      2023-03-24 07:54:41   notice : Working directory: /var/log/maxscale
      2023-03-24 07:54:41   notice : Module 'qc_sqlite' loaded from '/usr/lib/x86_64-linux-gnu/maxscale/libqc_sqlite.so'.
      2023-03-24 07:54:41   info   : qc_sqlite loaded.
      2023-03-24 07:54:41   notice : Query classification results are cached and reused. Memory used per thread: 3.15GiB
      2023-03-24 07:54:41   notice : Password encryption key file '/var/lib/maxscale/.secrets' not found, using configured passwords as plaintext.
      2023-03-24 07:54:41   notice : The systemd watchdog is Enabled. Internal timeout = 30s
      2023-03-24 07:54:41   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 139987977842176.
      2023-03-24 07:54:41   info   : No 'auto_tune' parameters specified, no auto tuning will be performed.
      2023-03-24 07:54:41   notice : [masking] Masking filter [Obfuscates-card] created.
      2023-03-24 07:54:41   notice : [masking] The parameter 'treat_string_arg_as_field' is enabled for Obfuscates-card, disabling the query classifier cache.
      2023-03-24 07:54:41   notice : Query classifier cache disabled.
      2023-03-24 07:54:41   notice : [cache] Creating thread specific cache.
      2023-03-24 07:54:41   notice : [cache] Created cache per thread.
      2023-03-24 07:54:41   notice : Using HS256 for JWT signatures
      2023-03-24 07:54:41   notice : Started REST API on [0.0.0.0]:8989
      2023-03-24 07:54:41   notice : 'server1' sent version string '10.5.15-MariaDB-0+deb11u1'. Detected type: 'MariaDB', version: 10.5.15.
      2023-03-24 07:54:41   notice : Server 'server1' charset: utf8mb4_general_ci
      2023-03-24 07:54:41   notice : Starting a total of 2 services...
      2023-03-24 07:54:41   notice : (Read-Write-Listener); Listening for connections at [::]:4006
      2023-03-24 07:54:41   notice : Service 'Read-Write-Service' started (1/2)
      2023-03-24 07:54:41   notice : (Read-Only-Listener); Listening for connections at [::]:4008
      2023-03-24 07:54:41   notice : Service 'Read-Only-Service' started (2/2)
      2023-03-24 07:54:41   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 139987898930944.
      2023-03-24 07:54:41   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:54:41   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 139987538405120.
      2023-03-24 07:54:41   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:54:41   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 139987521619712.
      2023-03-24 07:54:41   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:54:41   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 139987504834304.
      2023-03-24 07:54:41   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:54:41   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 139987211253504.
      2023-03-24 07:54:41   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:54:41   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 139987194468096.
      2023-03-24 07:54:41   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:54:41   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 139987177682688.
      2023-03-24 07:54:41   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:54:41   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 139987160897280.
      2023-03-24 07:54:41   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:54:41   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 139986934425344.
      2023-03-24 07:54:41   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:54:41   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 139986917639936.
      2023-03-24 07:54:41   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:54:41   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 139986900854528.
      2023-03-24 07:54:41   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:54:41   info   : [qc_sqlite] In-memory sqlite database successfully opened for thread 139986607273728.
      2023-03-24 07:54:41   info   : Epoll instance for listening sockets added to worker epoll instance.
      2023-03-24 07:54:41   notice : MaxScale started with 12 worker threads.
      2023-03-24 07:54:42   notice : Read 10 user@host entries from 'server1' for service 'Read-Only-Service'.
      2023-03-24 07:54:42   notice : Read 10 user@host entries from 'server1' for service 'Read-Write-Service'.
      2023-03-24 07:54:43   info   : Variables have changed on 'server1', next check in 10 seconds: 'session_track_system_variables = autocommit,character_set_client,character_set_connection,character_set_results,time_zone', 'wait_timeout = 28800'
      

      Attachments

        Activity

          kiskis igor kis added a comment -

          also if add
          soft_ttl=10
          hard_ttl=10
          to chache filter cfg file and try to restart got

          systemctl restart maxscale
          Job for maxscale.service failed because the control process exited with error code.
          See "systemctl status maxscale.service" and "journalctl -xe" for details.
          

          kiskis igor kis added a comment - also if add soft_ttl=10 hard_ttl=10 to chache filter cfg file and try to restart got systemctl restart maxscale Job for maxscale.service failed because the control process exited with error code. See "systemctl status maxscale.service" and "journalctl -xe" for details.
          johan.wikman Johan Wikman added a comment -

          Could you paste your MaxScale configuration. Please remove/obfuscate passwords.

          Johan

          johan.wikman Johan Wikman added a comment - Could you paste your MaxScale configuration. Please remove/obfuscate passwords. Johan
          kiskis igor kis added a comment -

          Im sorry.
          I've cleared main config from comments,
          then restarted and find errors
          2023-03-24 09:08:26 error : hard_ttl: Invalid duration: 10
          2023-03-24 09:08:26 error : soft_ttl: Invalid duration: 10
          changed to cache rule manually
          [cache]
          rules=/var/lib/maxscale/rules/cache.json
          type=filter
          module=cache
          storage=storage_inmemory
          cached_data=shared
          soft_ttl=5m
          hard_ttl=5m

          and now all work!

          kiskis igor kis added a comment - Im sorry. I've cleared main config from comments, then restarted and find errors 2023-03-24 09:08:26 error : hard_ttl: Invalid duration: 10 2023-03-24 09:08:26 error : soft_ttl: Invalid duration: 10 changed to cache rule manually [cache] rules=/var/lib/maxscale/rules/cache.json type=filter module=cache storage=storage_inmemory cached_data=shared soft_ttl=5m hard_ttl=5m and now all work!
          johan.wikman Johan Wikman added a comment -

          Good that you got it to work!

          I would be quite interested, though, in the original config and the exact steps need to cause MaxScale to crash, because no matter what, MaxScale should not crash.

          johan.wikman Johan Wikman added a comment - Good that you got it to work! I would be quite interested, though, in the original config and the exact steps need to cause MaxScale to crash, because no matter what, MaxScale should not crash.

          People

            johan.wikman Johan Wikman
            kiskis igor kis
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.