Uploaded image for project: 'MariaDB MaxScale'
  1. MariaDB MaxScale
  2. MXS-3380

MaxScale crash loop with cache filter + Redis

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 2.5.6
    • 2.5.8
    • cache
    • None
    • CentOS Linux release 7.7.1908 (Core)
    • MXS-SPRINT-124

    Description

      After enabling the cache filter with Redis on an RDBA customer's server (datacheck), MaxScale crashes. Log file attached (max-redis_error.txt). Customer runs Redis 5.0.6.

      Another MaxScale server (test) shows some similar errors but no crash. Same Redis version. The difference between datacheck and test is that datacheck has two app servers hitting it via a load balancer. They are not using cluster mode.

      2021-01-20 22:28:37   error  : [storage_redis] Expected status message in the context of MULTI, but received a NIL.
      2021-01-20 22:28:37   error  : [storage_redis] Expected status message in the context of queued command, but received a STRING.
      2021-01-20 22:28:37   warning: [storage_redis] Unexpected redis redis return type (STATUS) received.
      2021-01-20 22:28:37   error  : [storage_redis] Expected status message in the context of MULTI, but received a STRING.
      2021-01-20 22:28:37   warning: [storage_redis] Unexpected redis redis return type (STATUS) received.
      2021-01-20 22:28:37   error  : [storage_redis] Expected status message 'OK' in the context of MULTI, but received 'QUEUED'.
      2021-01-20 22:28:37   error  : [storage_redis] Failed when reading response to MULTI: no error, 
      2021-01-20 22:28:37   error  : [cache] Could not store new cache value, deleting a possibly existing old value.
      2021-01-20 22:28:37   warning: [storage_redis] Unexpected redis return type (ARRAY) received.
      2021-01-20 22:28:37   error  : [cache] Could not delete cache item, the value may now be stale.
      2021-01-20 22:28:41   error  : [storage_redis] Expected status message in the context of MULTI, but received a NIL.
      2021-01-20 22:28:41   warning: [storage_redis] Unexpected redis redis return type (STATUS) received.
      2021-01-20 22:28:41   error  : [storage_redis] Expected status message 'OK' in the context of MULTI, but received 'QUEUED'.
      2021-01-20 22:28:41   error  : [storage_redis] Failed when reading response to MULTI: no error, 
      2021-01-20 22:28:41   error  : [cache] Could not store new cache value, deleting a possibly existing old value.
      2021-01-20 22:28:41   warning: [storage_redis] Unexpected redis return type (ARRAY) received.
      2021-01-20 22:28:41   error  : [cache] Could not delete cache item, the value may now be stale.
      2021-01-20 22:28:42   error  : [storage_redis] Expected status message in the context of MULTI, but received a STRING.
      2021-01-20 22:28:42   warning: [storage_redis] Unexpected redis redis return type (STATUS) received.
      2021-01-20 22:28:42   error  : [storage_redis] Expected status message 'OK' in the context of MULTI, but received 'QUEUED'.
      2021-01-20 22:28:42   error  : [storage_redis] Failed when reading response to MULTI: no error, 
      2021-01-20 22:28:42   error  : [cache] Could not store new cache value, deleting a possibly existing old value.
      2021-01-20 22:28:42   warning: [storage_redis] Unexpected redis return type (ARRAY) received.
      2021-01-20 22:28:42   error  : [cache] Could not delete cache item, the value may now be stale.
      2021-01-20 22:28:42   error  : [storage_redis] Expected status message in the context of MULTI, but received a STRING.
      2021-01-20 22:28:42   warning: [storage_redis] Unexpected redis redis return type (STATUS) received.
      

      2021-01-20 22:32:57   error  : [cache] Could not delete cache item, the value may now be stale.
      *** Error in `/usr/bin/maxscale': free(): invalid pointer: 0x00007f8c08000c92 ***
      ======= Backtrace: =========
      /lib64/libc.so.6(+0x81679)[0x7f8ebb6d7679]
      /usr/lib64/maxscale/libstorage_redis.so(redisBufferWrite+0x245)[0x7f8e683a26a5]
      /usr/lib64/maxscale/libstorage_redis.so(redisGetReply+0x83)[0x7f8e683a27a3]
      /usr/lib64/maxscale/libstorage_redis.so(redisvCommand+0x27)[0x7f8e683a2a37]
      /usr/lib64/maxscale/libstorage_redis.so(+0x5bfd)[0x7f8e68396bfd]
      /usr/lib64/maxscale/libstorage_redis.so(+0x9a6c)[0x7f8e6839aa6c]
      /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x182dcb)[0x7f8ebe3b0dcb]
      /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(_ZN7maxbase10ThreadPool6Thread4mainEv+0x155)[0x7f8ebe2cf425]
      /usr/lib64/maxscale/libmaxscale-common.so.1.0.0(+0x228a0f)[0x7f8ebe456a0f]
      /lib64/libpthread.so.0(+0x7e65)[0x7f8ebd448e65]
      /lib64/libc.so.6(clone+0x6d)[0x7f8ebb75488d]
      

      Attachments

        1. datacheck-maxscale.cnf
          2 kB
          Todd Coker
        2. max-redis_error.txt
          58 kB
          Todd Coker
        3. redis-screenshot.png
          176 kB
          Todd Coker

        Issue Links

          Activity

            People

              johan.wikman Johan Wikman
              Coker Todd Coker
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.