Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-27457

MariaDB Galera Cluster galera.cache file getting bigger than specified gcache.size

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Done
    • 10.3.22, 10.4.12
    • N/A
    • Galera
    • Kubernetes, 10.3.22-debian-10-r1 Bitnami Docker image

    Description

      We have a 3 node Galera Cluster running on Kubernetes, behind 2 HAProxy PODs configured so, all queries are executed on the first POD/node of the cluster if available, and the other 2 nodes, provide HA (HA Proxy backend backup nodes). In the config file, gcache.size is configured to 5 GB, and when a new node is deployed, galera.cache file is 5.1GB so, it seems to get that configuration correctly. However, what we are seeing is galera.cache growing in size up to 80 GBs or more for that first node of the cluster. As far as we know, this file should not increase in size. The problem is also reproduced when scaling the cluster down to one only node. It does not stop growing. The version deployed is 10.3.22 (10.3.22-debian-10-r1 Bitnami Docker image) These are the wsrep provider options specified in my.cnf:

      wsrep_on=ON
      wsrep_provider=/opt/bitnami/mariadb/lib/libgalera_smm.so
      wsrep_provider_options="gcache.size=5G"
      wsrep_sst_method=mariabackup
      wsrep_slave_threads=4
      wsrep_cluster_address=gcomm://
      wsrep_cluster_name=galera
      wsrep_sst_auth="root:"
      innodb-flush-log-at-trx-commit=2
      # MYISAM REPLICATION SUPPORT #
      wsrep_replicate_myisam=ON
      

      We've been dealing with this situation for some time now, we can remove the first node(POD) and then the galera.cache is recreated so, we free disk space. The first node syncs through IST with any of the other 2 nodes and the HAProxy points to a backup node meanwhile, and then back to the first node when recovered, with no downtime. However, we want to avoid to do this. We can't figure out why galera.cache file size increases.

      Attachments

        Activity

          People

            danblack Daniel Black
            AlexNS ALEJANDRO NÚÑEZ SÁNCHEZ
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.