Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-27125

galera_new_cluster hangs with galera_recovery

    XMLWordPrintable

Details

    Description

      My cluster crashed and i cannot restart the first node with galera_new_cluster.
      The nodes are distributed geographically in France with medium latency of 12ms.
      It remains on /usr/bin/galera_recovery without doing anything.

      My cluster crashed and i cannot restart the first node with galera_new_cluster.

      The nodes are distributed geographically in France with medium latency of 12ms.

      It remains on /usr/bin/galera_recovery without doing anything.Htop screenshot

      This is my Galera config on all nodes (values may change in order to match node IP etc)...

      #
      # * Galera-related settings
      #
      # See the examples of server wsrep.cnf files in /usr/share/mysql
      # and read more at https://mariadb.com/kb/en/galera-cluster/
       
      [galera]
      # Mandatory settings
      wsrep_provider           = /usr/lib/libgalera_smm.so
      wsrep_on                 = ON
      wsrep_cluster_name       = "xxxxCLUSTER"
      wsrep_cluster_address    = "gcomm://xxx.xxx.xxx.100,xxx.xxx.xxx.101,xxx.xxx.xxx.102"
      binlog_format            = ROW
      default_storage_engine   = InnoDB
      #innodb_autoinc_lock_mode = 2
       
      wsrep_sst_method         = rsync
       
      # Allow server to accept connections on all interfaces.
      bind-address = 0.0.0.0
       
      #wsrep_sst_donor          = "DB2"
       
      # Optional settings
      wsrep_slave_threads      = 32
      wsrep_node_name          = "DB0"
      wsrep_node_address       = "xxx.xxx.xxx.100"
       
      #innodb_flush_log_at_trx_commit = 0
      

      Since i've been using Galera for years in his MariaDB 10.x version i know that safe_to_bootstrap has to be set 1 ecc...But nothing works with galera_new_cluster and i cannot figure out why.

      This is a typical log from wsrep_recovery

      2021-03-27  2:06:14 0 [Note] /usr/sbin/mysqld (mysqld 10.5.9-MariaDB-1:10.5.9+maria~buster) starting as process 8286 ...
      2021-03-27  2:06:14 0 [Note] InnoDB: Uses event mutexes
      2021-03-27  2:06:14 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2021-03-27  2:06:14 0 [Note] InnoDB: Number of pools: 1
      2021-03-27  2:06:14 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
      2021-03-27  2:06:14 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)
      2021-03-27  2:06:14 0 [Note] InnoDB: Using Linux native AIO
      2021-03-27  2:06:14 0 [Note] InnoDB: Initializing buffer pool, total size = 1073741824, chunk size = 134217728
      2021-03-27  2:06:14 0 [Note] InnoDB: Completed initialization of buffer pool
      2021-03-27  2:06:14 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
      2021-03-27  2:06:14 0 [Note] InnoDB: 128 rollback segments are active.
      2021-03-27  2:06:14 0 [Note] InnoDB: Creating shared tablespace for temporary tables
      2021-03-27  2:06:14 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
      2021-03-27  2:06:14 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
      2021-03-27  2:06:14 0 [Note] InnoDB: 10.5.9 started; log sequence number 57353507521; transaction id 2641101
      2021-03-27  2:06:14 0 [Warning] InnoDB: Skipping buffer pool dump/restore during wsrep recovery.
      2021-03-27  2:06:14 0 [Note] Plugin 'FEEDBACK' is disabled.
      2021-03-27  2:06:14 0 [Note] Recovering after a crash using tc.log
      2021-03-27  2:06:14 0 [Note] Starting crash recovery...
      2021-03-27  2:06:14 0 [Note] Crash recovery finished.
      2021-03-27  2:06:14 0 [Note] Server socket created on IP: '::'.
      2021-03-27  2:06:14 0 [Note] WSREP: Recovered position: 2edc32a3-8e96-11eb-9165-6281b552400d:7
      

      Attachments

        1. image-2021-11-26-10-16-07-313.png
          image-2021-11-26-10-16-07-313.png
          191 kB
        2. screenshot-1.png
          screenshot-1.png
          41 kB
        3. screenshot-2.png
          screenshot-2.png
          59 kB
        4. screenshot-3.png
          screenshot-3.png
          13 kB

        Issue Links

          Activity

            People

              thiru Thirunarayanan Balathandayuthapani
              yz s
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.