Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-28322

Galera node crash

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: 10.6.5
    • Fix Version/s: N/A
    • Component/s: None
    • Labels:
    • Environment:
      Ubuntu 20.04 LTS

      Description

      We are running a 2 node + arbitrator galera cluster. We are using haproxy in front of the cluster to allow splitting reads and writes over different ports
      node 1 is dedicated write node, with node 2 being backup write.
      both nodes are available as read nodes.

      node 2 crashed with the following logging:

      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: This could be because you hit a bug. It is also possible that this binary
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: or one of the libraries it was linked against is corrupt, improperly built,
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: or misconfigured. This error can also be caused by malfunctioning hardware.
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: We will try our best to scrape up some info that will hopefully help
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: diagnose the problem, but since we have already crashed,
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: something is definitely wrong and this may fail.
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: Server version: 10.6.5-MariaDB-1:10.6.5+maria~focal-log
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: key_buffer_size=131072
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: read_buffer_size=131072
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: max_used_connections=37
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: max_threads=258
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: thread_count=47
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: It is possible that mysqld could use up to
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 568228 K  bytes of memory
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: Hope that's ok; if not, decrease some variables in the equation.
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: Thread pointer: 0x7fb180000c58
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: Attempting backtrace. You can use the following information to find out
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: where mysqld died. If you see no messages after this, something went
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: terribly wrong...
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: stack_bottom = 0x7fb2d01dcdc8 thread_stack 0x49000
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: Printing to addr2line failed
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: /usr/sbin/mariadbd(my_print_stacktrace+0x32)[0x562d30809442]
      Apr 15 04:02:43 node02.mariadb01 mariadbd[2360093]: /usr/sbin/mariadbd(handle_fatal_signal+0x485)[0x562d302bc6a5]
      Apr 15 04:02:44 node02.mariadb01 mariadbd[2360093]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x143c0)[0x7fb33a5603c0]
      Apr 15 04:02:44 node02.mariadb01 mariadbd[2360093]: /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7fb33a06403b]
      Apr 15 04:02:45 node02.mariadb01 mariadbd[2360093]: /lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7fb33a043859]
      Apr 15 04:02:45 node02.mariadb01 mariadbd[2360093]: /lib/x86_64-linux-gnu/libc.so.6(+0x22729)[0x7fb33a043729]
      Apr 15 04:02:46 node02.mariadb01 mariadbd[2360093]: /lib/x86_64-linux-gnu/libc.so.6(+0x34006)[0x7fb33a055006]
      

      The node needs a full resync to be able to rejoin the cluster

        Attachments

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            rtuk Rick Tuk
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.