Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-24615

MariaDB 10.5.8 Galera node fails to start with WSREP: std::bad_alloc

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Duplicate
    • 10.5.8
    • N/A
    • Galera
    • None

    Description

      While running a MariaDB 10.5.8 Galera node and issuing `sudo systemctl restart mariadb` it failed to restart and run IST to join the cluster where it was a member of before the restart.

      Maybe galera.gcache got corrupted on shutdown?

      $ journalctl -fu mariadb
      ..
      systemd[1]: Starting MariaDB 10.5.8 database server...
      sh[52173]: WSREP: Recovered position 7f74fdea-cda3-11e9-ae2f-ab188b9a01f2:341781449
      mariadbd[52540]: 2021-01-17 20:57:30 0 [Note] /usr/sbin/mariadbd (mysqld 10.5.8-MariaDB-1:10.5.8+maria~bionic) starting as process 52540 ...
      mariadbd[52540]: 2021-01-17 20:57:30 0 [Note] WSREP: Loading provider /usr/lib/galera/libgalera_smm.so initial position: 7f74fdea-cda3-11e9-ae2f-ab188b9a01f2:341781449
      mariadbd[52540]: 2021-01-17 20:57:30 0 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
      mariadbd[52540]: 2021-01-17 20:57:30 0 [Note] WSREP: wsrep_load(): Galera 26.4.6(r1d8d67c8) by Codership Oy <info@codership.com> loaded successfully.
      mariadbd[52540]: 2021-01-17 20:57:30 0 [Note] WSREP: CRC-32C: using 64-bit x86 acceleration.
      mariadbd[52540]: 2021-01-17 20:57:30 0 [Note] WSREP: Found saved state: 7f74fdea-cda3-11e9-ae2f-ab188b9a01f2:-1, safe_to_bootstrap: 0
      mariadbd[52540]: 2021-01-17 20:57:30 0 [Note] WSREP: GCache DEBUG: opened preamble:
      mariadbd[52540]: Version: 2
      mariadbd[52540]: UUID: 7f74fdea-cda3-11e9-ae2f-ab188b9a01f2
      mariadbd[52540]: Seqno: -1 - -1
      mariadbd[52540]: Offset: -1
      mariadbd[52540]: Synced: 0
      mariadbd[52540]: 2021-01-17 20:57:30 0 [Note] WSREP: Recovering GCache ring buffer: version: 2, UUID: 7f74fdea-cda3-11e9-ae2f-ab188b9a01f2, offset: -1
      mariadbd[52540]: 2021-01-17 20:57:30 0 [Note] WSREP: GCache::RingBuffer initial scan...  0.0% (         0/5368709144 bytes) complete.
      mariadbd[52540]: 2021-01-17 20:57:36 0 [ERROR] WSREP: std::bad_alloc
      mariadbd[52540]: 2021-01-17 20:57:36 0 [ERROR] WSREP: Failed to create a new provider '/usr/lib/galera/libgalera_smm.so' with options 'gcache.size=5G; gcache.page_size=1G': Failed to initialize wsrep provider
      mariadbd[52540]: 2021-01-17 20:57:36 0 [ERROR] WSREP: Failed to load provider
      mariadbd[52540]: 2021-01-17 20:57:36 0 [ERROR] Aborting
      mariadbd[52540]: Warning: Memory not freed: 88
      systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
      systemd[1]: mariadb.service: Failed with result 'exit-code'.
      systemd[1]: Failed to start MariaDB 10.5.8 database server.
      

      I also came across this: https://stackoverflow.com/questions/64834855/mariadb-cant-start-wsrep-stdbad-alloc/ - so it seems this is affecting multiple users.

      Deleting the galera.gcache makes the server start, but then it runs a full SST and it is only a temporary solution, as whatever corrupted the galera.gcache file in the first place is still there.

       

      Attachments

        Issue Links

          Activity

            People

              Yurchenko Alexey
              otto Otto Kekäläinen
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.