Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-32162

debian-start attempts to use DB before WSREP/Galera complete

Details

    • Bug
    • Status: Open (View Workflow)
    • Minor
    • Resolution: Unresolved
    • 10.6.15
    • 10.6
    • Platform Debian
    • None
    • Ubuntu Jammy. Standard community packages

    Description

      I am working on upgrading our platform from 10.4 to 10.6, using Galera.

      I can see how the server is started, and the systemd service unit file declares to start various pre steps (to handle galera variables), launches the main mariadbd process, and then performs post start operations - i.e. start debian-start.

      I think, that mariadbd is forking its process, and hence returning to systemd before WSREP is willing to perform queries. However, debian-start is launching, and attempting to ensure that the root accounts are secure.

      I am using `galera_new_cluster` to bring up the first node of the cluster. I am yet to get as far as joining a second node and seeing if the issue exists there.

      I have only started looking at 10.6.15, so unsure as to how early in the version this problem exists. I can't recall having the issue in 10.4

      The mariadb logs entries, are separated from syslog.

      Sep 13 13:40:59 medx-dev systemd[1]: Starting MariaDB 10.6.15 database server...
      Sep 13 13:41:00 medx-dev sh[5323]: WSREP: Recovered position f428bd1b-4d80-11ee-95e9-47fa73e2fe24:2968
      Sep 13 13:41:15 medx-dev systemd[1]: Started MariaDB 10.6.15 database server.
      Sep 13 13:41:15 medx-dev /etc/mysql/debian-start[5682]: Upgrading MySQL tables if necessary.
      Sep 13 13:41:15 medx-dev /etc/mysql/debian-start[5686]: /usr/bin/mariadb-upgrade: the '--basedir' option is always ignored
      Sep 13 13:41:15 medx-dev /etc/mysql/debian-start[5693]: Checking for insecure root accounts.
      Sep 13 13:41:15 medx-dev debian-start[5679]: ERROR 1047 (08S01) at line 1: WSREP has not yet prepared node for application use
      

      Sep 13 13:41:15 medx-dev mariadbd[5571]: 2023-09-13 13:41:15 0 [Note] /usr/sbin/mariadbd: ready for connections.
      Sep 13 13:41:15 medx-dev mariadbd[5571]: Version: '10.6.15-MariaDB-1:10.6.15+maria~ubu1804-log'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
      Sep 13 13:41:18 medx-dev mariadbd[5571]: 2023-09-13 13:41:18 1 [Note] WSREP: Lowest cert index boundary for CC from group: 2969
      Sep 13 13:41:18 medx-dev mariadbd[5571]: 2023-09-13 13:41:18 1 [Note] WSREP: Min available from gcache for CC from group: 1
      Sep 13 13:41:18 medx-dev mariadbd[5571]: 2023-09-13 13:41:18 1 [Note] WSREP: Server medx-dev synced with group
      Sep 13 13:41:18 medx-dev mariadbd[5571]: 2023-09-13 13:41:18 1 [Note] WSREP: Server status change joined -> synced
      Sep 13 13:41:18 medx-dev mariadbd[5571]: 2023-09-13 13:41:18 1 [Note] WSREP: Synchronized with group, ready for connections
      

      Attachments

        Activity

          I have little bit learn about Galera as I'm not very familiar with it. As this could be the one solution I've just wondering when one does not have Galera available.

          illuusio Tuukka Pasanen added a comment - I have little bit learn about Galera as I'm not very familiar with it. As this could be the one solution I've just wondering when one does not have Galera available.

          Yes, that possibly is one solution. However, I would imagine there is probably more systems without Galera configured, than there are with Galera.

          There is also the question of understanding what has changed in the server process to break it since 10.4. I am fairly sure that this used to not be a problem. However, it might have been occuring, but just not logging as severely.

          Hopefully someone else from MariaDB can fill you in with the Galera info - as it becomes quite complex regarding how ready the server might be to handle requests.

          brendon Brendon Abbott added a comment - Yes, that possibly is one solution. However, I would imagine there is probably more systems without Galera configured, than there are with Galera. There is also the question of understanding what has changed in the server process to break it since 10.4. I am fairly sure that this used to not be a problem. However, it might have been occuring, but just not logging as severely. Hopefully someone else from MariaDB can fill you in with the Galera info - as it becomes quite complex regarding how ready the server might be to handle requests.
          illuusio Tuukka Pasanen added a comment - - edited

          Yes there is need little bit of source reading why it's printing that error on what commit has made the change

          illuusio Tuukka Pasanen added a comment - - edited Yes there is need little bit of source reading why it's printing that error on what commit has made the change
          illuusio Tuukka Pasanen added a comment - - edited

          Ok I've done little bit homework (not enough though) with this and as I haven't setup galera cluster cloud you provuide what what does:

          mariadb --batch -e "SHOW STATUS LIKE 'wsrep_local_state';"
          

          show before/after it's in working state

          illuusio Tuukka Pasanen added a comment - - edited Ok I've done little bit homework (not enough though) with this and as I haven't setup galera cluster cloud you provuide what what does: mariadb --batch -e "SHOW STATUS LIKE 'wsrep_local_state';" show before/after it's in working state

          The information requested is shown in the bash output, in a previous comment.

          brendon Brendon Abbott added a comment - The information requested is shown in the bash output, in a previous comment.

          People

            Unassigned Unassigned
            brendon Brendon Abbott
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.