Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-15748

Unable to stop mariadb.service or mysqld run with wsrep

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 10.2.14, 10.3.5
    • Fix Version/s: 10.3, 10.4
    • Component/s: Galera, Server, wsrep
    • Labels:
    • Environment:
      3 Master-Master MariDB cluster;CentOS 7.4

      Description

      Unable to stop mariadb.service or mysqld run with wsrep

      with either of systemctl mariadb.service or service mysql
      mysqld is not stopped and it's needed to remove manually mysld from the processlist
      in order to recover MariaDB cluster

      how to repeat :
      problem is observed in MariaDB cluster --galera 3 Mater-Master Nodes after recovering

      on all nodes was received the error , but mysqld it's found in the processlist

      # mysql -u root -p1
      ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (111)
      

      # ps aux | grep mysql
      mysql     6114  0.1  2.2 629180 45772 ?        Ssl  Mar30   5:00 /usr/sbin/mysqld --wsrep_start_position=dff6e041-1005-11e8-85c9-965f304f37bc:131618
      
      

       

      1. systemctl status mariadb.service
        ● mariadb.service - MariaDB 10.3.5 database server
        Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
        Drop-In: /etc/systemd/system/mariadb.service.d
        └─migrated-from-my.cnf-settings.conf
        Active: failed (Result: timeout) since Fri 2018-03-30 20:20:37 EEST; 2 days ago
        Docs: man:mysqld(8)
        https://mariadb.com/kb/en/library/systemd/
        Process: 6303 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=exited, status=1/FAILURE)
        Process: 6212 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= || VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ] && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=0/SUCCESS)
        Process: 6210 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
        Main PID: 6303 (code=exited, status=1/FAILURE)
        Status: "MariaDB server is down"
        CGroup: /system.slice/mariadb.service
        └─6114 /usr/sbin/mysqld --wsrep_start_position=dff6e041-1005-11e8-85c9-965f304f37bc:131618

      Mar 30 20:17:33 t4w6.xentio.lan systemd[1]: Starting MariaDB 10.3.5 database server...
      Mar 30 20:17:36 t4w6.xentio.lan sh[6212]: WSREP: Recovered position dff6e041-1005-11e8-85c9-965f304f37bc:131618
      Mar 30 20:17:36 t4w6.xentio.lan mysqld[6303]: 2018-03-30 20:17:36 0 [Note] /usr/sbin/mysqld (mysqld 10.3.5-MariaDB) starting as process 6303 ...
      Mar 30 20:17:37 t4w6.xentio.lan systemd[1]: mariadb.service: main process exited, code=exited, status=1/FAILURE
      Mar 30 20:19:07 t4w6.xentio.lan systemd[1]: mariadb.service stop-sigterm timed out. Skipping SIGKILL.
      Mar 30 20:20:37 t4w6.xentio.lan systemd[1]: mariadb.service stop-final-sigterm timed out. Skipping SIGKILL. Entering failed mode.
      Mar 30 20:20:37 t4w6.xentio.lan systemd[1]: Failed to start MariaDB 10.3.5 database server.
      Mar 30 20:20:37 t4w6.xentio.lan systemd[1]: Unit mariadb.service entered failed state.
      Mar 30 20:20:37 t4w6.xentio.lan systemd[1]: mariadb.service failed.

       
      Then try to stop mariadb.service or mysqld -- it's not returned error bud mysqld remained in the process list
      
      

      1. systemctl stop mariadb.service
        #

       
      
      

      1. service mysql stop
        Stopping mysql (via systemctl): [ OK ]
        #

       
       
       
       
       
       
      mysqld is not stopped 
      

      1. ps aux | grep mysql
        mysql 6114 0.1 2.2 629180 45772 ? Ssl Mar30 5:00 /usr/sbin/mysqld --wsrep_start_position=dff6e041-1005-11e8-85c9-965f304f37bc:131618

       
       
       
      
      

      [root@t4w3 ~]# ps aux | grep mysql
      mysql 966 0.0 2.2 629072 44992 ? Ssl Mar28 3:00 /usr/sbin/mysqld --wsrep_start_position=dff6e041-1005-11e8-85c9-965f304f37bc:131588
      \

      [root@t4w5 ~]# ps aux | grep mysql
      mysql 968 0.0 2.2 629072 46028 ? Ssl Mar28 2:56 /usr/sbin/mysqld --wsrep_start_position=00000000-0000-0000-0000-000000000000:-1

      [root@t4w6 ~]# ps aux | grep mysql
      mysql 982 0.1 2.4 629044 48412 ? Ssl Mar28 4:38 /usr/sbin/mysqld --wsrep_start_position=dff6e041-1005-11e8-85c9-965f304f37bc:131618

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                winstone Zdravelina Sokolovska
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: