Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-16656

DROP DATABASE crashes Galera Cluster (all nodes)

Details

    Description

      Hi,

      I tried to delete a database from the cluster when the entire cluster suddenly crashes due to a DROP DATABASE operation. The crash appeared on all nodes at the same time as u can see in the logfiles.

      I added all logfiles and our my.cnf which is the same on all three nodes.

      If you have any advices for our configuration, please let me know.

      Best Regards,

      Aljoscha

      Attachments

        1. my.conf
          2 kB
        2. big-galera-03_crash.log
          8 kB
        3. big-galera-02_crash.log
          8 kB
        4. big-galera-01_crash.log
          6 kB

        Activity

          Horrorhorst Aljoscha Dembowsky created issue -
          Horrorhorst Aljoscha Dembowsky made changes -
          Field Original Value New Value
          Description Hi,

          I tried to delete a database from the cluster when the entire cluster suddenly crashes due to a DROP DATABASE operation. The crash appeared on all nodes at the same time as u can see in the logfiles.

          I added all logfiles and our my.cnf which is the same on all three nodes.

          If you have any advices for our configuration, please let me know, because this crash is a nogo for a production environment.


          Best Regards,

          Aljoscha
          Hi,

          I tried to delete a database from the cluster when the entire cluster suddenly crashes due to a DROP DATABASE operation. The crash appeared on all nodes at the same time as u can see in the logfiles.

          I added all logfiles and our my.cnf which is the same on all three nodes.

          If you have any advices for our configuration, please let me know.

          Best Regards,

          Aljoscha
          elenst Elena Stepanova made changes -
          Fix Version/s 10.1 [ 16100 ]
          Assignee Rasmus Johansson [ ratzpo ]
          danblack Daniel Black added a comment - - edited

          Do you have a list of table names in that database? In particular is there a table name that forms a extension of the database name, like bob and bobs_friends?

          danblack Daniel Black added a comment - - edited Do you have a list of table names in that database? In particular is there a table name that forms a extension of the database name, like bob and bobs_friends ?

          There are and were no tables in the database, because the project never started and so I wanted to clean up the leftovers.

          MariaDB [(none)]> use example-net-live;
          Database changed
          MariaDB [example-net-live]> show tables;
          Empty set (0.00 sec)
          

          Horrorhorst Aljoscha Dembowsky added a comment - There are and were no tables in the database, because the project never started and so I wanted to clean up the leftovers. MariaDB [(none)]> use example-net-live; Database changed MariaDB [example-net-live]> show tables; Empty set (0.00 sec)
          Horrorhorst Aljoscha Dembowsky made changes -
          Environment Ubuntu 16.04 LTS
          Intel Xeon E5-2687Wv4 - 12/24t - 3GHz /3.5GHz
          128GB DDR4 ECC 2400 MHz
          10 Gbit/s Ethernet
          450gb NVMe SSD
          MaxScale/HA-Proxy for load balancing
          Ubuntu 16.04 LTS
          Intel Xeon E5-2687Wv4 - 12/24t - 3GHz /3.5GHz
          128GB DDR4 ECC 2400 MHz
          10 Gbit/s Ethernet
          2x450gb NVMe SSD RAID 1
          MaxScale/HA-Proxy for load balancing
          danblack Daniel Black made changes -
          Labels assertion crash galera need_feedback assertion crash galera
          Horrorhorst Aljoscha Dembowsky made changes -
          Comment [ Hi,

          do you need more informations or can I do something in particular do support the process? The bug is reproducible, so everytime we try to delete the DB the whole cluster crashes...
          Deleting a DB is a basic function and shouldnt crash a cluster but if u cant help us we need to look for alternatives. ]
          Horrorhorst Aljoscha Dembowsky added a comment - - edited

          Another day another db...

          Today the whole cluster just locked the query execution while a script tried to drop another database. This time the database was filled with data but still no tables formed like the db name. I had to stop the loadbalancer and kill all threads followed by quick service restart on the nodes. Afterwards the applications went online again.

          DB-Name: example-com-dev
          Tables:

          wp_*
          user_*
          

          (* as wildcard)

          I will review the logs later and append them to the files above

          Horrorhorst Aljoscha Dembowsky added a comment - - edited Another day another db... Today the whole cluster just locked the query execution while a script tried to drop another database. This time the database was filled with data but still no tables formed like the db name. I had to stop the loadbalancer and kill all threads followed by quick service restart on the nodes. Afterwards the applications went online again. DB-Name: example-com-dev Tables: wp_* user_* (* as wildcard) I will review the logs later and append them to the files above

          sysprg Can you take a look at this during the next 10.1 sprint?

          ratzpo Rasmus Johansson (Inactive) added a comment - sysprg Can you take a look at this during the next 10.1 sprint?
          ratzpo Rasmus Johansson (Inactive) made changes -
          Assignee Rasmus Johansson [ ratzpo ] Julius Goryavsky [ sysprg ]
          sysprg Julius Goryavsky made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          jplindst Jan Lindström (Inactive) made changes -
          Fix Version/s 10.3.11 [ 23141 ]
          Fix Version/s 10.1.37 [ 23204 ]
          Fix Version/s 10.2.19 [ 23207 ]
          Fix Version/s 10.1 [ 16100 ]
          Resolution Fixed [ 1 ]
          Status In Progress [ 3 ] Closed [ 6 ]
          serg Sergei Golubchik made changes -
          Workflow MariaDB v3 [ 88175 ] MariaDB v4 [ 154609 ]

          People

            sysprg Julius Goryavsky
            Horrorhorst Aljoscha Dembowsky
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.