Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-16877

system_versioning_alter_history with galera corruption

    XMLWordPrintable

    Details

      Description

      ALTER TABLE is not propagating over Galera cluster for tables with SYSTEM VERSIONING enabled. Even if the other nodes in the cluster have system_versioning_alter_history set to "KEEP" globally, they're not receiving the table schema alterations.

      Attempting to modify data in one of these tables after the schema mismatch causes a whole world of problems, worse than any other time I've seen inconsistent schemas in the past.

      Every node in the Galera cluster shuts down (this is to be expected). However, afterwards, it was near impossible to bring the cluster back online.

      All but one node in the cluster lost their "uuid" info. Their grastate.dat contained the following contents:

      # GALERA saved state
      version: 2.1
      uuid:    00000000-0000-0000-0000-000000000000
      seqno:   -1
      safe_to_bootstrap: 0
      

      Attempting to modify the safe_to_bootstrap value to "1" and running galera_new_cluster or service mysql start would fail. Attempting to manually run mysqld_safe --wsrep_new_cluster would show the following on the console, followed by 100% CPU usage on all cores, while never actually getting anywhere, even after letting it sit several minutes (no disk activity at all btw)

      mysqld_safe starting mysqld daemon with databases from /var/lib/mysql
      

      Luckily, there was one single node in the cluster that still had proper "uuid" information and I was able to bootstrap the cluster from there, then one by one have every other node run a full SST to get their contents back (but this is extremely time consuming)

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              nikitamalyavin Nikita Malyavin
              Reporter:
              darkain Vincent Milum Jr
              Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated: