Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-38977

Galera - G-single with process pauses

    XMLWordPrintable

Details

    Description

      MariaDB's documentation claims that Galera Cluster offers transaction isolation "between Serializable and Repeatable Read". However, with process pauses (or possibly in healthy clusters--I'm still testing), MariaDB Galera Cluster occasionally allows G-single, even with transactions which perform no predicate operations. This violates both Snapshot Isolation and Repeatable Read.

      For example, take https://s3.amazonaws.com/jepsen.io/analyses/mariadb-galera-12.1.2/20260303T013806-g-single.zip, which contained two transactions, show in the attached screenshot.

      The top transaction appended `10` to key `665`, and committed successfully. Concurrently, the bottom transaction performed a read (not shown), read key `665` and saw `[2, 3, 5]` twice in a row, then appended `6` to key `665`. The problem is that every later read of key `665` observed a value like `[2, 3, 5, 10, 6, ...]`, which implies that the top transaction's append of `10` happened after the bottom transaction read `[2, 3, 5]`, but before the bottom transaction appended `6`. This is a clear violation of Snapshot Isolation, which (see https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/tr-95-51.pdf) that for any committed transaction, any modified object should not have been changed by a different transaction since it was first read. This history also violates Repeatable Read, which should prevent any cycle over single-object operations like these.

      These cycles could involve a single key or multiple keys, and occurred with a variety of faults, including process pauses, crashes, or network partitions. Cycles could involve many transactions: in one case, transaction T1 read a row, then five separate transactions apparently "snuck in" to update that row before T1 updated it.

      You can reproduce this with the Jepsen MariaDB test suite, at https://github.com/jepsen-io/mysql. Try commit df8c29675809444b730a6ea5da0d80e243e7fc70, and something like:

      lein run test-all --db maria --nodes n1,n2,n3 -w append --concurrency 6n --nemesis kill --time-limit 300 --test-count 300 --innodb-flush-log-at-trx-commit 1 --expected-consistency-model snapshot-isolation --isolation repeatable-read

      This will take a while, sadly--I haven't had time to dig into this too deeply. Nevertheless, it generally finds a few instances each day on my setup!

      Attachments

        1. g-single.svg
          5 kB
          Kyle Kingsbury
        2. g-single-3.svg
          2 kB
          Kyle Kingsbury

        Activity

          People

            seppo Seppo Jaakola
            aphyr Kyle Kingsbury
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.