Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-9027

[PATCH] A customer reported strange locking behavior with TokuDB

Details

    • 10.0.23, 10.0.24, 10.0.26

    Description

      Customer reports the following:

      "We have uncovered a locking issue that doesn't "look right" to me. Here's how to reproduce it (I can send you a VM with all this setup if needed).

      1) Create and populate the sysbench sbtest1 table as a TokuDB table.
      2) Run a benchmark (any kind is fine...even a single-threaded one will do).
      3) Issue "create temporary table temp1 as select id from sbtest1;" or "insert into temp1 select id from sbtest1;" in a different session..
      4) Check the processlist in a 3rd session...you'll see that sbtest1 is locked from any insert/update/delete activity while the "select id from sbtest1" in step #3 runs.

      This only happens if log-bin is set to a value (ie. binlogging is turned on).

      This behavior happens whether the default transaction isolation is read-comitted, read-uncommitted, repeatable-read, or serializable....it should only happen with serializable. This behavior does NOT happen if the selected table is InnoDB, only if it's TokuDB. And again, this only happens if binlogging is on (which it must be for replication and disaster recovery)...if we turn binlogging off, the locks don't appear."

      The customer has a VM that duplicates issue if that is of help. I can be point of contact with them as required. - Jon

      Attachments

        Issue Links

          Activity

            It will be automatically merged when we get a new TokuDB.

            Unfortunately, TokuDB (as taken from the Percona Server source tarball) is currently unusable.
            When TokuDB will be fixed, I'll push it into the main tree and this bug will go away.

            serg Sergei Golubchik added a comment - It will be automatically merged when we get a new TokuDB. Unfortunately, TokuDB (as taken from the Percona Server source tarball) is currently unusable. When TokuDB will be fixed, I'll push it into the main tree and this bug will go away.
            philsweeney Phil Sweeney added a comment - - edited

            serg I can't see this suggested patch in the Percona repository so I don't think a TokuDB merge will help here. I believe this patch is just suggested by Rich for MariaDB.

            philsweeney Phil Sweeney added a comment - - edited serg I can't see this suggested patch in the Percona repository so I don't think a TokuDB merge will help here. I believe this patch is just suggested by Rich for MariaDB.

            Oh. I see. Thanks. Sorry for confusion.

            serg Sergei Golubchik added a comment - Oh. I see. Thanks. Sorry for confusion.

            This cannot be fixed with the richprohaska patch.

            Generally, If you do INSERT ... SELECT in a statement-based replication mode, you do want to prevent concurrent inserts into the source table. Otherwise a concurrent insert might finish before the long INSERT ... SELECT and it will be written to the binary log before the INSERT ... SELECT. That's why the server uses a strong read lock on the source table — TokuDB is not at fault here, it merely respects the default server behavior and leaves the lock level as is.

            You should use RBR if you want INSERT ... SELECT to allow concurrent inserts and also want your slaves to be consistent with the master.

            The suggested patch weakens the table lock, concurrent inserts become possible, and that breaks replication.

            InnoDB allows concurrent inserts, indeed, but only into the “not yet selected” part of the source table, so INSERT ... SELECT will see the inserted row, as if it was inserted before INSERT ... SELECT has started. InnoDB uses gap locks to prevent concurrent inserts into the “already selected” part of the source table.

            TokuDB needs to change its internal locking to be able to use weak read locks for INSERT ... SELECT without breaking the statement based replication.

            serg Sergei Golubchik added a comment - This cannot be fixed with the richprohaska patch. Generally, If you do INSERT ... SELECT in a statement-based replication mode, you do want to prevent concurrent inserts into the source table. Otherwise a concurrent insert might finish before the long INSERT ... SELECT and it will be written to the binary log before the INSERT ... SELECT . That's why the server uses a strong read lock on the source table — TokuDB is not at fault here, it merely respects the default server behavior and leaves the lock level as is. You should use RBR if you want INSERT ... SELECT to allow concurrent inserts and also want your slaves to be consistent with the master. The suggested patch weakens the table lock, concurrent inserts become possible, and that breaks replication. InnoDB allows concurrent inserts, indeed, but only into the “not yet selected” part of the source table, so INSERT ... SELECT will see the inserted row, as if it was inserted before INSERT ... SELECT has started. InnoDB uses gap locks to prevent concurrent inserts into the “already selected” part of the source table. TokuDB needs to change its internal locking to be able to use weak read locks for INSERT ... SELECT without breaking the statement based replication.

            I have also been affected by this bug on a few tables with INSERT SELECT and CREATE SELECT
            As soon as we converted the tables to InnoDB the issue stopped.

            marostegui Manuel Arostegui added a comment - I have also been affected by this bug on a few tables with INSERT SELECT and CREATE SELECT As soon as we converted the tables to InnoDB the issue stopped.

            People

              serg Sergei Golubchik
              jonathan.day@mariadb.com Jonathan Day (Inactive)
              Votes:
              2 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.