Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-33454

release row locks for non-modified rows at XA PREPARE

Details

    Description

      After MDEV-26682 InnoDB releases gap locks on XA PREPARE. This is safe and doesn't impact correctness, while fixing some deadlocks in parallel replication.

      But it's not enough, row locks can still cause deadlocks. This task is about releasing row locks on XA PREPARE. InnoDB cannot release all row locks, of course, it'll cause incorrect results if a concurrent transaction will be able to see uncommitted changes.

      Still InnoDB can release row locks on all unmodified rows at XA PREPARE — this should be safe.

      Attachments

        Issue Links

          Activity

            Implementing this should make the locks after XA PREPARE fully compatible with what would happen if the server is restarted: InnoDB table IX locks and implicit record locks would be resurrected based on undo log records.

            Concurrently running transactions that are waiting for a lock may invoke lock_rec_convert_impl_to_expl() to create an explicit record lock object on behalf of the lock-owning transaction so that they can attaching their waiting lock request on the explicit record lock object. Explicit locks would be released by trx_t::release_locks() during commit or rollback.

            Any clustered index record whose DB_TRX_ID belongs to a transaction that is in active or XA PREPARE state will be implicitly locked by that transaction. On XA PREPARE, we can release explicit exclusive locks on records whose DB_TRX_ID does not match the current transaction identifier.

            When it comes to locks on secondary index records, we might argue that if a transaction modified any secondary indexes, it would also modify the clustered index of the table, and that is where any conflicting transactions are guaranteed to get a locking conflict. However, we must keep in mind that there could be covering-index locking reads, or someone could run at SERIALIZABLE transaction isolation level. Pretending that a secondary index record that was modified by a XA PREPARE transaction is not locked by that transaction would lead to ACID violation, because the result would not be repeatable and deterministic, but depend on when or whether XA COMMIT or XA ROLLBACK was executed.

            Hence, before releasing any explicit exclusive locks on secondary index records, we must determine if the record was modified by the current transaction: trx == lock_sec_rec_some_has_impl(trx, rec, index offsets). I do not think that it would be correct to take any shortcut if the PAGE_MAX_TRX_ID of the secondary index page happens to be trx->id, because that field is never rolled back. That is, the PAGE_MAX_TRX_ID may point to a transaction that will not modify any records in the page, if there was any partial rollback of that transaction, for example due to a duplicate key in another index.

            marko Marko Mäkelä added a comment - Implementing this should make the locks after XA PREPARE fully compatible with what would happen if the server is restarted: InnoDB table IX locks and implicit record locks would be resurrected based on undo log records. Concurrently running transactions that are waiting for a lock may invoke lock_rec_convert_impl_to_expl() to create an explicit record lock object on behalf of the lock-owning transaction so that they can attaching their waiting lock request on the explicit record lock object. Explicit locks would be released by trx_t::release_locks() during commit or rollback. Any clustered index record whose DB_TRX_ID belongs to a transaction that is in active or XA PREPARE state will be implicitly locked by that transaction. On XA PREPARE , we can release explicit exclusive locks on records whose DB_TRX_ID does not match the current transaction identifier. When it comes to locks on secondary index records, we might argue that if a transaction modified any secondary indexes, it would also modify the clustered index of the table, and that is where any conflicting transactions are guaranteed to get a locking conflict. However, we must keep in mind that there could be covering-index locking reads, or someone could run at SERIALIZABLE transaction isolation level. Pretending that a secondary index record that was modified by a XA PREPARE transaction is not locked by that transaction would lead to ACID violation, because the result would not be repeatable and deterministic, but depend on when or whether XA COMMIT or XA ROLLBACK was executed. Hence, before releasing any explicit exclusive locks on secondary index records, we must determine if the record was modified by the current transaction: trx == lock_sec_rec_some_has_impl(trx, rec, index offsets) . I do not think that it would be correct to take any shortcut if the PAGE_MAX_TRX_ID of the secondary index page happens to be trx->id , because that field is never rolled back. That is, the PAGE_MAX_TRX_ID may point to a transaction that will not modify any records in the page, if there was any partial rollback of that transaction, for example due to a duplicate key in another index.

            I assume this only affects statement based replication?

            monty Michael Widenius added a comment - I assume this only affects statement based replication?

            I think this will mainly affect statement based replication.

            But in the last ~2 years we've fixed a bunch of issues where this would've applied to row-based too. In all cases these issues were tracked down to very subtle and very difficult to repeat InnoDB locking bugs. Nobody can be sure there aren't any more.

            As far as statement based replication is concerned, this will help in a normal and easily reproducible use case, not caused by any bugs.

            serg Sergei Golubchik added a comment - I think this will mainly affect statement based replication. But in the last ~2 years we've fixed a bunch of issues where this would've applied to row-based too. In all cases these issues were tracked down to very subtle and very difficult to repeat InnoDB locking bugs. Nobody can be sure there aren't any more. As far as statement based replication is concerned, this will help in a normal and easily reproducible use case, not caused by any bugs.
            vlad.lesin Vladislav Lesin added a comment - https://github.com/MariaDB/server/pull/3060#pullrequestreview-1951814737

            This fix causes a regression where XA COMMIT and SELECT ... FOR UPDATE can hang.
            This is to be fixed in MDEV-34466 XA prepare don't release unmodified records in non-blocking mode
            and in MDEV-34690 lock_rec_unlock_unmodified() causes deadlock.

            monty Michael Widenius added a comment - This fix causes a regression where XA COMMIT and SELECT ... FOR UPDATE can hang. This is to be fixed in MDEV-34466 XA prepare don't release unmodified records in non-blocking mode and in MDEV-34690 lock_rec_unlock_unmodified() causes deadlock.

            People

              marko Marko Mäkelä
              serg Sergei Golubchik
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.