[MDEV-32530] Race condition in lock_wait_rpl_report() Created: 2023-10-20  Updated: 2023-11-08  Resolved: 2023-10-24

Status: Closed
Project: MariaDB Server
Component/s: Locking, Storage Engine - InnoDB
Affects Version/s: N/A
Fix Version/s: 10.6.16, 10.10.7, 10.11.6, 11.0.4, 11.1.3, 11.2.2

Type: Bug Priority: Blocker
Reporter: Marko Mäkelä Assignee: Vladislav Lesin
Resolution: Fixed Votes: 0
Labels: race, regression, rr-profile-analyzed
Environment:

GNU/Linux, rr 5.6.0


Issue Links:
Problem/Incident
causes MDEV-32728 safe_mutex: Found wrong usage of mute... Closed
is caused by MDEV-32096 Parallel replication lags because inn... Closed

 Description   

MDEV-32096 introduced an incorrect optimization to lock_wait_rpl_report(), which manifested itself as the following debug assertion failure:

lock/lock0lock.cc:2068: dberr_t lock_wait(que_thr_t*): Assertion `!wait_lock == !trx->lock.wait_lock' failed.

In the rr replay trace, the problem occurs while we are able to acquire exclusive lock_sys.latch without waiting. The following patch should fix this:

diff --git a/storage/innobase/lock/lock0lock.cc b/storage/innobase/lock/lock0lock.cc
index 31e02d2451a..df51ceb16d8 100644
--- a/storage/innobase/lock/lock0lock.cc
+++ b/storage/innobase/lock/lock0lock.cc
@@ -1812,8 +1812,14 @@ static lock_t *lock_wait_rpl_report(trx_t *trx)
   }
   else if (!wait_lock->is_waiting())
   {
-    wait_lock= nullptr;
-    goto func_exit;
+    wait_lock= trx->lock.wait_lock;
+    if (!wait_lock)
+      goto func_exit;
+    if (!wait_lock->is_waiting())
+    {
+      wait_lock= nullptr;
+      goto func_exit;
+    }
   }
 
   if (wait_lock->is_table())

While this function was about to enter lock_sys.wr_lock_try(), another thread had updated the lock while holding a shared lock_sys.latch. The lock_sys.latch happened to have been released by the time lock_sys.wr_lock_try() executed the std::atomic::compare_exchange_strong() on the lock word, so the exclusive lock_sys.latch was granted without waiting.

In lock_sys_t::cancel() there is a similar lock_sys.wr_lock_try() pattern on record locks (which can be modified by other threads), but it is correctly reloading trx->lock.wait_lock after acquiring the lock_sys.latch.



 Comments   
Comment by Vladislav Lesin [ 2023-10-24 ]

Looks good to me.

Generated at Thu Feb 08 10:32:03 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.