Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Won't Do
-
12.3.2
-
Related to performance
-
Q3/2026 Server Maintenance
Description
On commit and on XA PREPARE, lock_release() and lock_release_on_prepare() release a transaction's explicit locks under the shared lock_sys.rd_lock(), acquiring each per-cell hash-latch and per-table lock-mutex with a trylock because trx->mutex is held in the reverse of the normal latch order (so blocking acquisition could deadlock).
A single failed trylock marks the whole pass unsuccesfull, and after a fixed cap of 5 such passes, the code falls back to exclusive lock_sys.wr_lock() for the entire transaction.
Under concurrency these trylocks fail transiently, when the target cell or table-latch is momentarily held by another thread, so passes can fail even while the transaction is still steadily releasing locks.
The fixed cap therefore escalates transactions that are making progress in unlocking, not just stuck ones.
Once taken, the exclusive lock_sys.latch blocks every concurrent shared-latch acquirer in lock_rec_lock() and lock_table(), consistent with the heavy contention issues seen as a high-number of waiting reader threads in traces and flamegraphs.
The chance of hitting the cap rises both with the contention level and the number of latches a transaction must trylock per pass.
A proposed fix, for which the PR will be opened shortly after this ticket is created, replaces the fixed cap with a saturating stall counter: a pass that frees no lock increments it, a pass that frees at least one lock decrements it (floored at zero), and escalation happens after 5 net stalls.
A hard-ceiling of 100 total passes bounds the loop independently, for the non-converging cases where concurrent activity keeps adding locks.
A genuinely stuck transaction still escalates after 5 stalls as before, while the transactions that make progress can possibly survive more passes.
The *_try() functions report progress through an out-parameter computed under trx->mutex, so no latching should be done in the caller to decide on the retry pattern.
Depends on MDEV-40129, which should land first: it reduces transient trylock failures within a single pass via a bounded spin, so the stall counter here will genuinely measure lack of progress rather than momentary contention.
This proposed fix has not been yet performance tested, and is intended as additional performance fix on top of the aforementioned ticket.
Attachments
Issue Links
- is blocked by
-
MDEV-40129 lock_release_try() / lock_release_on_prepare_try() may fail due to transients
-
- Closed
-
- relates to
-
PERF-475 Loading...