mysql_prlock_t and mysql_rwlock_t are performance hogs

Some time ago, Steve Shaw found out that the do_log_user() in the audit plugin was a significant performance problem. At the high level, we would seemingly have no problem when multiple threads are concurrently executing the following:

mysql_prlock_rdlock(&lock_operations);

/* ... */

mysql_prlock_unlock(&lock_operations);

The problem was that the implementation of both these operations is shortly acquiring a mutex. If the concurrent threads are executing in one of those operations at the same time, costly context switches to the kernel address space and back will be invoked.

Another problem with mysql_prlock_unlock() and mysql_rwlock_unlock() is that they execute a conditional branch while holding the latch. Each caller should already know whether they are supposed to release a shared or an exclusive latch, and invoke the appropriate primitive directly.

This performance problem in the audit plugin was fixed in MDEV-38126 by making use of a storage efficient implementation that is based on the Microsoft Windows SRWLOCK or some futex like system calls (MDEV-26476) when available.

When neither futex nor SRWLOCK are available, pthread_rwlock_t should not perform worse than mysql_prlock_t or mysql_rwlock_t. When I implemented the fix of MDEV-38126, such platforms (including IBM AIX and Apple macOS) were not a priority.

The following search will find a number of candidates to replace:

git grep 'rwlock_unlock|prlock_unlock'|sed -e 's/.*:[ \t]*//'|sort -u

One prominent component that is making use of these locks is MDL, even after the scalability fix that was implemented in MDEV-19749.