Currently, lock_t is a union of table locks and record lock bitmaps. There can be multiple record lock bitmaps in a transaction. A record lock bitmap is specific to an index leaf page and a locking mode. It could be more efficient to use a single bitmap per page, with 4 bits per record heap number to represent all combinations of record and gap lock modes: {shared,exclusive,gap-read,gap-write}.

There are two types of gap locks (covering the open range of keys from the preceding record to the anchor record). A gap-read lock (acquired by UPDATE, DELETE, or a locking SELECT) prevents any subsequent INSERT to that range.

INSERT will acquire a gap-write or insert-intention lock (LOCK_INSERT_INTENTION). It conflicts with previously granted gap-read locks. Gap-read locks may be granted on the same lock if a gap-write lock has been granted to an active transaction.

An INSERT requires both a gap-write lock and an exclusive lock on the key that is being inserted. Any amount of gap-read locks may be held on the same gap by different transactions.

In lock_rec_has_to_wait(), gap-write locks do not conflict with gap-write locks of other transactions, even though lock_mode_compatible() would not hold.

A transaction can wait for at most one record. MDEV-24671 already simplified lock waits by introducing the function lock_wait(). If we pack 2 record heap numbers per byte (4 bits per heap number) in the record lock bitmap, there is no space for a further LOCK_WAIT flag. We might want to create a special type of bitmap for a waiting lock request, to avoid allocating a large bitmap (if the record heap number is 3000, the size of a straightforward bitmap would be 1500 bytes). We might store the attributes of the single-record lock request in the trx_t object. Maybe we could just preallocate a record lock bitmap object in trx_t, to be used for all waiting requests?