QPS drops by 10% for some sysbench tests

QPS drops by about 10% in MariaDB 10.4 for some, but not all, sysbench tests that are read-only and do either point or range when run at high concurrency (40 clients, 48-core server) with 8 tables and 10M rows per table.

A reproduction script is here. This runs of the the sysbench tests that I use and I haven't tried to reduce it. I ran it with 40 clients, 8 tables and 10M rows per table on a 48-core server.

The tests for which this occurs are:

• point-query - rQPS drops from 0.99 in 10.3.39 to 0.92 in 10.4.8 to 0.95 in 10.4.34 and is still 0.92 in 12.1.1
• random-points, range=10 - rQPS drops from 0.99 in 10.3.39 to 0.91 in 10.4.8 to 0.92 in 10.4.34. It is down to 0.58 in 12.1.1 from MDEV-37992
• range-covered-pk - rQPS drops from 0.97 in 10.3.39 to 0.81 in 10.4.8 to 0.89 in 10.4.34 and remains at 0.89 in 12.1.1
• range-covered-si - rQPS drops from 0.96 in 10.3.39 to 0.80 in 10.4.8 to 0.89 in 10.4.34 and is at 0.88 in 12.1.1
• range-notcovered-pk - rQPS drops from 0.96 in 10.3.39 to 0.82 in 10.4.8 to 0.89 in 10.4.34 and is at 0.86 in 12.1.1
• read-only, range=100 - rQPS drops from 0.98 in 10.3.39 to 0.89 in 10.4.8 to 0.92 in 10.4.34 and is at 0.90 in 12.1.1
• read-only, range=10 - rQPS drops from 0.96 in 10.3.39 to 0.87 in 10.4.8 to 0.92 in 10.4.34 and is at 0.89 in 12.1.1

There isn't a regression for read-only with range=10000 so I will guess that InnoDB isn't the problem here.

I don't see an increase in context switches per query. I do see an increase in CPU per query – see cpu/o here