monty has done some large changes on structures in https://github.com/MariaDB/server/compare/bb-10.6-monty

Changes to improve the size of structures (https://github.com/MariaDB/server/commit/5907630b2b1e5a5dbff5201a9666ec56945af5a8) may have some unintended consequences.

While the thread local size pools is limited, saving 3% of those shouldn't cost performance by breaking the impacts of cache locality.

The code improvements to improve the initialization are less likely to have problems and for the most part seems to be improving initialization by facilitating reduction not broken by function calls.

Changes to structures however can have impacts:

• L1 cache size is limited
• With threads >> hardware CPUs this become more important as older (or just the same association) L1 caches are flushed/discarded
• Having structure similar members adjacent makes them likely to be on the same cache line and therefore can be loaded into cache in one go, and less cache lines are used.
• Without values in cache memory, the branch predictor circuitry is impeded from predicting branches or performing pre-fetching of data resulting in stalls.

An impact is possible to occur as such:

• query starts being parsed
• with a more space structure a higher number of L1 caches is used to preform initialization (or worse, TLB misses)
• kernel schedules another thread on the same CPU
• 2nd query parsing begins with same consequence
• many more reschedulings
• kernel schedules query 1 again
• memory needs to be re-read again as caches where flushed

Recommend for testing:

• large number of threads (to point of CPU saturation)
• no use of prepared statements (unfairly aids parsing)
• stats measurement - perf stat -B -e cache-references,cache-misses,cycles,instructions,branches,faults,migrations before/arfter
• function measurement perf record -e cache-misses,branches,faults to identify the functions where cache-misses are occurring and based on the code, which ones.

While these structures are hardy ideal in their current structure, they do have some common aspects like foreign keys, parsing, locking bits co-located (by virtue of coder readability more than intentional effects). By performance measuring the L1 cache misses we'll see if this is resulting in a regression, and if other aspects of cache locality can be improved.

I sincerely hope getting the minimal set of changes here is easy.