[MDEV-25774] profile build speed of the server code base Created: 2021-05-25  Updated: 2023-12-15

Status: Stalled
Project: MariaDB Server
Component/s: None
Fix Version/s: 10.4, 10.5, 10.6

Type: Task Priority: Major
Reporter: Nikita Malyavin Assignee: Nikita Malyavin
Resolution: Unresolved Votes: 1
Labels: None


 Description   

Changes to the most accessed headers require recompilation of 500 files avg, which takes about 3 mins on 6 cores. The time tends to grow YoY.
Several of build time improvements may be approached:

  • Header decomposition to improve files affected per line value.
  • Use of precompiled headers
  • Template optimization

This task is about the headers problem investigation.

  • Take a look how a time changes with precompiled headers enabled.
  • Work out the guidelines on enabling the precompiled headers generation for used system libraries.
  • And/or update build scripts to support precompiled headers generation.
  • If the build will still look too slow, analyze the template usages.

This tool may be helpful for the header analysis:
https://github.com/mikael-s-persson/templight

See also:

 Comments   
Comment by Nikita Malyavin [ 2022-01-11 ]

so far precompiling the headers reduces the build time about 5 times:
for ninja minbuild with PFS disabled it dropped from 2:30 to 0:30!

The major disadvantage for this method is that all the headers precompiled are stored in one precompiled file which is forcefully pre-included in every build.
Thus, by using PCH, one may end up with using symbols from the headers that are not actually included.

The alternative is of course c++20 submodules, which does the same, but smart.

The another alternative is GCC which handles .gch files differently: if the precompiled file is available, it is being used, or else the actual header is taken.

So for now I can try to use the alternative way for GCC, but the rest of compilers will remain screwed at either build speed or dependency integrity.

Comment by Nikita Malyavin [ 2022-01-29 ]

Some intermediate results are here:
https://github.com/MariaDB/server/commits/nikita/MDEV-25774

Comment by Aleksey Midenkov [ 2023-01-11 ]

Can it be cmake config option turned off by default?

It would be also good if configuration could choose between gcc/clang for compilation, between ld/lld for linker and whether to ccache or not.

Comment by Nikita Malyavin [ 2023-08-16 ]

GCC's gch approach has no benefit – it also can only use a single gch per invocation.

Only one precompiled header can be used in a particular compilation.

At the same time, c++20 modules support is released both in gcc and clang just not far from now.

GCC:
Module support can be enabled standard-independently, by -fmodules-ts
Moreover, -fmodule-header can help treating header as a module, looking forward to see what it's capable of.

On integration with cmake: https://www.kitware.com/import-cmake-c20-modules/

Comment by Julien Fritsch [ 2023-12-05 ]

Automated message:
----------------------------
Since this issue has not been updated since 6 weeks, it's time to move it back to Stalled.

Comment by JiraAutomate [ 2023-12-05 ]

Automated message:
----------------------------
Since this issue has not been updated since 6 weeks, it's time to move it back to Stalled.

Comment by Nikita Malyavin [ 2023-12-15 ]

https://gitlab.kitware.com/cmake/cmake/-/issues/25507

Generated at Thu Feb 08 09:40:15 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.