[MDEV-25090] Install libpmem-dev in the CI environments Created: 2021-03-09 Updated: 2021-08-04 Resolved: 2021-03-22 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Packaging, Storage Engine - InnoDB |
| Fix Version/s: | 10.6.0 |
| Type: | Task | Priority: | Blocker |
| Reporter: | Marko Mäkelä | Assignee: | Alexey Bychko (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | performance | ||
| Issue Links: |
|
||||||||||||||||
| Description |
|
Since MariaDB Server 10.5.1, a build configuration is supported but not enabled in our packages:
This will depend on libpmem-dev and introduce a small library dependency to the generated server package. I think that we should enable Persistent Memory support in MariaDB 10.6 on all applicable operating systems. Having the InnoDB redo log in a mount -o dax device should vastly reduce the transaction commit latency. This task would be a prerequisite for that. |
| Comments |
| Comment by Otto Kekäläinen [ 2021-03-10 ] |
|
marko Which version is required? There is libpmem1-dev and libpmem2-dev available. See "binaries" at https://tracker.debian.org/pkg/pmdk |
| Comment by Otto Kekäläinen [ 2021-03-10 ] |
|
Work-in-progress PR available at https://github.com/MariaDB/server/pull/1773/commits (see second commit). Old buildbot: http://buildbot.askmonty.org/buildbot/grid?branch=ok-10.6-uring-pmem&category=main&category=experimental Currently testing the latter commit on old buildbot does not work as it fails on `Unmet build dependencies: libpmem-dev` until dbart has added package libpmem-dev to builders that run the 10.6 branch. |
| Comment by Marko Mäkelä [ 2021-03-10 ] |
|
otto, thank you , I was not aware that libpmem2-dev exists. A quick look suggests that libpmem2 is an overkill for us; it seems to provide more high-level operations. We actually only use a tiny subset of what libpmem1 provides. It might be acceptable to just invoke the IA-32/AMD64 clflush instruction directly, and in |
| Comment by Otto Kekäläinen [ 2021-03-10 ] |
|
Note that libpmem-dev is only available on amd64, arm64 and ppc64el (not x86): https://packages.debian.org/search?searchon=names&keywords=libpmem-dev |
| Comment by Daniel Bartholomew [ 2021-03-12 ] |
|
libpmem-dev is also not available on Ubuntu Xenial, but that is going away next month so I don't think I will even bother making any releases for that |
| Comment by Daniel Bartholomew [ 2021-03-13 ] |
|
libpmem-dev is now installed on:
libpmem-devel is now installed on:
|
| Comment by Otto Kekäläinen [ 2021-03-14 ] |
|
Thanks dbart! Also the following builders would need the libpmem-dev: Package libpmem-dev is available on platforms amd64, arm64 and ppc64el. On Bionic the apt-madison tool when run by autobake-deb.sh sees libpmem-dev in the repositories so it should be installable (from bionic-updates, which is apparently enabled). |
| Comment by Marko Mäkelä [ 2021-03-15 ] |
|
We have a build failure on the following platforms due to missing libpmem-dev:
I confirmed that the following are using libpmem:
I also checked that kvm-deb-sid-x86 works, but without the library. |
| Comment by Daniel Bartholomew [ 2021-03-15 ] |
|
Yes, I'm starting with amd64, moving to other architectures today. Bionic-amd64 is done now. |
| Comment by Daniel Bartholomew [ 2021-03-16 ] |
|
libpmem-dev/devel is now installed on the following:
libpmem-dev/devel is not available on:
We don't have a valid subscription for rhel8-ppc64le, so this builder does not
|
| Comment by Daniel Bartholomew [ 2021-03-16 ] |
|
If I've missed a builder, let me know. Thanks. |
| Comment by Marko Mäkelä [ 2021-03-17 ] |
|
The Debian packaging change (setting WITH_PMEM=yes) that was contributed by otto as well as a change to replace WITH_PMEM:BOOL=OFF with WITH_PMEM:STRING=auto have now been pushed to 10.6 after a (mostly) successful bb-10.6-release staging run. |
| Comment by Daniel Bartholomew [ 2021-03-17 ] |
|
Installed libpmem onto the rhel8-install VM as rhel8 is our bintar builder for 10.6 and so the install VM needs the lib to be able to run tests. |
| Comment by Alexey Bychko (Inactive) [ 2021-03-18 ] |
|
marko probably we need to do the same for ES? |
| Comment by Daniel Bartholomew [ 2021-03-18 ] |
|
abychko Yes. We'll need libpmem-dev/devel installed on the ES builders. |
| Comment by Daniel Bartholomew [ 2021-03-18 ] |
|
Assigning to abychko to install libpmem-dev/devel into the Jenkins builders. After which this issue can be closed. |
| Comment by Alexey Bychko (Inactive) [ 2021-03-22 ] |
|
libpmem-dev doesn't exist for debian-9 https://packages.debian.org/search?searchon=names&keywords=+libpmem-dev |
| Comment by Alexey Bychko (Inactive) [ 2021-03-22 ] |
|
libpmem-dev is available from bionic and later https://packages.ubuntu.com/search?keywords=+libpmem-dev&searchon=names |
| Comment by Alexey Bychko (Inactive) [ 2021-03-22 ] |
|
added libpmem dev package to debian-10-buster, rhel-7, rhel-8, sles-12, sles-15, ubuntu-1804-lts, ubuntu-2004-lts. |