let's only do it in 10.5 and 10.6, as older bintars are already huge

This is not yet that much of a problem with Debian "bullseye" and Ubuntu "Focal" LTS, which switched to ncurses v6 as default, but still provide ncurses v5 shared libraries for backwards compatibility.

On RHEL 8 / CentOS 8 on the other hand, ncurses v6 is now the default, and there are no backwards compatibility packages (at least not in the default repositories AFAICT) that would provide the v5 shared libraries.

So on RHEL 8 / CentOS 8 it is not possible to use the command line client from the binary tarballs unless you manually copy over libncurses.5.so and libtinfo.5.so from a RHEL7/CentOS7 installation manually.

what we have to do if some platforms provide only dynamic libraries? compile custom ones or link dynamically?

checked ES 10.5, built AIO statically. preparing a patch

it will take time because different systems ship different libraries.
for example ubuntu 20.04 has both static and dynamic libs for aio.
in the same time it has only libncurses.a as static, all others like libtinfo are dynamic.
the most straightforward way is to prefer static libs if available and use dynamic if static is unavailable. but it will not solve the issue.

what are the options:

• recompile dev packages if needed to have static libraries (all platforms)
• bundle sources to the code tree and compile server without using system libs (for bintars only)
• build single custom image with all required libs installed to produce "universal" bintar if it's possible at all ( OpenSSL license? )
• use static libs if available, otherwise dynamic. probably it will not solve the issue, it will just increase bintar size

tested static libtinfo on centos-7 because of request from hholzgra

1 - it refuses to link with libtinfo.a from distribution, probably because of

  SET_TARGET_PROPERTIES(mariadb PROPERTIES ENABLE_EXPORTS TRUE)

and asking to recompile ncurses with -fPIC.

2 - client links just fine with recompiled ncurses, but it leads to following:
2.a - ldd mariadb shows much more linked libraries, because there is no dynamic libtinfo.so and client should resolve all its external links on its own. it's expected, but it will create 2-3x more dependencies similar to one we're trying to solve.
2.b - there are linker errors now (undefined references) in the rest of the project (a lot of them actually). they need to be investigated and fixed one-by one. not a mammoth work, but it will take some time.

seems aio and compression libraries may be linked statically without any negative side-effects

a macro + static AIO is here https://github.com/MariaDB/server/commit/4b622011127825f541eaceb3ee1c5ae25d022f42
serg is it acceptable solution in general?

I don't understand what you're doing. You can just build static versions of these libraries and put them where cmake can find them. It doesn't need any changes in cmake files.

unfortunately build ignores available static libraries.
it's a ticket about static tarballs, not about building custom libraries (but it may include it).
what I'm getting wrong?

as I realized after short discussion with Sergei - this ticket is not about static tarballs.
it's about compiling statical libs from description and installing them on BB virtual machines even. so, it's completely another task.
anyway I compiled libaio, lz4, ncurses, xz for centos-7 using /ds1819/vms-customizations/* as examples.
testing now

Can you also check mariadb-test isn't dynamicly linked against PCRE?

no, that's unrelated. our bintars link with PCRE statically. The user clearly said that he installed mariadb-server-10.5 from out APT repo. There must be something strange with his prce2 library, looks like it's a non-debian build.

Updated the kvm-bintar-centos74-amd64 builder with the packages alexey prepared for installation under $HOME/local and ran a test build in buildbot.

The static libaio did appear to work with `ldd mariadbd` no longer listing libaio as a dynamically loaded lib.

But liblzma and liblz4 still appear to be dynamically linked in the `mariadbd` binary. In the build log there is output that says "Could NOT find LZ4", but it does at one point say it found lzma.h, but I'm guessing in that case it found the wrong one. The static ncurses also didn't work as the `mariadb` binary still reports it is dynamically linked to the system libncurses lib.

Next I can try installing the libs under /usr/local instead of them being under $HOME/local and see if that helps.

I haven't been able to build a static ncurses that actually works.

whoever installed liblz4.a on the centos7 bintar builder did it incorrectly. Now 10.7 bintar builds fail with

/usr/bin/ld: /home/buildbot/local/lib/liblz4.a(lz4.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC

For MariaDB even static libraries have to be build with -fPIC, because they're linked into shared libraries.

The lz4 tarball I received from alexey didn't have a README, so I'm not sure how it was compiled. I've compiled a new static lz4 and uploaded it (and an associated README file) to hasky:/ds1819/vms-customizations/2021-09-17-centos7-amd64-misc-libs/

After updating the centos74 bintar build vm I kicked off a couple rebuilds of builds that had failed before and this time compiling was successful:

• http://buildbot.askmonty.org/buildbot/builders/kvm-bintar-centos74-amd64/builds/630
• http://buildbot.askmonty.org/buildbot/builders/kvm-bintar-centos74-amd64/builds/631

Is a static zstd possible for rocksdb (zstd >=1.1.3)?

$ ldd mariadb-10.2.40-linux-systemd-x86_64/lib/plugin/ha_rocksdb.so

ldd: warning: you do not have execution permission for `mariadb-10.2.40-linux-systemd-x86_64/lib/plugin/ha_rocksdb.so'

	linux-vdso.so.1 (0x00007ffe33f6a000)

	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fc4a0ee3000)

	libdl.so.2 => /lib64/libdl.so.2 (0x00007fc4a0edc000)

	librt.so.1 => /lib64/librt.so.1 (0x00007fc4a0ed1000)

	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fc4a0cb2000)

	libm.so.6 => /lib64/libm.so.6 (0x00007fc4a0b6e000)

	libc.so.6 => /lib64/libc.so.6 (0x00007fc4a099f000)

	/lib64/ld-linux-x86-64.so.2 (0x00007fc4a1768000)

	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fc4a0982000)

This would resolve MDEV-13281

I'm going to close this as 'done' for now. We can revisit and reopen, or create a new issue, later if necessary.

I ran into this in a support ticket regarding Ubuntu. Recent releases do not include libncurses5 and libtinfo5. Solution tested on Ubuntu 20.04.6 LTS (Focal Fossa):

wget http://security.ubuntu.com/ubuntu/pool/universe/n/ncurses/libtinfo5_6.2-0ubuntu2.1_amd64.deb

wget http://security.ubuntu.com/ubuntu/pool/universe/n/ncurses/libncurses5_6.2-0ubuntu2.1_amd64.deb

md5sum lib*

dd4899b9818c08b1dc0c30311af49c2a libncurses5_6.2-0ubuntu2.1_amd64.deb

e5107f48531e7f2ef07e54e7d2c41163 libtinfo5_6.2-0ubuntu2.1_amd64.deb

dpkg -i ./libtinfo5_6.2-0ubuntu2.1_amd64.deb

dpkg -i ./libncurses5_6.2-0ubuntu2.1_amd64.deb

Please consider reopening this issue. Ubuntu 23.10 (and, as I understand, Debian newer than "Bookworm") does NOT include libncurses5 any more, so the MariaDB binary distribution becomes uninstallable out of the box on this distribution - and supposedly also on Ubuntu 24.04 LTS.

You CAN circumvent the problem by installing libncurses5 from an older Ubuntu distribution as described in the comment above, but that's surely not sustainable.

Yes, with MDEV-32111 - in the current release (that is in progress) its changed and will be ncurses(6).

Or use containers.