Bintars specifications (starting with 11.8)

• Bintars are binaries built for every release of MariaDB.
• Bintars are meant to be used as a statically linked alternative to regular packages. This means that they must be able to run on base linux systems, without any other dependencies installed.
• Practical use cases:
  • Install MariaDB on an air-gapped system
  • Install multiple versions of MariaDB on the same system
  • Partially equivalent to running multiple containers of MariaDB but without the container infra.

1. Definition of Done

There will be an automatic validation script written to validate that a bintar archive is correct.
The validation script will run the following pseudocode:

1.1 Check library dependencies:

for all executables / libraries  inside the bintar run

	readelf <binary> | grep 'Shared Library'

The list of dynamically linked libraries in those binaries should not be more than the following list:
For mariadbd:

• /lib64/ld-linux-x86-64.so.2
• libcrypt.so.1
• libc.so.6
• libdl.so.2
• libgcc_s.so.1
• libm.so.6
• libpthread.so.0
• libstdc++.so.6
• libsystemd.so.0
• linux-vdso.so.1

For mariadb client:

• libdl.so.2
• libstdc++.so.6
• libm.so.6
• libgcc_s.so.1
• libc.so.6
• ld-linux-x86-64.so.2

For other clients and tools:

• /lib64/ld-linux-x86-64.so.2
• libboost_program_options.so.1.52.0
• libbrotlicommon.so.1
• libbrotlidec.so.1
• libcap.so.2
• libcom_err.so.2
• libcrypto.so.1.0.0
• libcrypto.so.3
• libcrypt.so.1
• libc.so.6
• libcurl.so.4
• libdl.so.2
• libgcc_s.so.1
• libgssapi_krb5.so.2
• libidn2.so.0
• libk5crypto.so.3
• libkeyutils.so.1
• libkrb5.so.3
• libpam.so.0
• libkrb5support.so.0
• libm.so.6
• libnghttp2.so.14
• libnghttp3.so.9
• libpsl.so.5
• libpthread.so.0
• libresolv.so.2
• librt.so.1
• libsnappy.so.1
• libssh2.so.1
• libssl.so.1.0.0
• libssl.so.3
• libstdc++.so.6
• libsystemd.so.0
• libunistring.so.5
• libz.so.1
• libzstd.so.1

1.2 Plugins enabled for MariaDB Server:

The binaries contain the following storage engine / plugins compiled:

• ./lib/galera/libgalera_smm.so
• ./lib/libgalera_smm.so
• ./lib/libmariadb.so
• ./lib/libmariadb.so.3
• ./lib/libmariadb.so.3
• ./lib/libmysqlclient_r.so
• ./lib/libmysqlclient.so
• ./lib/plugin/adt_null.so
• ./lib/plugin/auth_0x0100.so
• ./lib/plugin/auth_ed25519.so
• ./lib/plugin/auth_gssapi_client.so
• ./lib/plugin/auth_gssapi.so
• ./lib/plugin/auth_pam.so
• ./lib/plugin/auth_parsec.so
• ./lib/plugin/auth_pam_v1.so
• ./lib/plugin/auth_test_plugin.so
• ./lib/plugin/caching_sha2_password.so
• ./lib/plugin/client_ed25519.so
• ./lib/plugin/cracklib_password_check.so
• ./lib/plugin/debug_key_management.so
• ./lib/plugin/dialog_examples.so
• ./lib/plugin/dialog.so
• ./lib/plugin/disks.so
• ./lib/plugin/example_key_management.so
• ./lib/plugin/file_key_management.so
• ./lib/plugin/func_test.so
• ./lib/plugin/ha_archive.so
• ./lib/plugin/ha_blackhole.so
• ./lib/plugin/ha_connect.so
• ./lib/plugin/ha_example.so
• ./lib/plugin/ha_federated.so
• ./lib/plugin/ha_federatedx.so
• ./lib/plugin/ha_mroonga.so
• ./lib/plugin/handlersocket.so
• ./lib/plugin/ha_oqgraph.so
• ./lib/plugin/ha_rocksdb.so
• ./lib/plugin/ha_s3.so
• ./lib/plugin/hashicorp_key_management.so
• ./lib/plugin/ha_sphinx.so
• ./lib/plugin/ha_spider.so
• ./lib/plugin/ha_test_sql_discovery.so
• ./lib/plugin/libdaemon_example.so
• ./lib/plugin/locales.so
• ./lib/plugin/metadata_lock_info.so
• ./lib/plugin/mypluglib.so
• ./lib/plugin/mysql_clear_password.so
• ./lib/plugin/password_reuse_check.so
• ./lib/plugin/provider_bzip2.so
• ./lib/plugin/provider_lz4.so
• ./lib/plugin/provider_lzma.so
• ./lib/plugin/provider_lzo.so
• ./lib/plugin/provider_snappy.so
• ./lib/plugin/qa_auth_client.so
• ./lib/plugin/qa_auth_interface.so
• ./lib/plugin/qa_auth_server.so
• ./lib/plugin/query_cache_info.so
• ./lib/plugin/query_response_time.so
• ./lib/plugin/server_audit.so
• ./lib/plugin/simple_password_check.so
• ./lib/plugin/sql_errlog.so
• ./lib/plugin/test_sql_service.so
• ./lib/plugin/test_versioning.so
• ./lib/plugin/type_mysql_json.so
• ./lib/plugin/type_mysql_timestamp.so
• ./lib/plugin/type_test.so
• ./lib/plugin/wsrep_info.so
• ./lib/plugin/zstd.so
• ./mariadb-test/suite/plugins/pam/pam_mariadb_mtr.so
• ./share/pam_user_map.so

1.3 Runtime test

for all supported distributions, within a base container:

docker run -it debian:x

# replace debian with all distributions and x with the corresponding image.

The server component should not show any unresolved libraries.

• ldd mariadbd
  Smoke test:
• Install bintar
• Start the server
• Query using the client in the archive {select 1;}

2. Building bintars

Bintars are configured using the command line:
cmake .
-DCMAKE_BUILD_TYPE=RelWithDebInfo

1. Set by default in the server, but best to be explicit
   -DBUILD_CONFIG=mysql_release
2. Set by default
   -DPLATFORM=linux-systemd
3. So MariaDB version is suffixed with linux-systemd
   -DCMAKE_LIBRARY_PATH=/home/buildbot/lib
4. Exact path name is not critical, but it must be documented.
   TODO(cvicentiu) OTHER -D flags.
   Compiled with make package

3. Bintars build environment

3.1 Distribution of Choice:

The bintar builders should always be based on the

• Oldest released distribution version
• Is still supported by the distribution (not End of Life)
• Is supported by MariaDB
• Is SystemD enabled
  • For example, as of today, RHEL 8 & derivatives fit that criteria.

The reason for this is that it will have the oldest possible libc and libstdc++ dynamically linked and thus should be compatible with other distributions.
TODO in the future, evaluate if it is possible to create a complete static bintar, then we can use any distribution we would want.

3.2 Environment runtime

The bintar builders will use a Docker container based on the distribution that fits the criteria above.
This container base image will be changed as soon as that distribution becomes EOL, regardless of MariaDB Server release lifecycle.

3.3 Statically linked libraries

Bintar builders will need to have a set of statically compiled libraries to link with the server. These libraries are:
Libraries obtained from the following log of old buildbot builder https://buildbot.mariadb.net/buildbot/builders/kvm-bintar-centos74-amd64/builds/14357:

cat log.txt | grep '/home/buildbot/local/lib/\S*\.a ' -o | sort  | uniq

/home/buildbot/local/lib/libaio.a 

/home/buildbot/local/lib/liblz4.a 

/home/buildbot/local/lib/liblzma.a 

/home/buildbot/local/lib/libncurses.a 

/home/buildbot/local/lib/libpmem.a 

/home/buildbot/local/lib/libzstd.a 

These libraries are also linked as part of the old buildbot builder, but will remain dynamic:

 cat log.txt | grep '\-l\S* ' -o | sort | uniq

-lcrack 

-lcrypt 

-ldl 

-lgssapi_krb5 

-lk5crypto 

-lm 

-lpam 

-lpthread 

-lrt 

-lsystemd 

-lz 

Other libraries of interest that are dynamically linked in Debian, but not in Bintars:

libaudit.so.1

libboost_program_options.so.1.74.0

libbz2.so.1.0

libcap-ng.so.0

libcrack.so.2

libedit.so.2

libicudata.so.75

libicuuc.so.75

libJudy.so.1

libltdl.so.7

liblz4.so.1 - Statically linked in bintar

liblzma.so.5

liblzo2.so.2

libncurses.so.6 - Statically linked in bintar

libodbc.so.2

libpam.so.0

libpcre2-8.so.0

libtinfo.so.6

liburing.so.2

libxml2.so.2

Other libraries that are linked in bintars, but not in Debian

libboost_program_options.so.1.52.0

libcrypto.so.1.0.0

libdl.so.2

libpthread.so.0

librt.so.1

libssl.so.1.0.0

3.4 Build environment update process

Bintar builders will build the statically compiled libraries using the following approach:
WIP -> Commands not yet tested.

dnf download --source <library>

rpm -ivh <library> 

cd ~/rpmbuild/SOURCES/

tar -xvf *.tar.xz

# <custom compile step to build the library statically>

Copy the lib<library>.a and include files into /home/buildbot/lib path

• This build process ensures that when the distribution maintainers update each library, that we compile the most recent one.
• The build environment will run this update process periodically (roughly every other week, we'll reevaluate after some time if the builder is stable)
• The updates will be manually applied to the production builder once testing in dev environment succeeds.