[MDEV-23031] Make MariaDB Server build reproducibly Created: 2020-06-26  Updated: 2021-04-04

Status: Stalled
Project: MariaDB Server
Component/s: None
Affects Version/s: 10.1, 10.2, 10.3, 10.4, 10.5, 10.6
Fix Version/s: 10.5

Type: Bug Priority: Minor
Reporter: Otto Kekäläinen Assignee: Otto Kekäläinen
Resolution: Unresolved Votes: 0
Labels: None

Attachments: PNG File image-2020-06-26-22-21-45-919.png    
Issue Links:
Relates
relates to MCOL-4117 Don't disable ccache in ColumnStore C... Closed
relates to MDEV-7551 Debian and reproducible builds errors... Closed

 Description   

All open source projects should build reproducibly. This improves security by making supply chain attacks harder. This also has the nice side effect that tools like ccache and alike work better, since code builds in the same way from build to build for the same code.

For more information see https://reproducible-builds.org/

Current status for MariaDB 10.3 in Debian:
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/mariadb-10.3.html

The Mroonga issue is maybe already fixed upstream, results should be visible in 10.5 when I get around to upload it to Debian.

RocksDB issue is filed upstream at https://github.com/facebook/rocksdb/issues/7035

TokuDB is removed in 10.5, so that solves it.

ColumnStore will make 10.5 take a step backwards in this regard, but working on that in MCOL-4117

Here is a quick overview of current differences in two consecutive builds for the exact same source and commit id:

$ diffstat server.build1-build2.diff
 
 .git/ORIG_HEAD                                                                                                                |    2 
 .git/gitk.cache                                                                                                               |    3 
 .git/logs/HEAD                                                                                                                |    2 
 builddir/CMakeFiles/CMakeError.log                                                                                            |   18 +-
 
 builddir/CMakeFiles/CMakeOutput.log                                                                                           |   74 +++++-----
 
 builddir/Docs/INFO_BIN                                                                                                        |    2 
 builddir/Docs/INFO_SRC                                                                                                        |    2 
 builddir/storage/columnstore/columnstore/utils/configcpp/CMakeFiles/configcpp.dir/CXX.includecache                            |    2 
 builddir/storage/columnstore/columnstore/utils/configcpp/CMakeFiles/configcpp.dir/depend.internal                             |    3 
 builddir/storage/columnstore/columnstore/utils/configcpp/CMakeFiles/configcpp.dir/depend.make                                 |    3 
 builddir/storage/rocksdb/build_version.cc                                                                                     |    2 
 debian/changelog                                                                                                              |    2 
 debian/libmariadb-dev-compat/DEBIAN/md5sums                                                                                   |    2 
 debian/libmariadb-dev/DEBIAN/md5sums                                                                                          |    2 
 debian/libmariadb3-compat/DEBIAN/md5sums                                                                                      |    2 
 debian/libmariadb3/DEBIAN/md5sums                                                                                             |    2 
 debian/libmariadbclient18/DEBIAN/md5sums                                                                                      |    2 
 debian/libmariadbd-dev/DEBIAN/md5sums                                                                                         |    2 
 debian/libmariadbd19/DEBIAN/md5sums                                                                                           |    2 
 debian/libmysqlclient18/DEBIAN/md5sums                                                                                        |    2 
 debian/mariadb-backup/DEBIAN/md5sums                                                                                          |    2 
 debian/mariadb-client-10.5/DEBIAN/md5sums                                                                                     |    2 
 debian/mariadb-client-core-10.5/DEBIAN/md5sums                                                                                |    2 
 debian/mariadb-client/DEBIAN/md5sums                                                                                          |    2 
 debian/mariadb-common/DEBIAN/md5sums                                                                                          |    2 
 debian/mariadb-plugin-columnstore/DEBIAN/md5sums                                                                              |    2 
 debian/mariadb-plugin-connect/DEBIAN/md5sums                                                                                  |    2 
 debian/mariadb-plugin-cracklib-password-check/DEBIAN/md5sums                                                                  |    2 
 debian/mariadb-plugin-gssapi-client/DEBIAN/md5sums                                                                            |    2 
 debian/mariadb-plugin-gssapi-server/DEBIAN/md5sums                                                                            |    2 
 debian/mariadb-plugin-mroonga/DEBIAN/md5sums                                                                                  |    2 
 debian/mariadb-plugin-oqgraph/DEBIAN/md5sums                                                                                  |    2 
 debian/mariadb-plugin-rocksdb/DEBIAN/md5sums                                                                                  |    6 
 debian/mariadb-plugin-s3/DEBIAN/md5sums                                                                                       |    2 
 debian/mariadb-plugin-spider/DEBIAN/md5sums                                                                                   |    2 
 debian/mariadb-server-10.5/DEBIAN/md5sums                                                                                     |    2 
 debian/mariadb-server-core-10.5/DEBIAN/md5sums                                                                                |    2 
 debian/mariadb-server/DEBIAN/md5sums                                                                                          |    2 
 debian/mariadb-test-data/DEBIAN/md5sums                                                                                       |   10 -
 
 debian/mariadb-test/DEBIAN/md5sums                                                                                            |    2 
 debian/mysql-common/DEBIAN/md5sums                                                                                            |    2 
 server.build1/debian/tmp/dh-exec.PXrNyBzf                                                                                     |only
 
 server.build2/.git/index                                                                                                      |binary
 
 server.build2/builddir/storage/rocksdb/CMakeFiles/rocksdblib.dir/build_version.cc.o                                           |binary
 
 server.build2/builddir/storage/rocksdb/ha_rocksdb.so                                                                          |binary
 
 server.build2/builddir/storage/rocksdb/librocksdblib.a                                                                        |binary
 
 server.build2/builddir/storage/rocksdb/mariadb-ldb                                                                            |binary
 
 server.build2/builddir/storage/rocksdb/mysql_ldb                                                                              |binary
 
 server.build2/builddir/storage/rocksdb/sst_dump                                                                               |binary
 
 server.build2/debian/libmariadb-dev-compat/usr/share/doc/libmariadb-dev-compat/changelog.gz                                   |binary
 
 server.build2/debian/libmariadb-dev/usr/share/doc/libmariadb-dev/changelog.gz                                                 |binary
 
 server.build2/debian/libmariadb3-compat/usr/share/doc/libmariadb3-compat/changelog.gz                                         |binary
 
 server.build2/debian/libmariadb3/usr/share/doc/libmariadb3/changelog.gz                                                       |binary
 
 server.build2/debian/libmariadbclient18/usr/share/doc/libmariadbclient18/changelog.gz                                         |binary
 
 server.build2/debian/libmariadbd-dev/usr/share/doc/libmariadbd-dev/changelog.gz                                               |binary
 
 server.build2/debian/libmariadbd19/usr/share/doc/libmariadbd19/changelog.gz                                                   |binary
 
 server.build2/debian/libmysqlclient18/usr/share/doc/libmysqlclient18/changelog.gz                                             |binary
 
 server.build2/debian/mariadb-backup/usr/share/doc/mariadb-backup/changelog.gz                                                 |binary
 
 server.build2/debian/mariadb-client-10.5/usr/share/doc/mariadb-client-10.5/changelog.gz                                       |binary
 
 server.build2/debian/mariadb-client-core-10.5/usr/share/doc/mariadb-client-core-10.5/changelog.gz                             |binary
 
 server.build2/debian/mariadb-client/usr/share/doc/mariadb-client/changelog.gz                                                 |binary
 
 server.build2/debian/mariadb-common/usr/share/doc/mariadb-common/changelog.gz                                                 |binary
 
 server.build2/debian/mariadb-plugin-columnstore/usr/share/doc/mariadb-plugin-columnstore/changelog.gz                         |binary
 
 server.build2/debian/mariadb-plugin-connect/usr/share/doc/mariadb-plugin-connect/changelog.gz                                 |binary
 
 server.build2/debian/mariadb-plugin-cracklib-password-check/usr/share/doc/mariadb-plugin-cracklib-password-check/changelog.gz |binary
 
 server.build2/debian/mariadb-plugin-gssapi-client/usr/share/doc/mariadb-plugin-gssapi-client/changelog.gz                     |binary
 
 server.build2/debian/mariadb-plugin-gssapi-server/usr/share/doc/mariadb-plugin-gssapi-server/changelog.gz                     |binary
 
 server.build2/debian/mariadb-plugin-mroonga/usr/share/doc/mariadb-plugin-mroonga/changelog.gz                                 |binary
 
 server.build2/debian/mariadb-plugin-oqgraph/usr/share/doc/mariadb-plugin-oqgraph/changelog.gz                                 |binary
 
 server.build2/debian/mariadb-plugin-rocksdb/usr/bin/mariadb-ldb                                                               |binary
 
 server.build2/debian/mariadb-plugin-rocksdb/usr/bin/mysql_ldb                                                                 |binary
 
 server.build2/debian/mariadb-plugin-rocksdb/usr/lib/mysql/plugin/ha_rocksdb.so                                                |binary
 
 server.build2/debian/mariadb-plugin-rocksdb/usr/share/doc/mariadb-plugin-rocksdb/changelog.gz                                 |binary
 
 server.build2/debian/mariadb-plugin-s3/usr/share/doc/mariadb-plugin-s3/changelog.gz                                           |binary
 
 server.build2/debian/mariadb-plugin-spider/usr/share/doc/mariadb-plugin-spider/changelog.gz                                   |binary
 
 server.build2/debian/mariadb-server-10.5/usr/share/doc/mariadb-server-10.5/changelog.gz                                       |binary
 
 server.build2/debian/mariadb-server-core-10.5/usr/share/doc/mariadb-server-core-10.5/changelog.gz                             |binary
 
 server.build2/debian/mariadb-server/usr/share/doc/mariadb-server/changelog.gz                                                 |binary
 
 server.build2/debian/mariadb-test-data/usr/share/doc/mariadb-test-data/changelog.gz                                           |binary
 
 server.build2/debian/mariadb-test-data/usr/share/mysql/mysql-test/plugin/connect/connect/std_data/JavaWrappers.jar            |binary
 
 server.build2/debian/mariadb-test-data/usr/share/mysql/mysql-test/plugin/connect/connect/std_data/JdbcMariaDB.jar             |binary
 
 server.build2/debian/mariadb-test-data/usr/share/mysql/mysql-test/plugin/connect/connect/std_data/Mongo2.jar                  |binary
 
 server.build2/debian/mariadb-test-data/usr/share/mysql/mysql-test/plugin/connect/connect/std_data/Mongo3.jar                  |binary
 
 server.build2/debian/mariadb-test/usr/share/doc/mariadb-test/changelog.gz                                                     |binary
 
 server.build2/debian/mysql-common/usr/share/doc/mysql-common/changelog.gz                                                     |binary
 
 server.build2/debian/tmp/dh-exec.WmAf51Gu                                                                                     |only
 
 server.build2/debian/tmp/usr/bin/mariadb-ldb                                                                                  |binary
 
 server.build2/debian/tmp/usr/bin/mysql_ldb                                                                                    |binary
 
 server.build2/debian/tmp/usr/bin/sst_dump                                                                                     |binary
 
 server.build2/debian/tmp/usr/lib/mysql/plugin/ha_rocksdb.so                                                                   |binary

Generic part that seems to apply to the whole code base:

diff -r -U 0 server.build1/builddir/Docs/INFO_BIN server.build2/builddir/Docs/INFO_BIN
 
--- server.build1/builddir/Docs/INFO_BIN        2020-06-26 16:40:47.586982869 +0300
 
+++ server.build2/builddir/Docs/INFO_BIN        2020-06-26 21:41:20.950240151 +0300
 
@@ -2 +2 @@
 
-Build was run at 2020-06-26 13:40:47 on host 'd183b0c449af'
 
+Build was run at 2020-06-26 18:41:20 on host '0bfca1ffaaac'
 
diff -r -U 0 server.build1/builddir/Docs/INFO_SRC server.build2/builddir/Docs/INFO_SRC
 
--- server.build1/builddir/Docs/INFO_SRC        2020-06-26 16:40:47.606983483 +0300
 
+++ server.build2/builddir/Docs/INFO_SRC        2020-06-26 21:41:20.974241246 +0300

I have assigned myself on this and will use this issue to track progress on this front.

 Comments   
Comment by Sergei Petrunia [ 2021-02-09 ]

Rollowup to FOSDEM talk re MyRocks being reproducible

As far as I understand, the build is not reproducible because of these
variables in

storage/rocksdb/rocksdb/util/build_version.cc.in

const char* rocksdb_build_git_sha = "rocksdb_build_git_sha:@@GIT_SHA@@";
 
const char* rocksdb_build_git_date = "rocksdb_build_git_date:@@GIT_DATE_TIME@@";
 
const char* rocksdb_build_compile_date = __DATE__;

rocksdb_build_git_date is not used, while the other two are used in
storage/rocksdb/rocksdb/db/db_impl/db_impl.cc:DumpRocksDBBuildVersion():

  ROCKS_LOG_HEADER(log, "Git sha %s", rocksdb_build_git_sha);
 
  ROCKS_LOG_HEADER(log, "Compile date %s", rocksdb_build_compile_date);

I'm building from source git repository and I have:

(gdb) print rocksdb_build_git_sha
 
  $8 = 0x7fffe7b19bf0 "rocksdb_build_git_sha:@@"
 
(gdb) print rocksdb_build_compile_date
 
  $9 = 0x7fffe7b19c3d "Feb  9 2021"

build_version.cc.in is processed in storage/rocksdb/build_rocksdb.cmake: 

CONFIGURE_FILE(${ROCKSDB_SOURCE_DIR}/util/build_version.cc.in build_version.cc @ONLY)
 
INCLUDE_DIRECTORIES(${ROCKSDB_SOURCE_DIR}/util)
 
list(APPEND SOURCES ${CMAKE_CURRENT_BINARY_DIR}/build_version.cc)

This hints that we could do something else there, e.g. provide our own build_version.cc. There's no need to push any changes into RocksDB.

Comment by Sergei Petrunia [ 2021-02-09 ]

Hmm I also see that this was fixed on RocksDB side: https://github.com/facebook/rocksdb/issues/7035

Comment by Sergei Petrunia [ 2021-02-09 ]

It's better to merge upstream MyRocks/RocksDB, then.

Comment by Otto Kekäläinen [ 2021-04-04 ]

Seems RocksDB in 10.5 has not been updated in 2021 at all, so this issue is still pending for MariaDB Server:

MariaDB-10.5/storage/rocksdb/rocksdb$ git log
 
commit bba5e7bc21093d7cfa765e1280a7c4fdcd284288 (HEAD)
 
Author: sdong <siying.d@fb.com>
 
Date:   Mon Mar 2 16:34:36 2020 -0800

Generated at Thu Feb 08 09:19:19 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.