Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-23031

Make MariaDB Server build reproducibly

    XMLWordPrintable

Details

    • Bug
    • Status: Stalled (View Workflow)
    • Minor
    • Resolution: Unresolved
    • 10.1, 10.2, 10.3, 10.4, 10.5, 10.6
    • 10.5
    • None
    • None

    Description

      All open source projects should build reproducibly. This improves security by making supply chain attacks harder. This also has the nice side effect that tools like ccache and alike work better, since code builds in the same way from build to build for the same code.

      For more information see https://reproducible-builds.org/

      Current status for MariaDB 10.3 in Debian:
      https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/mariadb-10.3.html

      The Mroonga issue is maybe already fixed upstream, results should be visible in 10.5 when I get around to upload it to Debian.

      RocksDB issue is filed upstream at https://github.com/facebook/rocksdb/issues/7035

      TokuDB is removed in 10.5, so that solves it.

      ColumnStore will make 10.5 take a step backwards in this regard, but working on that in MCOL-4117

      Here is a quick overview of current differences in two consecutive builds for the exact same source and commit id:

      $ diffstat server.build1-build2.diff
       .git/ORIG_HEAD                                                                                                                |    2 
       .git/gitk.cache                                                                                                               |    3 
       .git/logs/HEAD                                                                                                                |    2 
       builddir/CMakeFiles/CMakeError.log                                                                                            |   18 +-
       builddir/CMakeFiles/CMakeOutput.log                                                                                           |   74 +++++-----
       builddir/Docs/INFO_BIN                                                                                                        |    2 
       builddir/Docs/INFO_SRC                                                                                                        |    2 
       builddir/storage/columnstore/columnstore/utils/configcpp/CMakeFiles/configcpp.dir/CXX.includecache                            |    2 
       builddir/storage/columnstore/columnstore/utils/configcpp/CMakeFiles/configcpp.dir/depend.internal                             |    3 
       builddir/storage/columnstore/columnstore/utils/configcpp/CMakeFiles/configcpp.dir/depend.make                                 |    3 
       builddir/storage/rocksdb/build_version.cc                                                                                     |    2 
       debian/changelog                                                                                                              |    2 
       debian/libmariadb-dev-compat/DEBIAN/md5sums                                                                                   |    2 
       debian/libmariadb-dev/DEBIAN/md5sums                                                                                          |    2 
       debian/libmariadb3-compat/DEBIAN/md5sums                                                                                      |    2 
       debian/libmariadb3/DEBIAN/md5sums                                                                                             |    2 
       debian/libmariadbclient18/DEBIAN/md5sums                                                                                      |    2 
       debian/libmariadbd-dev/DEBIAN/md5sums                                                                                         |    2 
       debian/libmariadbd19/DEBIAN/md5sums                                                                                           |    2 
       debian/libmysqlclient18/DEBIAN/md5sums                                                                                        |    2 
       debian/mariadb-backup/DEBIAN/md5sums                                                                                          |    2 
       debian/mariadb-client-10.5/DEBIAN/md5sums                                                                                     |    2 
       debian/mariadb-client-core-10.5/DEBIAN/md5sums                                                                                |    2 
       debian/mariadb-client/DEBIAN/md5sums                                                                                          |    2 
       debian/mariadb-common/DEBIAN/md5sums                                                                                          |    2 
       debian/mariadb-plugin-columnstore/DEBIAN/md5sums                                                                              |    2 
       debian/mariadb-plugin-connect/DEBIAN/md5sums                                                                                  |    2 
       debian/mariadb-plugin-cracklib-password-check/DEBIAN/md5sums                                                                  |    2 
       debian/mariadb-plugin-gssapi-client/DEBIAN/md5sums                                                                            |    2 
       debian/mariadb-plugin-gssapi-server/DEBIAN/md5sums                                                                            |    2 
       debian/mariadb-plugin-mroonga/DEBIAN/md5sums                                                                                  |    2 
       debian/mariadb-plugin-oqgraph/DEBIAN/md5sums                                                                                  |    2 
       debian/mariadb-plugin-rocksdb/DEBIAN/md5sums                                                                                  |    6 
       debian/mariadb-plugin-s3/DEBIAN/md5sums                                                                                       |    2 
       debian/mariadb-plugin-spider/DEBIAN/md5sums                                                                                   |    2 
       debian/mariadb-server-10.5/DEBIAN/md5sums                                                                                     |    2 
       debian/mariadb-server-core-10.5/DEBIAN/md5sums                                                                                |    2 
       debian/mariadb-server/DEBIAN/md5sums                                                                                          |    2 
       debian/mariadb-test-data/DEBIAN/md5sums                                                                                       |   10 -
       debian/mariadb-test/DEBIAN/md5sums                                                                                            |    2 
       debian/mysql-common/DEBIAN/md5sums                                                                                            |    2 
       server.build1/debian/tmp/dh-exec.PXrNyBzf                                                                                     |only
       server.build2/.git/index                                                                                                      |binary
       server.build2/builddir/storage/rocksdb/CMakeFiles/rocksdblib.dir/build_version.cc.o                                           |binary
       server.build2/builddir/storage/rocksdb/ha_rocksdb.so                                                                          |binary
       server.build2/builddir/storage/rocksdb/librocksdblib.a                                                                        |binary
       server.build2/builddir/storage/rocksdb/mariadb-ldb                                                                            |binary
       server.build2/builddir/storage/rocksdb/mysql_ldb                                                                              |binary
       server.build2/builddir/storage/rocksdb/sst_dump                                                                               |binary
       server.build2/debian/libmariadb-dev-compat/usr/share/doc/libmariadb-dev-compat/changelog.gz                                   |binary
       server.build2/debian/libmariadb-dev/usr/share/doc/libmariadb-dev/changelog.gz                                                 |binary
       server.build2/debian/libmariadb3-compat/usr/share/doc/libmariadb3-compat/changelog.gz                                         |binary
       server.build2/debian/libmariadb3/usr/share/doc/libmariadb3/changelog.gz                                                       |binary
       server.build2/debian/libmariadbclient18/usr/share/doc/libmariadbclient18/changelog.gz                                         |binary
       server.build2/debian/libmariadbd-dev/usr/share/doc/libmariadbd-dev/changelog.gz                                               |binary
       server.build2/debian/libmariadbd19/usr/share/doc/libmariadbd19/changelog.gz                                                   |binary
       server.build2/debian/libmysqlclient18/usr/share/doc/libmysqlclient18/changelog.gz                                             |binary
       server.build2/debian/mariadb-backup/usr/share/doc/mariadb-backup/changelog.gz                                                 |binary
       server.build2/debian/mariadb-client-10.5/usr/share/doc/mariadb-client-10.5/changelog.gz                                       |binary
       server.build2/debian/mariadb-client-core-10.5/usr/share/doc/mariadb-client-core-10.5/changelog.gz                             |binary
       server.build2/debian/mariadb-client/usr/share/doc/mariadb-client/changelog.gz                                                 |binary
       server.build2/debian/mariadb-common/usr/share/doc/mariadb-common/changelog.gz                                                 |binary
       server.build2/debian/mariadb-plugin-columnstore/usr/share/doc/mariadb-plugin-columnstore/changelog.gz                         |binary
       server.build2/debian/mariadb-plugin-connect/usr/share/doc/mariadb-plugin-connect/changelog.gz                                 |binary
       server.build2/debian/mariadb-plugin-cracklib-password-check/usr/share/doc/mariadb-plugin-cracklib-password-check/changelog.gz |binary
       server.build2/debian/mariadb-plugin-gssapi-client/usr/share/doc/mariadb-plugin-gssapi-client/changelog.gz                     |binary
       server.build2/debian/mariadb-plugin-gssapi-server/usr/share/doc/mariadb-plugin-gssapi-server/changelog.gz                     |binary
       server.build2/debian/mariadb-plugin-mroonga/usr/share/doc/mariadb-plugin-mroonga/changelog.gz                                 |binary
       server.build2/debian/mariadb-plugin-oqgraph/usr/share/doc/mariadb-plugin-oqgraph/changelog.gz                                 |binary
       server.build2/debian/mariadb-plugin-rocksdb/usr/bin/mariadb-ldb                                                               |binary
       server.build2/debian/mariadb-plugin-rocksdb/usr/bin/mysql_ldb                                                                 |binary
       server.build2/debian/mariadb-plugin-rocksdb/usr/lib/mysql/plugin/ha_rocksdb.so                                                |binary
       server.build2/debian/mariadb-plugin-rocksdb/usr/share/doc/mariadb-plugin-rocksdb/changelog.gz                                 |binary
       server.build2/debian/mariadb-plugin-s3/usr/share/doc/mariadb-plugin-s3/changelog.gz                                           |binary
       server.build2/debian/mariadb-plugin-spider/usr/share/doc/mariadb-plugin-spider/changelog.gz                                   |binary
       server.build2/debian/mariadb-server-10.5/usr/share/doc/mariadb-server-10.5/changelog.gz                                       |binary
       server.build2/debian/mariadb-server-core-10.5/usr/share/doc/mariadb-server-core-10.5/changelog.gz                             |binary
       server.build2/debian/mariadb-server/usr/share/doc/mariadb-server/changelog.gz                                                 |binary
       server.build2/debian/mariadb-test-data/usr/share/doc/mariadb-test-data/changelog.gz                                           |binary
       server.build2/debian/mariadb-test-data/usr/share/mysql/mysql-test/plugin/connect/connect/std_data/JavaWrappers.jar            |binary
       server.build2/debian/mariadb-test-data/usr/share/mysql/mysql-test/plugin/connect/connect/std_data/JdbcMariaDB.jar             |binary
       server.build2/debian/mariadb-test-data/usr/share/mysql/mysql-test/plugin/connect/connect/std_data/Mongo2.jar                  |binary
       server.build2/debian/mariadb-test-data/usr/share/mysql/mysql-test/plugin/connect/connect/std_data/Mongo3.jar                  |binary
       server.build2/debian/mariadb-test/usr/share/doc/mariadb-test/changelog.gz                                                     |binary
       server.build2/debian/mysql-common/usr/share/doc/mysql-common/changelog.gz                                                     |binary
       server.build2/debian/tmp/dh-exec.WmAf51Gu                                                                                     |only
       server.build2/debian/tmp/usr/bin/mariadb-ldb                                                                                  |binary
       server.build2/debian/tmp/usr/bin/mysql_ldb                                                                                    |binary
       server.build2/debian/tmp/usr/bin/sst_dump                                                                                     |binary
       server.build2/debian/tmp/usr/lib/mysql/plugin/ha_rocksdb.so                                                                   |binary
      

      Generic part that seems to apply to the whole code base:

      diff -r -U 0 server.build1/builddir/Docs/INFO_BIN server.build2/builddir/Docs/INFO_BIN
      --- server.build1/builddir/Docs/INFO_BIN        2020-06-26 16:40:47.586982869 +0300
      +++ server.build2/builddir/Docs/INFO_BIN        2020-06-26 21:41:20.950240151 +0300
      @@ -2 +2 @@
      -Build was run at 2020-06-26 13:40:47 on host 'd183b0c449af'
      +Build was run at 2020-06-26 18:41:20 on host '0bfca1ffaaac'
      diff -r -U 0 server.build1/builddir/Docs/INFO_SRC server.build2/builddir/Docs/INFO_SRC
      --- server.build1/builddir/Docs/INFO_SRC        2020-06-26 16:40:47.606983483 +0300
      +++ server.build2/builddir/Docs/INFO_SRC        2020-06-26 21:41:20.974241246 +0300
      

      I have assigned myself on this and will use this issue to track progress on this front.

      Attachments

        Issue Links

          Activity

            People

              otto Otto Kekäläinen
              otto Otto Kekäläinen
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.