[MDEV-21942] Building 10.5 requires Internet access Created: 2020-03-14  Updated: 2020-03-18  Resolved: 2020-03-16

Status: Closed
Project: MariaDB Server
Component/s: Packaging
Affects Version/s: 10.5.2
Fix Version/s: 10.5.2

Type: Bug Priority: Critical
Reporter: Otto Kekäläinen Assignee: Sergei Golubchik
Resolution: Fixed Votes: 0
Labels: None

Attachments: PNG File buildbot.askmonty.org.png    

 Description   

While building the latest 10.5 git head I noticed that if requires Internet access during the build. If there is no Internet, it will fail with:

-- Using src='http://ftp.pcre.org/pub/pcre/pcre2-10.34.zip'
CMake Error at pcre2-stamp/download-pcre2.cmake:159 (message):
  Each download failed!
 
    error: downloading 'http://ftp.pcre.org/pub/pcre/pcre2-10.34.zip' failed
         status_code: 6
         status_string: "Couldn't resolve host name"
         log:
         --- LOG BEGIN ---
         Could not resolve host: ftp.pcre.org

Requiring mandatory Internet access for a build is bad practice and universally forbidden across all Linux distributions and all places that care about security and reliability.

All build dependencies should be stated explicitly in the build dependencies, and there should be no need to access the public Internet during the download. If there are any such extra steps, they should be strictly optional. Preferably though there would be no such step at all anywhere.

I could also go into detail to explain how this feature can be used moderately easily to backdoor all MariaDB 10.5 builds and completely compromise the supply chain security, but I assume all involved parties understand this anyway and this feature was just some temporary misjudgment.



 Comments   
Comment by Sergei Golubchik [ 2020-03-14 ]

PCRE2 is a build requirements. The build will automatically decide whether to use the system PCRE2 or try to download one.
You can also force the desired outcome with -DWITH_PCRE=system or -DWITH_PCRE=bundled.

Note that on our debian builders nothing is downloaded, see e.g. https://buildbot.askmonty.org/buildbot/builders/kvm-deb-stretch-amd64/builds/12728/steps/compile/logs/stdio

But it looks like dependencies aren't correctly specified in the control file, it should require libpcre2-dev, not libpcre3-dev.

And the build isn't easy to backdoor, because cmake verifies the md5 hash of the downloaded tarball. So an attacker would need to create a poisoned tarball with exactly the same md5 hash to be able to compromise the build.

Comment by Otto Kekäläinen [ 2020-03-14 ]

I see what you are doing in https://github.com/MariaDB/server/commit/e1301160097a25e62fd1c660bb69ada61bac7e39

However, libpcre3-dev (>= 2:8.35-3.2~) is actually available in all Debian and Ubuntu since long (https://packages.debian.org/search?suite=all&section=all&arch=any&searchon=names&keywords=libpcre3-dev). The d/control file line was inherited from https://github.com/MariaDB/server/commit/7f570bef1e3a6b19925bf7e58e02b975548039eb and the same has been in Debian official packaging for years: https://salsa.debian.org/mariadb-team/mariadb-10.4/-/blob/master/debian/control#L25

Are you sure downgrading to libpcre2 is the correct solution?

Comment by Otto Kekäläinen [ 2020-03-14 ]

serg I am happy to review/test if you make a PR about your fix.

Comment by Sergei Golubchik [ 2020-03-14 ]

It's not downgrading, it's upgrading. Some weird Debian oddity. For whatever historical reasons old pcre got called libpcre3 in Debian. It is very old and in a slow maintenance mode, not many bugs are getting fixed there. The current active pcre version is pcre2. Which is called libpcre2 in Debian.

Comment by Otto Kekäläinen [ 2020-03-14 ]

Right. That seems to be available from Debian Stretch and Ubuntu Xenial onwards. If we still want working packages for older distro releases than that, the autobake-deb.sh needs some adaptations. At least on buildbot we still have Jessie building:

Do you want me to finalize this change now when I know what needs to be done?

Comment by Otto Kekäläinen [ 2020-03-14 ]

https://packages.debian.org/search?suite=all&section=all&arch=any&searchon=names&keywords=libpcre2-dev
https://packages.ubuntu.com/search?suite=all&section=all&arch=any&keywords=libpcre2-dev&searchon=names

Comment by Sergei Golubchik [ 2020-03-14 ]

I think we can safely skip Jessie. Its support ends in June 2020, and 10.5 might not even be GA by that time.
Do you think there's anything else to be done besides that one-liner change:

-               libpcre3-dev (>= 2:8.35-3.2~),
+               libpcre2-dev,

?

Comment by Otto Kekäläinen [ 2020-03-14 ]

There is a bit more. I will have this fixed tomorrow once all CI has passed etc: https://github.com/ottok/mariadb/commit/02938af652baea22d3f1c261e0420e851512f74c

Seems there are regressions on both buildbot.askmonty.org and Travis-CI I need to fix before I can put this in... I really hope one day there is a policy that nothing can be merged on a master branch unless all tests are green..

Comment by Otto Kekäläinen [ 2020-03-18 ]

Jessie is now failing with:

-- Looking for pcre2_match_8 in pcre2-8
-- Looking for pcre2_match_8 in pcre2-8 - not found
CMake Error at cmake/pcre.cmake:62 (MESSAGE):
  system pcre2-8 library is not found or unusable
Call Stack (most recent call first):
  CMakeLists.txt:381 (CHECK_PCRE)
 
 
-- Configuring incomplete, errors occurred!
See also "/home/buildbot/buildbot/build/mariadb-10.5.2/builddir/CMakeFiles/CMakeOutput.log".
See also "/home/buildbot/buildbot/build/mariadb-10.5.2/builddir/CMakeFiles/CMakeError.log".
debian/rules:68: recipe for target 'override_dh_auto_configure' failed
make[1]: *** [override_dh_auto_configure] Error 1
make[1]: Leaving directory '/home/buildbot/buildbot/build/mariadb-10.5.2'
debian/rules:177: recipe for target 'build' failed
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2

These failures happen even when the old libpcre3-dev is around.

But as stated, maybe we can ignore that.

Generated at Thu Feb 08 09:11:00 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.