Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-29955

MariaDB does not make full use of pkgconfig

Details

    Description

      Hi.

      This problem i had also in previous releases.

      I've installed openssl, pcre2, curl, zlib, ncurses ect. from source.
      My installation structure is "/own/dep/..."

      The .pc-files (PKG-Config) are in /usr/share/pkgconfig and if i test each of them with pkg-config (e.g "pkg-config zlib --libs / --cflags") all is fine.

      For compiling mariadb i set the cmake-options:

      • -DWITH_ZLIB=system
      • -DWITH_LIBFMT=system
      • -DWITH_PCRE=system
      • ...

      but zlib, libfmt, pcre2-8 and ncurses are not found:

      Could NOT find ZLIB (missing: ZLIB_LIBRARY ZLIB_INCLUDE_DIR)
      ...
      CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
        Could NOT find Curses (missing: CURSES_LIBRARY CURSES_INCLUDE_PATH)
      ...
      CMake Error at cmake/pcre.cmake:84 (MESSAGE):
        system pcre2-8 library is not found or unusable
      

      Why mariadb doesn't find these packages while others like openssl and curl are found?

      The output of pkg-config (libdir | includedir | libs | cflags):

      zlib:
      ====================================

      • /own/dep/zlib/lib
      • /own/dep/zlib/include
      • -L/own/dep/zlib/lib -lz
      • -I/own/dep/zlib/include
        ====================================

      fmt:
      ====================================

      • /own/dep/libfmt/lib
      • /own/dep/libfmt/include
      • -L/own/dep/libfmt/lib -lfmt
      • -I/own/dep/libfmt/include
        ====================================

      ncurses:
      ====================================

      • /own/dep/libncurses/lib
      • /own/dep/libncurses/include/ncurses
      • -L/own/dep/pcre2/lib -lncurses -lpcre2-posix
      • -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=600 -I/own/dep/libncurses/include/ncurses -I/own/dep/libncurses/include
        ====================================

      libpcre2-8 / libpcre2-posix
      ====================================

      • /own/dep/pcre2/lib
      • /own/dep/pcre2/include
      • -L/own/dep/pcre2/lib -lpcre2-8 / -lpcre2-posix
      • -I/own/dep/pcre2/include
        ====================================

      I know that i can use additional options like -DZLIB_... or -DCURSES_...

      But why, as i said mariadb found openssl and curl with pkg-config but not the others?

      Other programs like nginx and php found all of them!

      Attachments

        1. 1_CMakeCache.txt
          48 kB
          Dimitri
        2. 1_CMakeError.log
          90 kB
          Dimitri
        3. 1_CMakeOutput.log
          302 kB
          Dimitri
        4. 2_CMakeCache.txt
          50 kB
          Dimitri
        5. 2_CMakeError.log
          92 kB
          Dimitri
        6. 2_CMakeOutput.log
          306 kB
          Dimitri
        7. screenshot-1.png
          35 kB
          Vladislav Vaintroub

        Issue Links

          Activity

            danblack Daniel Black added a comment - - edited

            Can you attach CMakeFiles/CMake{Output,Error}.log?

            danblack Daniel Black added a comment - - edited Can you attach CMakeFiles/CMake{Output,Error}.log ?
            JustMe Dimitri added a comment - - edited

            Attached all Files.

            Files with 1_* are after a fresh extracting and first cmake call.
            Files with 2_* are after deleting CMakeCache ect. and second cmake call with additional options "-DCURSES_LIBRARY=/own/dep/libncurses/lib/libncurses.so -DCURSES_INCLUDE_PATH=/own/dep/libncurses/include/ncurses"

            After first call it ends with:
            =========================
            CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
            Could NOT find Curses (missing: CURSES_LIBRARY CURSES_INCLUDE_PATH)
            Call Stack (most recent call first):
            /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
            /usr/share/cmake-3.22/Modules/FindCurses.cmake:268 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
            cmake/readline.cmake:55 (FIND_PACKAGE)
            cmake/readline.cmake:185 (FIND_CURSES)
            CMakeLists.txt:388 (MYSQL_CHECK_READLINE)
            =========================

            After second call it ends with:
            =========================
            CMake Error at cmake/pcre.cmake:84 (MESSAGE):
            system pcre2-8 library is not found or unusable
            Call Stack (most recent call first):
            CMakeLists.txt:392 (CHECK_PCRE)
            =========================

            I don't understand why after adding CURSES_INCLUDE_PATH in the second call the files "curses.h" and "term.h" still not found:
            =========================
            – Looking for include files curses.h, term.h
            – Looking for include files curses.h, term.h - not found
            =========================

            These files are in /own/dep/libncurses/include/ncurses, as i set in CURSES_INCLUDE_PATH...

            After set the following environment variables pcre2 is found:
            =========================
            export CXXFLAGS=-I/own/dep/pcre2/include
            export LDFLAGS=-L/own/dep/pcre2/lib
            ...
            – Looking for pcre2_match_8 in pcre2-8
            – Looking for pcre2_match_8 in pcre2-8 - found
            =========================

            After next cmake call it ends with:
            =========================
            CMake Error at cmake/libfmt.cmake:45 (MESSAGE):
            system libfmt library is not found or unusable
            Call Stack (most recent call first):
            CMakeLists.txt:393 (CHECK_LIBFMT)
            =========================

            After adding libfmt-paths to CXX/LDFLAGS the next call is successful done...

            ... but compilation ends with the following:
            =========================
            [ 40%] Building C object client/CMakeFiles/mariadb-import.dir/mysqlimport.c.o
            /own/tmp/mariadb-10.9.3/client/mysql.cc:81:10: fatal error: curses.h: No such file or directory
            81 | #include <curses.h>

            compilation terminated.
            =========================

            Why? "curses.h" is in /own/dep/libncurses/include/ncurses. cmake knows this path.

            JustMe Dimitri added a comment - - edited Attached all Files. Files with 1_* are after a fresh extracting and first cmake call. Files with 2_* are after deleting CMakeCache ect. and second cmake call with additional options "-DCURSES_LIBRARY=/own/dep/libncurses/lib/libncurses.so -DCURSES_INCLUDE_PATH=/own/dep/libncurses/include/ncurses" After first call it ends with: ========================= CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message): Could NOT find Curses (missing: CURSES_LIBRARY CURSES_INCLUDE_PATH) Call Stack (most recent call first): /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE) /usr/share/cmake-3.22/Modules/FindCurses.cmake:268 (FIND_PACKAGE_HANDLE_STANDARD_ARGS) cmake/readline.cmake:55 (FIND_PACKAGE) cmake/readline.cmake:185 (FIND_CURSES) CMakeLists.txt:388 (MYSQL_CHECK_READLINE) ========================= After second call it ends with: ========================= CMake Error at cmake/pcre.cmake:84 (MESSAGE): system pcre2-8 library is not found or unusable Call Stack (most recent call first): CMakeLists.txt:392 (CHECK_PCRE) ========================= I don't understand why after adding CURSES_INCLUDE_PATH in the second call the files "curses.h" and "term.h" still not found: ========================= – Looking for include files curses.h, term.h – Looking for include files curses.h, term.h - not found ========================= These files are in /own/dep/libncurses/include/ncurses, as i set in CURSES_INCLUDE_PATH... After set the following environment variables pcre2 is found: ========================= export CXXFLAGS=-I/own/dep/pcre2/include export LDFLAGS=-L/own/dep/pcre2/lib ... – Looking for pcre2_match_8 in pcre2-8 – Looking for pcre2_match_8 in pcre2-8 - found ========================= After next cmake call it ends with: ========================= CMake Error at cmake/libfmt.cmake:45 (MESSAGE): system libfmt library is not found or unusable Call Stack (most recent call first): CMakeLists.txt:393 (CHECK_LIBFMT) ========================= After adding libfmt-paths to CXX/LDFLAGS the next call is successful done... ... but compilation ends with the following: ========================= [ 40%] Building C object client/CMakeFiles/mariadb-import.dir/mysqlimport.c.o /own/tmp/mariadb-10.9.3/client/mysql.cc:81:10: fatal error: curses.h: No such file or directory 81 | #include <curses.h> compilation terminated. ========================= Why? "curses.h" is in /own/dep/libncurses/include/ncurses. cmake knows this path.
            cmenghsi Meng-Hsiu Chiang added a comment - - edited

            For zlib, when using "non-standard" path, FindZLIB depends on a variable ZLIB_ROOT (which isn't documented) to figure out where the library is. I've made it work and will check other dependencies mentioned above (ncurses, pcre, and libfmt) to see how many packages has the same issue and fix them altogether, or do it separately if they have different causes.

            cmenghsi Meng-Hsiu Chiang added a comment - - edited For zlib, when using "non-standard" path, FindZLIB depends on a variable ZLIB_ROOT (which isn't documented) to figure out where the library is. I've made it work and will check other dependencies mentioned above (ncurses, pcre, and libfmt) to see how many packages has the same issue and fix them altogether, or do it separately if they have different causes.
            Gosselin Dave Gosselin added a comment -

            wlad and cmenghsi, git commit 55cb2c2916a26b699ed8260427002243682c7110 breaks compilation of MariaDB 10.5 on macOS latest. Thankfully, going forward, we have a 'Required' buildbot on macOS which will block PRs that don't successfully compile on macOS. Reverting that commit locally allows the build to complete successfully.

            Gosselin Dave Gosselin added a comment - wlad and cmenghsi , git commit 55cb2c2916a26b699ed8260427002243682c7110 breaks compilation of MariaDB 10.5 on macOS latest. Thankfully, going forward, we have a 'Required' buildbot on macOS which will block PRs that don't successfully compile on macOS. Reverting that commit locally allows the build to complete successfully.
            wlad Vladislav Vaintroub added a comment - - edited

            Gosselin, https://github.com/mariadb/server/commit/55cb2c2916a26b699ed8260427002243682c7110 shows successful compilation on aarch64-macos. I can't not know how much this differs from "macos latest", but I do not see obvious mistakes in handling this pull request, with or without "required" macOS builder. All buildbot-builders that are visible on github have successfully finished on this commit, or are "green" at least.

            wlad Vladislav Vaintroub added a comment - - edited Gosselin , https://github.com/mariadb/server/commit/55cb2c2916a26b699ed8260427002243682c7110 shows successful compilation on aarch64-macos. I can't not know how much this differs from "macos latest", but I do not see obvious mistakes in handling this pull request, with or without "required" macOS builder. All buildbot-builders that are visible on github have successfully finished on this commit, or are "green" at least.
            Gosselin Dave Gosselin added a comment -

            wlad you are correct, there is no problem with this change. I had a conflict in my environment with homebrew and the default system zlib which this change uncovered. Correcting my environment issue fixed the problem. My apologies for the distraction.

            Gosselin Dave Gosselin added a comment - wlad you are correct, there is no problem with this change. I had a conflict in my environment with homebrew and the default system zlib which this change uncovered. Correcting my environment issue fixed the problem. My apologies for the distraction.
            Gosselin Dave Gosselin added a comment -

            serg I think that this was a bug at some point since there is a merged PR for this ticket, but maybe I'm wrong?

            Gosselin Dave Gosselin added a comment - serg I think that this was a bug at some point since there is a merged PR for this ticket, but maybe I'm wrong?

            Seems this was implemented via https://github.com/MariaDB/server/commit/3a33ae86010051a1022c392f2936b5fc9f3ab19e which had a small flaw. Fix has been pending review at
            https://github.com/MariaDB/server/pull/3225 for a while now..

            otto Otto Kekäläinen added a comment - Seems this was implemented via https://github.com/MariaDB/server/commit/3a33ae86010051a1022c392f2936b5fc9f3ab19e which had a small flaw. Fix has been pending review at https://github.com/MariaDB/server/pull/3225 for a while now..

            People

              wlad Vladislav Vaintroub
              JustMe Dimitri
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.