[MDEV-23306] fatal error: pcre2.h: No such file or directory Created: 2020-07-28  Updated: 2021-09-28  Resolved: 2021-09-28

Status: Closed
Project: MariaDB Server
Component/s: Compiling
Affects Version/s: 10.5.4
Fix Version/s: 10.5.13, 10.6.5

Type: Bug Priority: Major
Reporter: Mingli-Yu Assignee: Vladislav Vaintroub
Resolution: Fixed Votes: 0
Labels: None
Environment:

ubuntu 18.04



 Description   

Build fails sometimes with below error, seems it is a race condition, not occurs every time.

/build/tmp/work/corei7-64-wrs-linux/mariadb/10.5.4-r0/mariadb-10.5.4/sql/item_cmpfunc.cc:37:10: fatal error: pcre2.h: No such file or directory                                                                                  
   37 | #include "pcre2.h"                 /* pcre2 header file */



 Comments   
Comment by Mingli-Yu [ 2020-08-03 ]

Notice there is below logic in sql/CMakeLists.txt, it should gurantee the pcre2 built first and then start to build sql, but through the error info, seems the pcre2 build doesn't complete when begin to build sql.

TARGET_LINK_LIBRARIES(sql
  mysys mysys_ssl dbug strings vio pcre2-8
  tpool
  ${LIBWRAP} ${LIBCRYPT} ${LIBDL} ${CMAKE_THREAD_LIBS_INIT}
  ${SSL_LIBRARIES}
  ${LIBSYSTEMD})
 
IF(TARGET pcre2)
  ADD_DEPENDENCIES(sql pcre2)
ENDIF()

BTW, I notice below logic in build/sql/CMakeFiles/sql.dir/depend.make in a successfull build env.

sql/CMakeFiles/sql.dir/item_cmpfunc.cc.o: extra/pcre2/src/pcre2-build/pcre2.h

But in the failed build env, the file build/sql/CMakeFiles/sql.dir/depend.make only contains below two lines:

\# Empty dependencies file for sql.
 \# This may be replaced when dependencies are built.

Comment by Sergei Golubchik [ 2020-08-03 ]

supposedly it happens because pcre2 build in cmake/pcre.cmake does not specify that pcre2.h is generated in that build, so the build tool might think that pcre2.h would exist even before pcre2 libraries are built.

Mingli-Yu, what cmake generator (make? ninja? something else?) do you use and how do you invoke it? (make? ninja? cmake --build?)

Comment by Mingli-Yu [ 2020-08-04 ]

Hi Sergei Golubchik,

The cmake generator is make in my env.

Is it possible to define the dependency for pcre2.h in sql/CMakeLists.txt?

BTW, what's the difference for -DWITH_PCRE=system and -DWITH_PCRE=auto? Anyway I switch to use -DWITH_PCRE=system to avoid the occasional build failure.

Thanks,

Comment by Sergei Golubchik [ 2020-08-08 ]

system will make the build to use the system pcre2. The build will fail if system pcre2 isn't usable.
bundled will download, build pcre2, and link it statically into mariadb.
auto will try to use system, but if it's unusable will fall back to bundled, instead of aborting the build.

Yes, it's possible to specify the dependency explicitly. cmake needs to know that

  • some files in sql/ need pcre2.h
  • pcre2 target generates pcre2.h

I suspect cmake can detect the former automatically, but it definitely should be told about the latter.

Comment by Alexander Miloslavskiy [ 2021-07-10 ]

To my understanding, the problem is that libmysqld\CMakeLists.txt doesn't depend on pcre2.

This fixes it for me:

IF(TARGET pcre2)
	ADD_DEPENDENCIES(sql_embedded pcre2)
ENDIF()

The problem is reproducible when using bundled PCRE from a very clean repo.

Comment by Alexander Miloslavskiy [ 2021-07-10 ]

Note that there are two projects that use item_cmpfunc.cc: libmysqld\CMakeLists.txt and sql\CMakeLists.txt, and only the latter depends on pcre2. Therefore, depending on which is built first, it will either fail or succeed.

Comment by Vladislav Vaintroub [ 2021-09-28 ]

reproducible when building sql_embedded only, e.g

ninja sql_embedded

Any target should be buildable by itself, and not depend on some order buildtool chooses. Here, dependencies are broken

wlad@workpc:~/10.5/xxx$ ninja sql_embedded
[218/381] Building CXX object libmysqld/CMakeFiles/sql_embedded.dir/__/sql/item_cmpfunc.cc.o
FAILED: libmysqld/CMakeFiles/sql_embedded.dir/__/sql/item_cmpfunc.cc.o
/usr/bin/c++  -DDBUG_TRACE -DEMBEDDED_LIBRARY -DHAVE_CONFIG_H -DHAVE_OPENSSL -DMYSQL_SERVER -D_FILE_OFFSET_BITS=64 -I../wsrep-lib/include -I../wsrep-lib/wsrep-API/v26 -Iinclude -I../include -I../libmysqld -I../sql -I../tpool -Isql -Iextra/pcre2/src/pcre2-build -Iextra/pcre2/src/pcre2/src -I../zlib -Izlib -pie -fPIC -fstack-protector --param=ssp-buffer-size=4 -O2 -g -DNDEBUG -fno-omit-frame-pointer -D_FORTIFY_SOURCE=2 -DDBUG_OFF -Wall -Wextra -Wformat-security -Wno-format-truncation -Wno-init-self -Wno-nonnull-compare -Wno-unused-parameter -Woverloaded-virtual -Wnon-virtual-dtor -Wvla -Wwrite-strings   -fPIC -std=gnu++11 -MD -MT libmysqld/CMakeFiles/sql_embedded.dir/__/sql/item_cmpfunc.cc.o -MF libmysqld/CMakeFiles/sql_embedded.dir/__/sql/item_cmpfunc.cc.o.d -o libmysqld/CMakeFiles/sql_embedded.dir/__/sql/item_cmpfunc.cc.o -c ../sql/item_cmpfunc.cc
../sql/item_cmpfunc.cc:37:10: fatal error: pcre2.h: No such file or directory
   37 | #include "pcre2.h"                 /* pcre2 header file */
      |          ^~~~~~~~~

Generated at Thu Feb 08 09:21:25 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.