[MDEV-32149] cross-compiling on Windows x64 for arm64 failed Created: 2023-09-11  Updated: 2023-09-24  Resolved: 2023-09-24

Status: Closed
Project: MariaDB Server
Component/s: Compiling
Affects Version/s: 11.3
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Luan Castro Assignee: Vladislav Vaintroub
Resolution: Duplicate Votes: 0
Labels: None
Environment:

OS: Windows 10
VS: Visual Studio 2022(17.7.2)

Attachments: Text File build.log     File cross.cmake     File import_executables.cmake    
Issue Links:
Duplicate
duplicates MDEV-25870 Windows - Fix cross-compilation for ... Closed

 Description   

Repro steps:
1. git clone --recurse-submodules https://github.com/MariaDB/server.git
2. Open "x64 Native Tools Command Prompt for VS 2022"
3. Copy import_executables.cmake and cross.cmake to \server folder.
4. mkdir build_arm64
5. cd build_arm64
6. cmake -G "Visual Studio 17 2022" -A arm64 -DCMAKE_SYSTEM_VERSION=10.0.22621.0 -DCMAKE_PREFIX_PATH=F:\tools\gnuwin32\bin -DWITHOUT_SERVER=ON -DIMPORT_EXECUTABLES=.\server\import_executables.cmake -DCMAKE_TOOLCHAIN_FILE=.\server\cross.cmake ..
7. msbuild /m /p:Platform=arm64 /p:Configuration=Release MariaDB.sln /t:Rebuild

*Error info:*
 
       "F:\gitP\server\build_arm64\MariaDB.sln" (Rebuild target) (1) ->
 
       "F:\gitP\server\build_arm64\ALL_BUILD.vcxproj.metaproj" (Rebuild target) (2) ->
 
       "F:\gitP\server\build_arm64\extra\GenError.vcxproj.metaproj" (Rebuild target) (3) ->
 
       "F:\gitP\server\build_arm64\extra\GenError.vcxproj" (Rebuild target) (122) ->
 
       (CustomBuild target) -> 
         C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(249,5): error MSB8066: Custom build for 'F:\gitP\server\build_arm64\CMakeFiles\2063d0f4b95f7ec35589a07d0ac970fb\mysqld_error.h.tmp.rule;F:\gitP\server\build_arm64\CMakeFiles\4bebb7941980b53edaa8c4f4a2916266\GenError.rule;F:\gitP\server\extra\CMakeLists.txt' exited with code 3. [F:\gitP\server\build_arm64\extra\GenError.vcxproj]
 
 
 
 
 
       "F:\gitP\server\build_arm64\MariaDB.sln" (Rebuild target) (1) ->
 
       "F:\gitP\server\build_arm64\ALL_BUILD.vcxproj.metaproj" (Rebuild target) (2) ->
 
       "F:\gitP\server\build_arm64\unittest\mysys\aes-t.vcxproj.metaproj" (Rebuild target) (7) ->
 
       "F:\gitP\server\build_arm64\dbug\dbug.vcxproj.metaproj" (Rebuild target) (8) ->
 
       "F:\gitP\server\build_arm64\strings\strings.vcxproj.metaproj" (Rebuild target) (9) ->
 
       "F:\gitP\server\build_arm64\mysys\mysys.vcxproj.metaproj" (Rebuild target) (10) ->
 
       "F:\gitP\server\build_arm64\strings\GenUnicodeDataSource.vcxproj.metaproj" (Rebuild target) (11) ->
 
       "F:\gitP\server\build_arm64\strings\GenUnicodeDataSource.vcxproj" (Rebuild target) (124) ->
 
         C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(249,5): error MSB8066: Custom build for 'F:\gitP\server\build_arm64\CMakeFiles\8186c3645bb3b42ff691fd0d0f5a1e5c\ctype-uca1400data.h.rule;F:\gitP\server\build_arm64\CMakeFiles\69538b2d008a4940d9c1672cb70131b2\GenUnicodeDataSource.rule;F:\gitP\server\strings\CMakeLists.txt' exited with code 3. [F:\gitP\server\build_arm64\strings\GenUnicodeDataSource.vcxproj]
 
 
 
 
 
       "F:\gitP\server\build_arm64\MariaDB.sln" (Rebuild target) (1) ->
 
       "F:\gitP\server\build_arm64\ALL_BUILD.vcxproj.metaproj" (Rebuild target) (2) ->
 
       "F:\gitP\server\build_arm64\scripts\GenFixPrivs.vcxproj.metaproj" (Rebuild target) (4) ->
 
       "F:\gitP\server\build_arm64\scripts\GenFixPrivs.vcxproj" (Rebuild target) (125) ->
 
         C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(249,5): error MSB8066: Custom build for 'F:\gitP\server\build_arm64\CMakeFiles\d697fe595eaf5283e987fce47d1bc414\mariadb_fix_privilege_tables_sql.c.rule;F:\gitP\server\build_arm64\CMakeFiles\dcf6a1e5c7529bdb4636e4f0605047a1\GenFixPrivs.rule;F:\gitP\server\scripts\CMakeLists.txt' exited with code 3. [F:\gitP\server\build_arm64\scripts\GenFixPrivs.vcxproj]



 Comments   
Comment by Daniel Black [ 2023-09-12 ]

Hi,

As we don't have cross compiled builds in CI we don't officially support it.

That said, looking at a cross compulation in debian - http://crossqa.debian.net/build/mariadb_1:10.11.4-1_s390x_20230911041536.log - it appears that a native build of the executables in import_executables needs to happen before the attempt at cross compulation. Mapping what Debian does it might require:

cd host
 
cmake  -DCMAKE_BUILD_TYPE=None  ..
 
cmake --build . --target=import_executables

Also to get ARM cross compile working in mysys/CMakeLists.txt an:

ELSEIF(MSVC_ARM64)
 
  SET(MYSYS_SOURCES ${MYSYS_SOURCES} crc32/crc32_arm64.c)
 
....

to build with mysys/crc32/crc32_arm64.c, and that file might need a constant implementation of

#if defined _M_ARM64
 
 
 
 int crc32_aarch64_available(void) { return 1; }
 
 const char *crc32c_aarch64_available(void)
 
  {
 
    return "Using ARMv8 crc32 + pmull instructions"; }
 
#endif

Patches and edits to https://mariadb.com/kb/en/Building_MariaDB_on_Windows/ most welcome.

Comment by Vladislav Vaintroub [ 2023-09-13 ]

I got this cross-compilation working couple of years ago, though never actually had the hardware to test the result.
It still works, it is just so that the recent version of CMake do not set CMAKE_CROSSCOMPILING when ARM64 is built on AMD64. I'm not sure why CMake folk changed that, it used to automatically determine cross-compilation, and it was rather convenient. But, the new behavior is documented https://cmake.org/cmake/help/latest/variable/CMAKE_CROSSCOMPILING.html

Anyway, it is simple to fix, you just provide additional parameter -DCMAKE_SYSTEM_NAME=Windows , as they suggest in CMAKE_CROSSCOMPILING docs

That works for me

   cmake .. -A ARM64 -DIMPORT_EXECUTABLES=C:/work/server/xxx/import_executables.cmake  -DCMAKE_SYSTEM_NAME=Windows
 
   cmake --build . --config RelWithDebInfo --target package

and spits out mariadb-11.3.0-winarm64.zip

As I said, it was never tested here, so there is a good chance it would not work. It does not have an optimized crc32, I think that needs to be implemented once we get a chance to test it.

Comment by Daniel Black [ 2023-09-18 ]

v-luanca, how did you go? Did you get it going? Is there something we can prepare into a documentation outcome for this?

Comment by Vladislav Vaintroub [ 2023-09-24 ]

I verified that the below works OK, providing native bison is there.

From the source directory.

mkdir build_native
 
cd build_native
 
cmake ..
 
cmake --build . --config RelWithDebInfo --target import_executables -j
 
cd ..
 
mkdir build_arm64
 
cd build_arm64
 
cmake .. -A ARM64 -DIMPORT_EXECUTABLES=../build_native/import_executables.cmake  -DCMAKE_SYSTEM_NAME=Windows
 
cmake --build . --config RelWithDebInfo -j

If one builds with Release config, it would be necessary to replace RelWithDebInfo with Release above, in both cases where it is used , i.e for both the native and and the cross build

Comment by Vladislav Vaintroub [ 2023-09-24 ]

So, summary : it works fine , but "repro steps" do not set CMAKE_SYSTEM_NAME which is necessary in the recent versions of CMake to recognize cross-compilation (even if CMAKE_SYSTEM_NAME does not change, it needs to be set).

Comment by Vladislav Vaintroub [ 2023-09-24 ]

In fact, MDEV-25870 , that was fixed 2 years ago, already contained working description on "how to compile on ARM64"

Generated at Thu Feb 08 10:29:12 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.