[MCOL-4471] Properly document each and every build dependency of ColumnStore Created: 2020-12-26  Updated: 2021-04-03

Status: Open
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: None
Fix Version/s: Icebox

Type: Bug Priority: Major
Reporter: Otto Kekäläinen Assignee: Roman
Resolution: Unresolved Votes: 0
Labels: None


 Description   

People seem to be spending a lot of time on ColumnStore build & packaging issues. Some of the time is wasted because the build dependencies of ColumnStore are not clearly defined.

The building phase is marked TODO in https://github.com/mariadb-corporation/mariadb-columnstore-engine/blob/develop/BUILD.md

Source code contains a binary (sic!) version of a specific libjemalloc.so (filed at https://jira.mariadb.org/browse/MDEV-24124).

The CI pipeline at https://github.com/mariadb-corporation/mariadb-columnstore-engine/blob/develop/.drone.jsonnet does a lot of slimming down of the build flags.

These are signs that the current state is quite unclear.

It should be documented:

  • What exactly are the external build dependencies? On a clean system, what should be installed in addition to the MariaDB Server and ColumStore sources?
  • How does ColumnStore depend on MariaDB Server?
  • Is it enough to have the sources available, or does it actually need to build MariaDB fully or partially?
  • If partially, which plugins and storage engines must be built?
  • Which of the following MariaDB Server CMakeLists.txt includes need to run?

INCLUDE(misc)
 
INCLUDE(mysql_version)
 
INCLUDE(cpack_source_ignore_files)
 
INCLUDE(install_layout)
 
INCLUDE(submodules)
 
 
 
# Add macros
 
INCLUDE(character_sets)
 
INCLUDE(cpu_info)
 
INCLUDE(zlib)
 
INCLUDE(ssl)
 
INCLUDE(readline)
 
INCLUDE(libutils)
 
INCLUDE(dtrace)
 
INCLUDE(pcre)
 
INCLUDE(ctest)
 
INCLUDE(plugin)
 
INCLUDE(install_macros)
 
INCLUDE(systemd)
 
INCLUDE(mysql_add_executable)
 
INCLUDE(symlinks)
 
INCLUDE(compile_flags)
 
INCLUDE(pmem)



 Comments   
Comment by Roman [ 2020-12-30 ]

External dependencies are listed here and here.
As of now MCS reuses some object files from MDB they are listed in the engine's CMakeLists.txt under MARIADB_CLIENT_LIBS and MARIADB_STRING_LIBS so MCS needs some object files produced by MDB compilation.
MCS needs ARIA engine for sure but it should be built-in. Can't tell you about the full list of plugins needed.
From the list of the server's CMakeLists.txt includes we don't need: zlib, pcre. Can't tell about the other.

Comment by Otto Kekäläinen [ 2020-12-30 ]

Thanks drrtuy! The external dependencies are pretty well already mapped in debian/control. Thanks for shedding light on the internal dependencies.

I assigned to you so that you can document the build in some permanent place. You already have a stub at https://github.com/mariadb-corporation/mariadb-columnstore-engine/blob/develop/BUILD.md, why not finish it?

Comment by Otto Kekäläinen [ 2020-12-30 ]

Related:

Comment by Roman [ 2020-12-30 ]

Unfortunately the team can't address documentation issues now. Moreover all
of us use some kind of build automation so we are waiting for a newcomer to
help with this.

Regards,
Roman Nozdrin
ColumnStore Engineering
MariaDB Corporation

On Wed, Dec 30, 2020 at 6:33 PM Otto Kekäläinen (Jira) <jira@mariadb.org>

Comment by Otto Kekäläinen [ 2021-04-03 ]

How about now 4 months later, can you now address documentation issues and document what are the actual and real build dependencies for ColumnStore?

PR https://github.com/mariadb-corporation/mariadb-columnstore-engine/pull/1701 is now updated, but the debian/control file contents has way too many build dependencies until it is slimmed down to match the actual needs of ColumnStore.

Generated at Thu Feb 08 02:50:35 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.