[MCOL-1739] Split mcsapi installation into different packages for C++, Java and Python Created: 2018-09-21  Updated: 2023-10-26  Resolved: 2018-11-27

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: 1.1.6
Fix Version/s: 1.2.2

Type: New Feature Priority: Major
Reporter: Jens Röwekamp (Inactive) Assignee: Zdravelina Sokolovska (Inactive)
Resolution: Fixed Votes: 0
Labels: packagechange, relnote
Environment:

CentOS 7, Debian9, Debian 8, Ubuntu 16.04, Ubuntu 18.04,


Issue Links:
Blocks
blocks MCOL-1740 mcsimport - depend on mcsapi only Closed
PartOf
is part of MCOL-1889 mcs 1.2.2 release notes Closed
Relates
relates to MCOL-1744 Remove unnecessary CentOS 7 mcsapi pa... Closed
relates to MCOL-1988 Ubuntu 18 - mcsapi broken dependencie... Closed
Sprint: 2018-20

 Description   

Currently we have one installation packet for mcsapi, javamcsapi, and pymcsapi.

Thus, a customer needs to meet all dependencies (Python 2.7, Python 3, Java ...) to install the packet even though he might just wants to use one component.

The same problem arises for the data-adapters build on top of mcsapi.

Therefore, it would make sense to split the installation from one packet into individual packets for mcsapi, javamcsapi, pymcsapi, Spark Scala connector and Spark Python connector.

 Comments   
Comment by Jens Röwekamp (Inactive) [ 2018-11-13 ]

The Debian / Ubuntu / CentOS packages of mariadb-columnstore-api are now split into its components with dependencies set between each other.
On Windows there is still one MSI installer but with options to install each component separately.

On Linux we now have packages for:

  • mariadb-columnstore-api-cpp (the base cpp shared libraries)
  • mariadb-columnstore-api-cpp_devel (the cpp header files and examples for development)
  • mariadb-columnstore-api-java (the Java shared libraries for usage and development)
  • mariadb-columnstore-api-python (the Python 2.7 shared libraries for usage and development)
  • mariadb-columnstore-api-python3 (the Python 3 shared libraries for usage and development)
  • mariadb-columnstore-api-spark (the Spark ColumnStoreExporter for Scala)
  • mariadb-columnstore-api-pyspark (the PySpark ColumnStoreExporter for Python 2.7)
  • mariadb-columnstore-api-pyspark3 (the PySpark ColumnStoreExporter for Python 3)

@ben.thompson I don't know if we have to apply changes to buildbot

I further don't know how this change impacts other projects. (i.e. data-adapters, mcsimport & dbaas container) We might have to update these projects as well to use the new packages.

For QA

  • Do an install test with the new packages on Debian / Ubuntu / CentOS and Windows
  • Further test if the dependencies are automatically resolved when installing from our repository (e.g. try just to install mariadb-columnstore-api-spark via apt and yum and check that mariadb-columnstore-api-java and mariadb-columnstore-api-cpp are also installed)
Comment by Zdravelina Sokolovska (Inactive) [ 2018-11-27 ]

dependencies from mcsapi are resolved automatically through mariadb-columnstore-api bundle packages

[root@localhost ~]# yum list available mariadb-columnstore-*
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile

base: mirrors.neterra.net
epel: mirrors.nav.ro
extras: mirrors.neterra.net
ovirt-4.2: mirror.slu.cz
ovirt-4.2-epel: mirrors.nav.ro
updates: mirrors.neterra.net
Available Packages
mariadb-columnstore-api-cpp_devel.x86_64 1.2.2-1 mariadb-columnstore-api
mariadb-columnstore-api-java.x86_64 1.2.2-1 mariadb-columnstore-api
mariadb-columnstore-api-pyspark.x86_64 1.2.2-1 mariadb-columnstore-api
mariadb-columnstore-api-pyspark3.x86_64 1.2.2-1 mariadb-columnstore-api
mariadb-columnstore-api-python.x86_64 1.2.2-1 mariadb-columnstore-api
mariadb-columnstore-api-python3.x86_64 1.2.2-1 mariadb-columnstore-api
mariadb-columnstore-api-spark.x86_64 1.2.2-1 mariadb-columnstore-api
mariadb-columnstore-client.x86_64 1.2.2-1.el7.centos mariadb-columnstore-1.2.2
mariadb-columnstore-common.x86_64 1.2.2-1.el7.centos mariadb-columnstore-1.2.2
mariadb-columnstore-data-adapters-avro-kafka-adapter.x86_64 1.2.2-1 mariadb-columnstore-data-adapters
mariadb-columnstore-data-adapters-maxscale-cdc-adapter.x86_64 1.2.2-1 mariadb-columnstore-data-adapters
mariadb-columnstore-gssapi-server.x86_64 1.2.2-1.el7.centos mariadb-columnstore-1.2.2
mariadb-columnstore-libs.x86_64 1.2.2-1 mariadb-columnstore-1.2.2
mariadb-columnstore-platform.x86_64 1.2.2-1 mariadb-columnstore-1.2.2
mariadb-columnstore-rocksdb-engine.x86_64 1.2.2-1.el7.centos mariadb-columnstore-1.2.2
mariadb-columnstore-server.x86_64 1.2.2-1.el7.centos mariadb-columnstore-1.2.2
mariadb-columnstore-shared.x86_64 1.2.2-1.el7.centos mariadb-columnstore-1.2.2
mariadb-columnstore-storage-engine.x86_64 1.2.2-1 mariadb-columnstore-1.2.2
mariadb-columnstore-tokudb-engine.x86_64 1.2.2-1.el7.centos mariadb-columnstore-1.2.2
mariadb-columnstore-tools.x86_64

[root@localhost ~]# yum install mariadb-columnstore-tools
 
Loaded plugins: fastestmirror
 
Loading mirror speeds from cached hostfile
 
 
 
    base: mirrors.neterra.net
 
    epel: mirrors.nav.ro
 
    extras: mirrors.neterra.net
 
    ovirt-4.2: mirror.slu.cz
 
    ovirt-4.2-epel: mirrors.nav.ro
 
    updates: mirrors.neterra.net
 
    Resolving Dependencies
 
    --> Running transaction check
 
    ---> Package mariadb-columnstore-tools.x86_64 0:1.2.2-1 will be installed
 
    --> Processing Dependency: mariadb-columnstore-api-cpp >= 1.2 for package: mariadb-columnstore-tools-1.2.2-1.x86_64
 
    --> Processing Dependency: libmcsapi.so.1()(64bit) for package: mariadb-columnstore-tools-1.2.2-1.x86_64
 
    --> Running transaction check
 
    ---> Package mariadb-columnstore-api-cpp.x86_64 0:1.2.2-1 will be installed
 
    --> Finished Dependency Resolution
 
 
 
Dependencies Resolved
 
 
 
===================================================================================================================================================================================================================================
 
Package Arch Version Repository Size
 
===================================================================================================================================================================================================================================
 
Installing:
 
mariadb-columnstore-tools x86_64 1.2.2-1 Local-Install 232 k
 
Installing for dependencies:
 
mariadb-columnstore-api-cpp x86_64 1.2.2-1 Local-Install 821 k
 
 
 
Transaction Summary
 
===================================================================================================================================================================================================================================
 
Install 1 Package (+1 Dependent package)
 
 
 
Total download size: 1.0 M
 
Installed size: 4.5 M
 
Is this ok [y/d/N]: y
 
Downloading packages:
 
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 
Total 97 MB/s | 1.0 MB 00:00:00
 
Running transaction check
 
Running transaction test
 
Transaction test succeeded
 
Running transaction
 
Installing : mariadb-columnstore-api-cpp-1.2.2-1.x86_64 1/2
 
Installing : mariadb-columnstore-tools-1.2.2-1.x86_64 2/2
 
Verifying : mariadb-columnstore-tools-1.2.2-1.x86_64 1/2
 
Verifying : mariadb-columnstore-api-cpp-1.2.2-1.x86_64 2/2
 
 
 
Installed:
 
mariadb-columnstore-tools.x86_64 0:1.2.2-1
 
 
 
Dependency Installed:
 
mariadb-columnstore-api-cpp.x86_64 0:1.2.2-1
 
 
 
Complete!
 
 
 
as well
 
 
 
[root@localhost ~]# yum install mariadb-columnstore-api-spark
 
Loaded plugins: fastestmirror
 
Loading mirror speeds from cached hostfile
 
 
 
    base: mirrors.neterra.net
 
    epel: mirrors.nav.ro
 
    extras: mirrors.neterra.net
 
    ovirt-4.2: mirror.slu.cz
 
    ovirt-4.2-epel: mirrors.nav.ro
 
    updates: mirrors.neterra.net
 
    Resolving Dependencies
 
    --> Running transaction check
 
    ---> Package mariadb-columnstore-api-spark.x86_64 0:1.2.2-1 will be installed
 
    --> Processing Dependency: mariadb-columnstore-api-java = 1.2.2 for package: mariadb-columnstore-api-spark-1.2.2-1.x86_64
 
    --> Running transaction check
 
    ---> Package mariadb-columnstore-api-java.x86_64 0:1.2.2-1 will be installed
 
    --> Processing Dependency: mariadb-columnstore-api-cpp = 1.2.2 for package: mariadb-columnstore-api-java-1.2.2-1.x86_64
 
    --> Processing Dependency: libmcsapi.so.1()(64bit) for package: mariadb-columnstore-api-java-1.2.2-1.x86_64
 
    --> Running transaction check
 
    ---> Package mariadb-columnstore-api-cpp.x86_64 0:1.2.2-1 will be installed
 
    --> Finished Dependency Resolution
 
 
 
Dependencies Resolved
 
 
 
===================================================================================================================================================================================================================================
 
Package Arch Version Repository Size
 
===================================================================================================================================================================================================================================
 
Installing:
 
mariadb-columnstore-api-spark x86_64 1.2.2-1 Local-Install 25 k
 
Installing for dependencies:
 
mariadb-columnstore-api-cpp x86_64 1.2.2-1 Local-Install 821 k
 
mariadb-columnstore-api-java x86_64 1.2.2-1 Local-Install 216 k
 
 
 
Transaction Summary
 
===================================================================================================================================================================================================================================
 
Install 1 Package (+2 Dependent packages)
 
 
 
Total download size: 1.0 M
 
Installed size: 4.5 M
 
Is this ok [y/d/N]: y
 
Downloading packages:
 
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 
Total 52 MB/s | 1.0 MB 00:00:00
 
Running transaction check
 
Running transaction test
 
Transaction test succeeded
 
Running transaction
 
Installing : mariadb-columnstore-api-cpp-1.2.2-1.x86_64 1/3
 
Installing : mariadb-columnstore-api-java-1.2.2-1.x86_64 2/3
 
Installing : mariadb-columnstore-api-spark-1.2.2-1.x86_64 3/3
 
Verifying : mariadb-columnstore-api-spark-1.2.2-1.x86_64 1/3
 
Verifying : mariadb-columnstore-api-cpp-1.2.2-1.x86_64 2/3
 
Verifying : mariadb-columnstore-api-java-1.2.2-1.x86_64 3/3
 
 
 
Installed:
 
mariadb-columnstore-api-spark.x86_64 0:1.2.2-1
 
 
 
Dependency Installed:
 
mariadb-columnstore-api-cpp.x86_64 0:1.2.2-1 mariadb-columnstore-api-java.x86_64 0:1.2.2-1
 
 
 
Complete!
 
 
 
also the mcs maxscale adapter , although of the confusion package name [mariadb-columnstore-maxscale-cdc-adapters-1.2.2-1-x86_64-centos7.rpm] , is installed successfully resolving dependency from mariadb-columnstore-api-cpp
 
 
 
[root@localhost ~]# yum install mariadb-columnstore-maxscale-cdc-adapters
 
Loaded plugins: fastestmirror
 
Loading mirror speeds from cached hostfile
 
 
 
    base: mirrors.neterra.net
 
    epel: mirrors.nav.ro
 
    extras: mirrors.neterra.net
 
    ovirt-4.2: mirror.slu.cz
 
    ovirt-4.2-epel: mirrors.nav.ro
 
    updates: mirrors.neterra.net
 
    No package mariadb-columnstore-maxscale-cdc-adapters available.
 
    Error: Nothing to do
 
    [root@localhost ~]#
 
 
 
 
 
[root@localhost ~]# yum install mariadb-columnstore-data-adapters-maxscale-cdc-adapter
 
Loaded plugins: fastestmirror
 
Loading mirror speeds from cached hostfile
 
 
 
    base: mirrors.neterra.net
 
    epel: mirrors.nav.ro
 
    extras: mirrors.neterra.net
 
    ovirt-4.2: mirror.slu.cz
 
    ovirt-4.2-epel: mirrors.nav.ro
 
    updates: mirrors.neterra.net
 
    Resolving Dependencies
 
    --> Running transaction check
 
    ---> Package mariadb-columnstore-data-adapters-maxscale-cdc-adapter.x86_64 0:1.2.2-1 will be installed
 
    --> Processing Dependency: libmcsapi.so.1()(64bit) for package: mariadb-columnstore-data-adapters-maxscale-cdc-adapter-1.2.2-1.x86_64
 
    --> Running transaction check
 
    ---> Package mariadb-columnstore-api-cpp.x86_64 0:1.2.2-1 will be installed
 
    --> Finished Dependency Resolution
 
 
 
Dependencies Resolved
 
 
 
===================================================================================================================================================================================================================================
 
Package Arch Version Repository Size
 
===================================================================================================================================================================================================================================
 
Installing:
 
mariadb-columnstore-data-adapters-maxscale-cdc-adapter x86_64 1.2.2-1 mariadb-columnstore-data-adapters 209 k
 
Installing for dependencies:
 
mariadb-columnstore-api-cpp x86_64 1.2.2-1 mariadb-columnstore-api 821 k
 
 
 
Transaction Summary
 
===================================================================================================================================================================================================================================
 
Install 1 Package (+1 Dependent package)
 
 
 
Total download size: 1.0 M
 
Installed size: 4.2 M
 
Is this ok [y/d/N]: y
 
Downloading packages:
 
(1/2): mariadb-columnstore-maxscale-cdc-adapters-1.2.2-1-x86_64-centos7.rpm | 209 kB 00:00:00
 
(2/2): mariadb-columnstore-api-1.2.2-1-x86_64-centos7-cpp.rpm | 821 kB 00:00:01
 
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 
Total 999 kB/s | 1.0 MB 00:00:01
 
Running transaction check
 
Running transaction test
 
Transaction test succeeded
 
Running transaction
 
Installing : mariadb-columnstore-api-cpp-1.2.2-1.x86_64 1/2
 
Installing : mariadb-columnstore-data-adapters-maxscale-cdc-adapter-1.2.2-1.x86_64 2/2
 
Verifying : mariadb-columnstore-data-adapters-maxscale-cdc-adapter-1.2.2-1.x86_64 1/2
 
Verifying : mariadb-columnstore-api-cpp-1.2.2-1.x86_64 2/2
 
 
 
Installed:
 
mariadb-columnstore-data-adapters-maxscale-cdc-adapter.x86_64 0:1.2.2-1
 
 
 
Dependency Installed:
 
mariadb-columnstore-api-cpp.x86_64 0:1.2.2-1

Generated at Thu Feb 08 02:31:00 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.