[MCOL-1256] Usage of columnstore.xml in CS Bulk Loader plugin: Usability concerns Created: 2018-03-09  Updated: 2023-11-21  Resolved: 2018-03-16

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: None
Fix Version/s: Icebox

Type: New Feature Priority: Minor
Reporter: Elena Kotsinova (Inactive) Assignee: David Thompson (Inactive)
Resolution: Duplicate Votes: 0
Labels: usability


 Description   

The usage of the Columnstore.xml file in CS Bulk Loader plugin raises the following usability concerns:

Use case:
The customer wants to run several ETLs based on pentaho jobs on one server dedicated for this.
Every different ETL ( i.e. job) will connect to a different ColumnStore server.
It is required to use variables during design of the jobs to achieve this.
Variables are stored in kettle.properties file and this is the place where modifications can be done if they are needed.
For example if database used by job has been moved to other server, the customer easily can modify only kettle.properties file on the ETL server.
This kind of modifications related to maintenance of the production servers don't require developers intervention.
I.e. it is no needed to edit the job in PDI design tool and correct the connections and other configurations.

First usability issue:
One of usefull requerement for CS Bulk Loader in this case will be to work with variable defined for path of the columnstore.sml file, because of the easear maintanance.
This exact requrement is not met - issue is reported MCOL-1255

Second usability issue:
The columnstore.xml file has the same name for every ColumnStore server.
When Operation team setup ETL server with more than one jobs, they must copy xml file from every CS server, rename it or put it in different location in order to distinguish them.
And in case MCOL-1255 is fixed, ops team must update kettle.properties with correct paths. Over wise the path to the xml file must be setup during design which means the designer must know the future production setup.
All this setup produces higher level of risk of human errors and higher level on maintenance cost.

There is an ugly workaround - to modify transformation file and update value in tag <columnStoreXML>



 Comments   
Comment by David Thompson (Inactive) [ 2018-03-12 ]

On the 2nd issue, there is nothing stopping you from having differently named files per environment such as ColumnStoreStaging.xml and this is what i'd recommend. You'd of course have to rename the file from the original system. Will discuss first issue on 1255.

Comment by Elena Kotsinova (Inactive) [ 2018-03-13 ]

Fixing the 1255 in a right way (allow variable for file path) will improve a lot the usability.
And with very clear user guide I think this will work.

The ideal variant will be the information about the CS topology to be retrieved from UM, so adapter will use only DB connection setup.

Comment by Jens Röwekamp (Inactive) [ 2018-03-16 ]

fixed in MCOL-1255

Generated at Thu Feb 08 02:27:23 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.