[MCOL-1255] PDI CS Bulk Load plugin can't work with variable defined in kettle.properties file Created: 2018-03-09  Updated: 2023-10-26  Resolved: 2018-03-14

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: None
Fix Version/s: 1.1.4

Type: Bug Priority: Major
Reporter: Elena Kotsinova (Inactive) Assignee: Elena Kotsinova (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Sprint: 2018-05, 2018-06

 Description   

CS Bulk Loader can't work with variable for ColumnStore XML file path defined in the kettle.properties.

1. Open PDI spoon tool and open test.kjb
2. Open export-to-mariadb transformation object.
3. In Spoon main menu select Edit > Edit the kettle.properties file
4. Click right mouse button and select one of Insert options.
5. In new empty line add Variable name = DATABASE_CS_XML and Value = /home/pentaho/Downloads/test-job/Columnstore.xml (this is only an example for correct path to the file)
6. In export-to-mariadb transformation > MariaDB ColumnStore Bulk Loader component > Settings change ColumnStore XML field to use DATABASE_CS_XML variable
7. Save all changes
8. Run the test.kjb

Result:
Job fails with error

2018/03/09 16:25:11 - MariaDB ColumnStore Bulk Loader.0 - ERROR (version 8.0.0.0-28, build 8.0.0.0-28 from 2017-11-05 07.27.50 by buildguy) : The ColumnStoreDriver couldn't be instantiated.
 
2018/03/09 16:25:11 - MariaDB ColumnStore Bulk Loader.0 - ERROR (version 8.0.0.0-28, build 8.0.0.0-28 from 2017-11-05 07.27.50 by buildguy) : Error initializing step [MariaDB ColumnStore Bulk Loader]

If user tries to edit the bulk loader component again the following message is displayed:
Please select a valid Columnstore.xml file to proceed.

Expected:
Job finishes successfully if variable for columstore.xml file path is used.

 Comments   
Comment by Jens Röwekamp (Inactive) [ 2018-03-09 ]

Removed the option to use variables for setting the configuration path of Columnstore.xml

Comment by Elena Kotsinova (Inactive) [ 2018-03-12 ]

Please review this design decision.

This will remove possibilities for easy maintenance (related to MCOL-1256).

Comment by Andrew Hutchings (Inactive) [ 2018-03-12 ]

Since I know nothing about PDI, Kettle or Java I'm not sure exactly what I'm supposed to be reviewing here? Also I am not seeing what decision I'm being asked to make?

Comment by David Thompson (Inactive) [ 2018-03-12 ]

This was working by accident previously, need to think through how to use this correctly. The question would be whether distinctly named files is a solution to your overall problem in 1256?

Comment by Elena Kotsinova (Inactive) [ 2018-03-13 ]

Possibility to use variable for path of the columnstore.xml file location will improve portability and maintainability of the Pentaho jobs packages.
Thus change in production environment setup will be easily configured in only one place - kettle.properties file. No need for developers involvement to fix configuration parameter inside the jobs - this is like to hard-code environment parameters in the code.
Because of this it is better to fix adapter to use variables rather than to remove this option from the adapter.

Comment by David Thompson (Inactive) [ 2018-03-13 ]

Please proceed with the change to properly support variables to support the use case Elena has above.

Comment by Jens Röwekamp (Inactive) [ 2018-03-13 ]

proceeded as instructed, variables can now be used to define the Columnstore.xml path.

Comment by Elena Kotsinova (Inactive) [ 2018-03-14 ]

Verified for build 1.1.4 form 14 March 2018

Generated at Thu Feb 08 02:27:23 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.