[MCOL-4742] cmapi should respect dbroot path from columnstore.xml -post-install document needed Created: 2021-06-01  Updated: 2022-11-28  Resolved: 2022-11-28

Status: Closed
Project: MariaDB ColumnStore
Component/s: cmapi
Affects Version/s: None
Fix Version/s: 6.4.1

Type: Task Priority: Major
Reporter: Richard Stracke Assignee: Alan Mologorsky
Resolution: Fixed Votes: 0
Labels: cmapi

Sprint: 2021-17, 2022-22

 Description   

It seems, that dbroot paths are handled hardcoded in cmapi.

from node_manipulation.py

    etree.SubElement(sysconf_node, f"DBRoot{next_dbroot_id}").text =\
 
      f"/var/lib/columnstore/data{next_dbroot_id}"
 
    current_dbroot_id = next_dbroot_id



 Comments   
Comment by alexey vorovich (Inactive) [ 2022-03-24 ]

alan.mologorsky is that a quick fix?

Comment by Roman [ 2022-06-30 ]

The implementation shares the dbroot1 prefix path for all other dbroots.

Comment by Daniel Lee (Inactive) [ 2022-07-05 ]

Build tested: cmapi-6.4.1 (b681)

Moving /var/lib/columnstore/data1 to another location and updating /etc/columnstore/Columnstore.xml accordingly would not work.

.
 
.
 
.
 
<DBRootCount>1</DBRootCount>
 
<DBRoot1>/etc/columnstore/data1</DBRoot1>
 
<DBRMRoot>/var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves</DBRMRoot>
 
<TableLockSaveFile>/var/lib/columnstore/data1/systemFiles/dbrm/tablelocks</TableLockSaveFile>
 
.
 
.
 
.

That is because there are other entries in Columnstore.xml still referencing the original location.

<DBRMRoot>/var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves</DBRMRoot>
 
<TableLockSaveFile>/var/lib/columnstore/data1/systemFiles/dbrm/tablelocks</TableLockSaveFile>
 
<TxnIDFile>/var/lib/columnstore/data1/systemFiles/dbrm/SMTxnID</TxnIDFile>
 
<OIDBitmapFile>/var/lib/columnstore/data1/systemFiles/dbrm/oidbitmap</OIDBitmapFile>
 
<BulkRollbackDir>/var/lib/columnstore/data1/systemFiles/bulkRollback</BulkRollbackDir>

The only way to get it to work is to make a new "data1" directory in the desired location, then move /var/lib/columnstore/data1/000.dir to under the new 'data1' directory.

I checked the cmapi source code. It seems that only <dbrootn> (where n is the dbroot number). Other references are still hard coded. My opinion is that we need a better and complete solution to this hard-coded path issue.

If the purpose of the ticket is indeed to fix the hardcoded dbroot issue only, then we still have other issues. I checked the source coded and that there are still other areas that still refence to "/var/lib/columnstore/dbrootn". 

[dlee@aloha cmapi]$ grep -r '/var/lib' *
 
cmapi_server/test/CS-config-test.xml:        <DBRoot1>/var/lib/columnstore/data1</DBRoot1>
 
cmapi_server/test/CS-config-test.xml:        <DBRMRoot>/var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves</DBRMRoot>
 
cmapi_server/test/CS-config-test.xml:        <TableLockSaveFile>/var/lib/columnstore/data1/systemFiles/dbrm/tablelocks</TableLockSaveFile>
 
cmapi_server/test/CS-config-test.xml:        <TxnIDFile>/var/lib/columnstore/data1/systemFiles/dbrm/SMTxnID</TxnIDFile>
 
cmapi_server/test/CS-config-test.xml:        <OIDBitmapFile>/var/lib/columnstore/data1/systemFiles/dbrm/oidbitmap</OIDBitmapFile>
 
cmapi_server/test/CS-config-test.xml:        <BulkRoot>/var/lib/columnstore/data/bulk</BulkRoot>
 
cmapi_server/test/CS-config-test.xml:        <BulkRollbackDir>/var/lib/columnstore/data1/systemFiles/bulkRollback</BulkRollbackDir>
 
cmapi_server/test/Columnstore_apply_config.xml:        <DBRoot1>/var/lib/columnstore/data1</DBRoot1>
 
cmapi_server/test/Columnstore_apply_config.xml:        <DBRMRoot>/var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves</DBRMRoot>
 
cmapi_server/test/Columnstore_apply_config.xml:        <TableLockSaveFile>/var/lib/columnstore/data1/systemFiles/dbrm/tablelocks</TableLockSaveFile>
 
cmapi_server/test/Columnstore_apply_config.xml:		<DBRoot2>/var/lib/columnstore/data2</DBRoot2>
 
cmapi_server/test/Columnstore_apply_config.xml:        <TxnIDFile>/var/lib/columnstore/data1/systemFiles/dbrm/SMTxnID</TxnIDFile>
 
cmapi_server/test/Columnstore_apply_config.xml:        <OIDBitmapFile>/var/lib/columnstore/data1/systemFiles/dbrm/oidbitmap</OIDBitmapFile>
 
cmapi_server/test/Columnstore_apply_config.xml:        <BulkRoot>/var/lib/columnstore/data/bulk</BulkRoot>
 
cmapi_server/test/Columnstore_apply_config.xml:        <BulkRollbackDir>/var/lib/columnstore/data1/systemFiles/bulkRollback</BulkRollbackDir>
 
cmapi_server/SingleNode.xml:        <DBRoot1>/var/lib/columnstore/data1</DBRoot1>
 
cmapi_server/SingleNode.xml:        <DBRMRoot>/var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves</DBRMRoot>
 
cmapi_server/SingleNode.xml:        <TableLockSaveFile>/var/lib/columnstore/data1/systemFiles/dbrm/tablelocks</TableLockSaveFile>
 
cmapi_server/SingleNode.xml:        <TxnIDFile>/var/lib/columnstore/data1/systemFiles/dbrm/SMTxnID</TxnIDFile>
 
cmapi_server/SingleNode.xml:        <OIDBitmapFile>/var/lib/columnstore/data1/systemFiles/dbrm/oidbitmap</OIDBitmapFile>
 
cmapi_server/SingleNode.xml:        <BulkRoot>/var/lib/columnstore/data/bulk</BulkRoot>
 
cmapi_server/SingleNode.xml:        <BulkRollbackDir>/var/lib/columnstore/data1/systemFiles/bulkRollback</BulkRollbackDir>
 
cmapi_server/constants.py:MCS_DATA_PATH = '/var/lib/columnstore'
 
mcs_node_control/test/Columnstore_old.xml:        <DBRoot1>/var/lib/columnstore/data1</DBRoot1>
 
mcs_node_control/test/Columnstore_old.xml:        <DBRMRoot>/var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves</DBRMRoot>
 
mcs_node_control/test/Columnstore_old.xml:        <TableLockSaveFile>/var/lib/columnstore/data1/systemFiles/dbrm/tablelocks</TableLockSaveFile>
 
mcs_node_control/test/Columnstore_old.xml:        <TxnIDFile>/var/lib/columnstore/data1/systemFiles/dbrm/SMTxnID</TxnIDFile>
 
mcs_node_control/test/Columnstore_old.xml:        <OIDBitmapFile>/var/lib/columnstore/data1/systemFiles/dbrm/oidbitmap</OIDBitmapFile>
 
mcs_node_control/test/Columnstore_old.xml:        <BulkRoot>/var/lib/columnstore/data/bulk</BulkRoot>
 
mcs_node_control/test/Columnstore_old.xml:        <BulkRollbackDir>/var/lib/columnstore/data1/systemFiles/bulkRollback</BulkRollbackDir>
 
mcs_node_control/custom_dispatchers/container.sh:    MCS_INSTALL_PATH=/var/lib/columnstore
 
mcs_node_control/custom_dispatchers/container.sh:    MCS_INSTALL_PATH=/var/lib/columnstore
 
mcs_node_control/custom_dispatchers/container.sh:    MCS_INSTALL_PATH=/var/lib/columnstore

Generated at Thu Feb 08 02:52:38 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.