[MCOL-4483] Consolidate Log Location Created: 2020-12-30  Updated: 2021-03-19  Resolved: 2021-03-04

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: None
Fix Version/s: 5.5.2

Type: Bug Priority: Critical
Reporter: Kathryn Sizemore Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 1
Labels: UX

Attachments: Text File missing_bad.txt    
Issue Links:
PartOf
includes MCOL-571 CS provides incorrect path in logs to... Closed
includes MCOL-4494 CMAPI should log to standard columnst... Closed
Relates
relates to MCOL-3683 Add optional errordir for .bad and .... Closed
Epic Link: ColumnStore Message & Logging Improvements
Sprint: 2021-1, 2021-2, 2021-3, 2021-4

 Description   

ColumnStore logs are located in /tmp, /var/lib/columnstore/data/bulk/log, /var/log/mariadb/columnstore, etc. cmapi doesn't even have a log file. You have to use the journalctl utility. Can this be simplified and standardized into one location for all processes.

Additionally, when using cpimport the .bad file does not get written to any folder. It instead throws a 'permissions' error into the job log.

 Comments   
Comment by Ben Thompson (Inactive) [ 2021-01-15 ]

WORKAROUND FOR MISSING .bad FILES:

set the variable "WorkingDIrectory" in /usr/lib/systemd/system/mcs-writeengineserver.service

[Service]
Type=forking
WorkingDirectory=/home/mysql/

make sure the user running the writeengineserver process (mysql default) has write access to this directory

this is just going to have a file temporarily while cpimport.bin creates and sends the data for the bad file to cpimport on PM1

after updating the file restart WES process with
systemctl daemon-reload
systemctl restart mcs-writeengineserver.service

Comment by Daniel Lee (Inactive) [ 2021-02-22 ]

Build tested: 5.5.2 (Drone #1718)

Tested on multi-node cluster, and different cpimport modes
Tested cpimport simple mode (cpimport dbname tablename datafile)
Tested cpimport batch mode using colxml and job number

/var/log/mariadb/columnstore/data is now being used in place of /var/lib/columnstore/data.
/var/log/mariadb/columnstore/cpimport is now used for cpimport logs.

I observed the following:

/tmp/columnstore_tmp_file directory is still being used or some log files
/var/lib/columnstor/data still exists. It should not be created any more.
/var/log/mariadb/columnstore/data/bulk/log should not be created and/or used.

Comment by Ben Thompson (Inactive) [ 2021-02-22 ]

/tmp/columnstore_tmp_files is still getting some tmp files as expected it was not solely for cpimport logging there might be the output of a brm related command going there still but its not really useful so left it alone in tmp dir.
/var/lib/columnstore/data is not exclusively for cpimport AFAIK. just leaving as existing directory for now.
/var/log/mariadb/columnstore/data/bulk/log is "used" but should be empty most of the time.

Comment by Daniel Lee (Inactive) [ 2021-02-22 ]

There are cpimport logs in /tmp/columnstore_tmp_files.
In the /var/lib/columnstore/data directory, there is the bulk sub directory. Leaving it there creates confusion. Certain users may complain about these sub directories being empty.
Same thing for /var/log/mariadb/columnstore/data/bulk/log. Since it has been migrated to /var/log/mariadb/columnstore/bulk/log, what is it being used? We should consolidate it to avoid confusion.

Comment by Daniel Lee (Inactive) [ 2021-03-04 ]

Build verified: 5.5.2 (Drone #1810)

The mentioned directories have been removed

Generated at Thu Feb 08 02:50:41 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.