ColumnStore logs are located in /tmp, /var/lib/columnstore/data/bulk/log, /var/log/mariadb/columnstore, etc. cmapi doesn't even have a log file. You have to use the journalctl utility. Can this be simplified and standardized into one location for all processes.
Additionally, when using cpimport the .bad file does not get written to any folder. It instead throws a 'permissions' error into the job log.
make sure the user running the writeengineserver process (mysql default) has write access to this directory
this is just going to have a file temporarily while cpimport.bin creates and sends the data for the bad file to cpimport on PM1
after updating the file restart WES process with
systemctl daemon-reload
systemctl restart mcs-writeengineserver.service
Ben Thompson (Inactive)
added a comment - - edited WORKAROUND FOR MISSING .bad FILES:
set the variable "WorkingDIrectory" in /usr/lib/systemd/system/mcs-writeengineserver.service
[Service]
Type=forking
WorkingDirectory=/home/mysql/
make sure the user running the writeengineserver process (mysql default) has write access to this directory
this is just going to have a file temporarily while cpimport.bin creates and sends the data for the bad file to cpimport on PM1
after updating the file restart WES process with
systemctl daemon-reload
systemctl restart mcs-writeengineserver.service
Tested on multi-node cluster, and different cpimport modes
Tested cpimport simple mode (cpimport dbname tablename datafile)
Tested cpimport batch mode using colxml and job number
/var/log/mariadb/columnstore/data is now being used in place of /var/lib/columnstore/data.
/var/log/mariadb/columnstore/cpimport is now used for cpimport logs.
I observed the following:
/tmp/columnstore_tmp_file directory is still being used or some log files
/var/lib/columnstor/data still exists. It should not be created any more.
/var/log/mariadb/columnstore/data/bulk/log should not be created and/or used.
Daniel Lee (Inactive)
added a comment - Build tested: 5.5.2 (Drone #1718)
Tested on multi-node cluster, and different cpimport modes
Tested cpimport simple mode (cpimport dbname tablename datafile)
Tested cpimport batch mode using colxml and job number
/var/log/mariadb/columnstore/data is now being used in place of /var/lib/columnstore/data.
/var/log/mariadb/columnstore/cpimport is now used for cpimport logs.
I observed the following:
/tmp/columnstore_tmp_file directory is still being used or some log files
/var/lib/columnstor/data still exists. It should not be created any more.
/var/log/mariadb/columnstore/data/bulk/log should not be created and/or used.
/tmp/columnstore_tmp_files is still getting some tmp files as expected it was not solely for cpimport logging there might be the output of a brm related command going there still but its not really useful so left it alone in tmp dir.
/var/lib/columnstore/data is not exclusively for cpimport AFAIK. just leaving as existing directory for now.
/var/log/mariadb/columnstore/data/bulk/log is "used" but should be empty most of the time.
Ben Thompson (Inactive)
added a comment - /tmp/columnstore_tmp_files is still getting some tmp files as expected it was not solely for cpimport logging there might be the output of a brm related command going there still but its not really useful so left it alone in tmp dir.
/var/lib/columnstore/data is not exclusively for cpimport AFAIK. just leaving as existing directory for now.
/var/log/mariadb/columnstore/data/bulk/log is "used" but should be empty most of the time.
There are cpimport logs in /tmp/columnstore_tmp_files.
In the /var/lib/columnstore/data directory, there is the bulk sub directory. Leaving it there creates confusion. Certain users may complain about these sub directories being empty.
Same thing for /var/log/mariadb/columnstore/data/bulk/log. Since it has been migrated to /var/log/mariadb/columnstore/bulk/log, what is it being used? We should consolidate it to avoid confusion.
Daniel Lee (Inactive)
added a comment - There are cpimport logs in /tmp/columnstore_tmp_files.
In the /var/lib/columnstore/data directory, there is the bulk sub directory. Leaving it there creates confusion. Certain users may complain about these sub directories being empty.
Same thing for /var/log/mariadb/columnstore/data/bulk/log. Since it has been migrated to /var/log/mariadb/columnstore/bulk/log, what is it being used? We should consolidate it to avoid confusion.
Bug
Critical
WORKAROUND FOR MISSING .bad FILES:
set the variable "WorkingDIrectory" in /usr/lib/systemd/system/mcs-writeengineserver.service
[Service]
Type=forking
WorkingDirectory=/home/mysql/
make sure the user running the writeengineserver process (mysql default) has write access to this directory
this is just going to have a file temporarily while cpimport.bin creates and sends the data for the bad file to cpimport on PM1
after updating the file restart WES process with
systemctl daemon-reload
systemctl restart mcs-writeengineserver.service