Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
None
-
None
-
2021-1, 2021-2, 2021-3, 2021-4
Description
ColumnStore logs are located in /tmp, /var/lib/columnstore/data/bulk/log, /var/log/mariadb/columnstore, etc. cmapi doesn't even have a log file. You have to use the journalctl utility. Can this be simplified and standardized into one location for all processes.
Additionally, when using cpimport the .bad file does not get written to any folder. It instead throws a 'permissions' error into the job log.
Attachments
Issue Links
Activity
| Field | Original Value | New Value |
|---|---|---|
| Assignee | Todd Stoffel [ toddstoffel ] |
| Fix Version/s | 6.1 [ 25201 ] |
| Rank | Ranked higher |
| Labels | UX |
| Epic Link |
|
| Priority | Minor [ 4 ] | Critical [ 2 ] |
| Fix Version/s | 5.6.1 [ 25031 ] | |
| Fix Version/s | 6.1 [ 25201 ] |
| Rank | Ranked higher |
| Assignee | Todd Stoffel [ toddstoffel ] | Ben Thompson [ ben.thompson ] |
| Sprint | 2021-1 [ 480 ] |
| Rank | Ranked higher |
| Rank | Ranked higher |
| Rank | Ranked higher |
| Description | ColumnStore logs are located in /tmp, /var/lib/columnstore/data/bulk/log, /var/log/mariadb/columnstore, etc. cmapi doesn't even have a log file. You have to use the journalctl utility. Can this be simplified and standardized into one location for all processes. |
ColumnStore logs are located in /tmp, /var/lib/columnstore/data/bulk/log, /var/log/mariadb/columnstore, etc. cmapi doesn't even have a log file. You have to use the journalctl utility. Can this be simplified and standardized into one location for all processes.
Additionally, when using cpimport the .bad file does not get written to any folder. It instead throws a 'permissions' error into the job log. |
| Status | Open [ 1 ] | In Progress [ 3 ] |
| Attachment | missing_bad.txt [ 55683 ] |
| Comment |
[ If setting a new bulk log folder as described by [~David.Hall] above, please remember to:
{code:java} chown -R mysql:mysql /var/log/mariadb/columnstore/bulk {code} ] |
| Issue Type | Task [ 3 ] | Bug [ 1 ] |
| Rank | Ranked higher |
| Fix Version/s | 5.5.2 [ 25601 ] |
| Fix Version/s | 5.6.1 [ 25031 ] |
| Sprint | 2021-1 [ 480 ] | 2021-1, 2021-2 [ 480, 481 ] |
| Rank | Ranked higher |
| Rank | Ranked higher |
| Sprint | 2021-1, 2021-2 [ 480, 481 ] | 2021-1, 2021-2, 2021-3 [ 480, 481, 498 ] |
| Assignee | Ben Thompson [ ben.thompson ] | Roman [ drrtuy ] |
| Status | In Progress [ 3 ] | In Review [ 10002 ] |
| Assignee | Roman [ drrtuy ] | David Hall [ david.hall ] |
| Status | In Review [ 10002 ] | In Testing [ 10301 ] |
| Assignee | David Hall [ david.hall ] | Daniel Lee [ dleeyh ] |
| Fix Version/s | 6.6.1 [ 25633 ] |
| Fix Version/s | 6.6.1 [ 25633 ] |
Build tested: 5.5.2 (Drone #1718)
Tested on multi-node cluster, and different cpimport modes
Tested cpimport simple mode (cpimport dbname tablename datafile)
Tested cpimport batch mode using colxml and job number
/var/log/mariadb/columnstore/data is now being used in place of /var/lib/columnstore/data.
/var/log/mariadb/columnstore/cpimport is now used for cpimport logs.
I observed the following:
/tmp/columnstore_tmp_file directory is still being used or some log files
/var/lib/columnstor/data still exists. It should not be created any more.
/var/log/mariadb/columnstore/data/bulk/log should not be created and/or used.
| Assignee | Daniel Lee [ dleeyh ] | Ben Thompson [ ben.thompson ] |
| Status | In Testing [ 10301 ] | Stalled [ 10000 ] |
| Sprint | 2021-1, 2021-2, 2021-3 [ 480, 481, 498 ] | 2021-1, 2021-2, 2021-3, 2021-4 [ 480, 481, 498, 499 ] |
/tmp/columnstore_tmp_files is still getting some tmp files as expected it was not solely for cpimport logging there might be the output of a brm related command going there still but its not really useful so left it alone in tmp dir.
/var/lib/columnstore/data is not exclusively for cpimport AFAIK. just leaving as existing directory for now.
/var/log/mariadb/columnstore/data/bulk/log is "used" but should be empty most of the time.
There are cpimport logs in /tmp/columnstore_tmp_files.
In the /var/lib/columnstore/data directory, there is the bulk sub directory. Leaving it there creates confusion. Certain users may complain about these sub directories being empty.
Same thing for /var/log/mariadb/columnstore/data/bulk/log. Since it has been migrated to /var/log/mariadb/columnstore/bulk/log, what is it being used? We should consolidate it to avoid confusion.
| Rank | Ranked higher |
| Status | Stalled [ 10000 ] | In Progress [ 3 ] |
| Assignee | Ben Thompson [ ben.thompson ] | David Hall [ david.hall ] |
| Status | In Progress [ 3 ] | In Review [ 10002 ] |
| Status | In Review [ 10002 ] | In Testing [ 10301 ] |
| Assignee | David Hall [ david.hall ] | Daniel Lee [ dleeyh ] |
Build verified: 5.5.2 (Drone #1810)
The mentioned directories have been removed
| Resolution | Fixed [ 1 ] | |
| Status | In Testing [ 10301 ] | Closed [ 6 ] |
| Rank | Ranked higher |
WORKAROUND FOR MISSING .bad FILES:
set the variable "WorkingDIrectory" in /usr/lib/systemd/system/mcs-writeengineserver.service
[Service]
Type=forking
WorkingDirectory=/home/mysql/
make sure the user running the writeengineserver process (mysql default) has write access to this directory
this is just going to have a file temporarily while cpimport.bin creates and sends the data for the bad file to cpimport on PM1
after updating the file restart WES process with
systemctl daemon-reload
systemctl restart mcs-writeengineserver.service