[MCOL-1266] Too much logging and not clearing up Created: 2018-03-13  Updated: 2023-10-26  Resolved: 2018-03-13

Status: Closed
Project: MariaDB ColumnStore
Component/s: ?
Affects Version/s: 1.1.2
Fix Version/s: Icebox

Type: Bug Priority: Major
Reporter: BOBY PETER Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Environment:

REHL 7

getsoftwareinfo Tue Mar 13 13:42:25 2018

Name : mariadb-columnstore-platform
Version : 1.1.2
Release : 1
Architecture: x86_64
Install Date: Wed 29 Nov 2017 11:33:08 AM EST
Group : Applications/Databases
Size : 113942794
License : Copyright (c) 2016 MariaDB Corporation Ab., all rights reserved; redistributable under the terms of the GPL, see the file COPYING for details.
Signature : (none)
Source RPM : mariadb-columnstore-platform-1.1.2-1.src.rpm
Build Date : Thu 16 Nov 2017 10:20:27 AM EST

Issue Links:
Duplicate
duplicates MCOL-365 tmp and log files are never cleared o... Closed

 Description   

I'm loading data into multiple columnstore tables through Cpimport. It is creating too much log files in the system and is not cleaning it.

  • /usr/local/mariadb/columnstore/data/bulk/log
  • /tmp/columnstore_tmp_files
    It is growing day by day and I started to see warning related to space in root.

Is there any flag or threshold that I could adjust so that it will keep these files only for few days?

[root@usiaddcdsm13 columnstore_tmp_files]# ls -ltr /usr/local/mariadb/columnstore/data/bulk/log | wc
667 5996 41307

[root@usiaddcdsm13 columnstore_tmp_files]# ls -ltr /tmp/columnstore_tmp_files | wc
1442 12971 98847

 Comments   
Comment by Andrew Hutchings (Inactive) [ 2018-03-13 ]

Closing as duplicate of MCOL-365. Please vote for that issue so that we can prioritise it accordingly.

Comment by David Hill (Inactive) [ 2018-03-13 ]

As noted in this document, MariaDB ColumnStore does do log rotation and 7 day cleanup for for the system and MariaDB SQL logs.

https://mariadb.com/kb/en/library/system-troubleshooting-mariadb-columnstore/#mariadb-columnstore-logging

So this file could be editted to also perform log cleanup on the data bulkload logs.

So you could add this to the logrotate file mentioned in the documentation. This will rotate the logs daily and delete them after 3 days.

/usr/local/mariadb/columnstore/data/bulk/log/*

{ missingok rotate 3 daily maxage 3 }

/tmp/columnstore_tmp_files/*

{ missingok rotate 3 daily maxage 3 }
Generated at Thu Feb 08 02:27:27 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.