This should be implemented as a set of mcsadmin commands to support:

• starting (asynchronously)
• retrieving status
• stopping

This should disable writes while running. Not sure if it's realistic or not to consider concurrent queries?

Added command 4 - Redistribute to the mcsadmin code

I couldn't get this thing into review mode, so it's in test for now. DHill, please review.

reviewed completed...

Needs some test cases from David Hall....

2 of them should be: dbroot assigned to pm1, and dbroot assigned to another pm..

Did recommend changing the command name from redistribute to redistributeData

and here is a link to the document, which will be publish in 1.0.6 release

https://mariadb.com/kb/en/mariadb/columnstore-redistribute-data/

Name changed to redistributeData.

Because this command works at partition granularity, it won't move any data for smaller tables (See documentation).

There are three tests that make sense. Start a multi PM system.

1) Add large amounts of data using cpimport mode 2 or 3 to create an unbalanced table (or more). Run redistributeData and watch the result.

For example using lineitem, run this query before and after to see how many rows moved. The counts should be about the same (+/- 64000000).
select count, idbdbroot(l_orderkey) from lineitem group by idbdbroot(l_orderkey);

2) Start with a multi PM system. Have an extra PM ready to add (or at least an extra dbroot or two). Add a bunch of data, then add the new PM in and run redistributeData. See what happens.

3). After test (2), run redistributeData again, but with "remove" and the newly added dbroots. It should empty those new roots, moving the data to the old dbroots.

Per discussion on the Slack channel today, redistributeData does not check for database write suspension for now. It will run as the use requests it. That set's the testing scope for QA for testing this feature for this release.

Build tested: 1.0.6-1

mcsadmin> getsoft
getsoftwareinfo Wed Dec 7 19:15:31 2016

Name : mariadb-columnstore-platform
Version : 1.0.6
Release : 1
Architecture: x86_64
Install Date: Wed 07 Dec 2016 03:17:06 PM EST
Group : Applications/Databases
Size : 10017001
License : Copyright (c) 2016 MariaDB Corporation Ab., all rights reserved; redistributable under the terms of the GPL, see the file COPYING for details.
Signature : (none)
Source RPM : mariadb-columnstore-platform-1.0.6-1.src.rpm
Build Date : Wed 07 Dec 2016 02:08:09 PM EST

"START REMOVE" does move any data. It finished immediately

[6:10]
mcsadmin> redistributedata start remove 3
redistributedata Wed Dec 7 19:07:42 2016
redistributeData START Removing dbroots: 3
Source dbroots: 1 2 3 4
Destination dbroots: 1 2 4

WriteEngineServer returned status 1: Cleared.
WriteEngineServer returned status 2: Redistribute is started.
mcsadmin> redistributedata status
redistributedata Wed Dec 7 19:07:47 2016
WriteEngineServer returned status 3: Redistribute is finished.
0 success, 0 skipped, 0 failed.
Total time: 0 seconds.

[6:10]
--------------------------------+

count

idbdbroot(l_orderkey)

--------------------------------+

--------------------------------+
4 rows in set (8.80 sec)

In the KB article, this statement "Even TRUNCATE doesn't really get rid of the partition, though the space may eventually get re-used. DROP PARTITION will affect the balancing of Redistribute." is not correct. TRUNCATE drops all files for the columns and pre allocate each column with an abbreviated extent file.

reopen per my last comment.

The KB has been changed to remove the comment about TRUNCATE.

All references to start remove have been removed from the help file and KB.
MCOL-455 has been opened to address the start remove issue.

Closing this.

Needs testing

Fixed the problem with the help file

Build verified: 1.0.6-1 Github source

[root@localhost mariadb-columnstore-server]# git show
commit 4c075221724fb8dfc0c1d51e223343a0b3f2ee64
Author: david hill <david.hill@mariadb.com>
Date: Thu Dec 8 09:42:04 2016 -0600

Update README.md

[root@localhost mariadb-columnstore-server]# cd mariadb-columnstore-engine/
[root@localhost mariadb-columnstore-engine]# git show
commit 9152d920eba16e84674376ef4049f313c15e3fdc
Merge: d6717cc f5532d5
Author: dhall-InfiniDB <david.hall@mariadb.com>
Date: Thu Dec 8 11:30:02 2016 -0600

Merge pull request #76 from mariadb-corporation/MCOL-307

MCOL-307 ConsoleCmds.xml needs to have sequential arg numbers

mcsadmin> redistributedata
redistributedata Thu Dec 8 18:30:24 2016
redistributeData must have one of START, STOP or STATUS
mcsadmin> help redistributedata
help Thu Dec 8 18:30:30 2016

Command: redistributeData

Description: Redistribute table data accross all dbroots to balance disk usage

Arguments: START to begin a redistribution
STOP to stop redistribution before completion
STATUS to to view statistics and progress

Also verified KB article regarding TRUNCATE.

"Deleted records still take up space, so deleting a bunch of rows won't affect Redistribute."

The "START REMOVE" issue is not being tracked by ticket MCOL-455. Closing this ticket.