I believe we need to extend the task and create an utility to load data into a single column. This will be helpfull for backup/restore purposes.

tntnatbry - Roman says that the utility already exists, except that it does not work for compressed data, and as such needs finishing. He will give you all the pointers needed.

The current mcsRebuildEM says "Requirement: all DBRoots must be on this node". Not sure how to do this with a 3 node system. Probably need to look into that.

It also has a line:
"Only internal DBRootStorageType is supported, provided: "
We need this to work on external and s3 files and for multi-node systems. It appears mutli-node is the most likely to need it.

This utility should give us the ability to restore any system, regardless of number of nodes or type of backing file system, to:
1) restore from BRM_saves if possible.
2) rebuild from data sources if possible.
3) option to override (1) in favor of (2). Sometimes the BRM_saves that are available are out of date and data would be lost.

Data retention is more important than speed, so if we need to chug for a bit to accomplish this, so be it.

This effort should also contain either a separate utility, or as part of the same process, the ability to rebuild the calponsystemcatalog from the .frm files and the data file/extent map. While there are plans to modify the way systemcatalog operates, we need to get this now.

The code should be modular enough (or whatever) to handle serious changes in the extent map and system catalog formats and storage. Future plans include such.

After 4566 was done, this utility is now as complete as it should be.

Let's doc it.