[MCOL-3316] cpimport column based formats - text,binary Created: 2019-05-17 Updated: 2020-08-25 Resolved: 2020-04-02 |
|
| Status: | Closed |
| Project: | MariaDB ColumnStore |
| Component/s: | N/A |
| Affects Version/s: | None |
| Fix Version/s: | N/A |
| Type: | New Feature | Priority: | Major |
| Reporter: | David Hill (Inactive) | Assignee: | Todd Stoffel (Inactive) |
| Resolution: | Won't Do | Votes: | 0 |
| Labels: | None | ||
| Description |
|
Customer feature request we encountered some cases were we had to transform column based format file ( orc,parquet,ect...) into row files (csv) to get loaded through cpimport which internally re-parsed into columns; even the binary mode of cpimport is row-based. Could we add a feature that allow to cpimport column based format (text,binary ) ? |
| Comments |
| Comment by David Hill (Inactive) [ 2019-05-20 ] |
|
Answer from customer Not exactly, this might be inconvenient for tables with a large number of rows. Right now, we have orc file that are naturally process as a python dict , one big list per column name ( key in the dict). I guess the convenient twxt format would be to have a a single row be all the value of a column in the table. A transposed csv. Patrice. |
| Comment by Andrew Hutchings (Inactive) [ 2019-09-26 ] |
|
So, this is a feature to run a command which loads lots of single column files at the same time? |
| Comment by David Hill (Inactive) [ 2019-09-26 ] |
|
Response from customer No Still like cpimport, cpimport_column_format schema table file the file is just for one table. |
| Comment by Todd Stoffel (Inactive) [ 2019-09-28 ] |
|
Need feedback from support/sales |
| Comment by Todd Stoffel (Inactive) [ 2020-04-02 ] |
|
There was no follow up about an NRE so closing. |