Profile / optimize SM
(MCOL-3417)
|
|
| Status: | Closed |
| Project: | MariaDB ColumnStore |
| Component/s: | ? |
| Affects Version/s: | None |
| Fix Version/s: | 1.4.0 |
| Type: | Sub-Task | Priority: | Major |
| Reporter: | Patrick LeBlanc (Inactive) | Assignee: | Patrick LeBlanc (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Description |
|
Importing part table from a 500gb dbt3 dataset. When the target table is compressed, observed behavior is expected and good (19k calls to write()). When uncompressed, the opposite (750k calls to write()). Small writes make the overhead parts in SM dominate. In particular, what rises to the top of the profile are functions related to json parsing. The first thing I will do is implement a parsed-json cache, then reevaluate. |
| Comments |
| Comment by Patrick LeBlanc (Inactive) [ 2019-08-14 ] |
|
Cut end-to-end runtime 40-50% for this test case. Took more changes than I expected. |
[MCOL-3418]