[MCOL-1081] Improve performance of system catalog Created: 2017-12-05  Updated: 2022-11-05  Resolved: 2022-11-05

Status: Closed
Project: MariaDB ColumnStore
Component/s: N/A
Affects Version/s: None
Fix Version/s: Icebox

Type: New Feature Priority: Major
Reporter: Andrew Hutchings (Inactive) Assignee: Todd Stoffel (Inactive)
Resolution: Won't Do Votes: 0
Labels: None

Issue Links:
PartOf
includes MCOL-1484 Use condition pushdown in I_S tables Closed

 Description   

System Catalog columnRIDs is quite slow when there are many columns in an installation. When iterating over a table this takes a very long time to execute and populate. This matters for information_schema and mcsapi.

Proposed changes:

1. system catalog should not flush the entire cache on a single table change, just the table that changed
2. tableinfo() call to columnRIDs should use cache
3. for information_schema and mcsapi there should be a call that retrieves the entire catalog without the filter conditions instead of lots of loops that we do at the moment.
4. mcsapi (in DBRM's doGetSystemCatalog) we should switch to FE mode:

systemCatalogPtr->identity(execplan::CalpontSystemCatalog::FE);



 Comments   
Comment by Roman [ 2018-03-13 ]

Considering perf output I could [reduce IS_files executionhttps://hastebin.com/ijibuxukuh.go timings]. Unfortunately the timings are not so great, when the buffer cache is emty: it's about 7.5 secondss for the same amount of files. It's an improvement comparing with initial timings though. Here is the commit.

Comment by Todd Stoffel (Inactive) [ 2022-11-05 ]

Item is out of date. Closing due to inactivity. If you feel this was done in error please open a new ticket.

Generated at Thu Feb 08 02:26:03 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.