[MDEV-4454] Query Cache Partitioning Created: 2013-04-29  Updated: 2016-02-17

Status: Open
Project: MariaDB Server
Component/s: None
Fix Version/s: None

Type: Task Priority: Minor
Reporter: Oleksandr Byelkin Assignee: Oleksandr Byelkin
Resolution: Unresolved Votes: 4
Labels: None

Issue Links:
Relates
relates to MDEV-4682 QUERY CACHE - add STATISTICS per quer... Closed
relates to MDEV-6371 QUERY CACHE - Reaping Thread Open

 Description   

Make QC partition, i.e. many QC to avoid bottleneck of mutexes in it.

Query should be divided by QCs without parsing (QC answer should be given without parsing the query)

1) Divide query by its text hash + flags value.
+ even distribution
+ fully automatic and 0 management
– duplicating queries
– no ability to manage QCs by task

2) Divide queries by current DB and other flags (client text attributes for example (new feature of MySQL 5.6 and MariaDB 10.*)
+ more precise management by task
+ less duplication in the caches
– more complex setup

QC can be changed to plugin intrface and we can plug old (for small installations) and petitioned (for advanced).

General problem:

  • Current engine interface suppose that there is only one QC and we can't avoid storing the same tables/queries in different QC's

 Comments   
Comment by roberto spadim [ 2014-06-21 ]

Hi guys any news here?

Comment by Sergei Golubchik [ 2014-06-21 ]

as you can see, this task has "Fix Version: 10.2.0" at the moment. Which means, you should not expect it to be done before 10.2.0. But it doesn't mean this will be done in 10.2.0 we didn't start planning 10.2 yet.

Comment by roberto spadim [ 2014-09-22 ]

changing query cache, will change qc_info plugin

Comment by roberto spadim [ 2014-09-23 ]

i was reading query cache code and some reports about write contentions (http://www.percona.com/blog/2012/09/05/write-contentions-on-the-query-cache/)

maybe should be interesting something like myisam/aria concurrent insert, or group commits
when inserting a query block or a result block at query cache:

1) store query at query block as 'writing' status, we can execute a lock free feature (must check how to execute this without overfit query cache size), others process that execute the same query will not include query twice, but query cache invalidation will be blocked
2) store blocks in a 'temporary' block before insert to query cache, this reduce query cache lock since we don't need a lock every time, but must be memory limit safe
3) after all blocks stored in this temporary block, include this blocks to query cache block (check about size limits), maybe we could do 'batch' inserts instead of many small writes
4) if process die we should change query cache status from 'writing' to 'invalid' and remove links from tables used

well i think that's all

Comment by Sergey Vojtovich [ 2016-02-17 ]

If the aim of this task is scalability, why not to switch to lock-free hash?

Generated at Thu Feb 08 06:56:33 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.