[MXS-3721] Add QoS filter Created: 2021-08-12  Updated: 2023-04-04

Status: Stalled
Project: MariaDB MaxScale
Component/s: None
Affects Version/s: None
Fix Version/s: Icebox

Type: New Feature Priority: Major
Reporter: Manjot Singh (Inactive) Assignee: Niclas Antti
Resolution: Unresolved Votes: 1
Labels: None

Sprint: MXS-SPRINT-168, MXS-SPRINT-169, MXS-SPRINT-170

 Description   

Maxscale knows how fast queries are (TPM filter could be chained in here). Using this knowledge, maxscale could learn and queue queries in such a way as to provide faster returns for certain queries without allowing blocking queries through.

There should be controls allowing for certain users, schemas or query comments to skip QoS.



 Comments   
Comment by Rob Schwyzer [ 2021-08-13 ]

Maybe something where hints or comments could be leveraged or expanded as well? For example, many customers I've interacted with have use-cases where they are submitting queries for reports or similar and for these queries, the customers do not care much about how long the query takes, just that the query completes successfully. It would be helpful to enable users/applications to provide hints or similar data to MaxScale to indicate when a query is explicitly low priority for completion time to enable MaxScale to make better QoS decisions as part of this feature.

Comment by markus makela [ 2022-07-18 ]

A simple approach to this would be to add a new routing hint type that would prioritize the query that has it. Combining it with idle_session_pool_time, MaxScale could push the query to the front of the queue instead of adding it to the back where it normally would end up at.

Comment by Rob Schwyzer [ 2022-07-18 ]

That makes sense, but most customers I've talked to view this problem the opposite way. That isn't to say we shouldn't solve priority queries- we definitely have customers interested in those as well. But we should consider if there is anything else we can or should do for queries customers want to specifically mark as low priority.

If nothing else, explicitly low priority queries could be held up to x (ideally configurable) time before being put into routing algorithms when idle_session_pool_time is in-use and concurrent client connections/queries to MaxScale have (on average, or similar stat) exceeded available backend connections (aka the sum of backends' max_routing_connections for a router).

Could also consider something like expanding on ADAPTIVE_ROUTING where explicitly low-priority queries can withhold being submitted to a backend until a backend's load/latency drops below a (configurable) metric.

Fwiw, I understand the priority queries markus makela described (which are good and which imo we should do), the max_routing_connections-observant de-prioritized queries, and the ADAPTIVE_ROUTING-aware de-prioritized queries are probably three, separate feature requests. I wouldn't have any issue tackling priority queries first and the two, de-prioritized query scenarios I brought up here separately (possibly with a later release target).

Comment by Manjot Singh (Inactive) [ 2022-07-18 ]

Could SQL syntax be used as hints? for example the WAIT syntax?

Comment by markus makela [ 2022-09-09 ]

Combining some of the features from the topfilter and tpmfilter to show where the time is spent would help evaluate how well the filter is working.

Generated at Thu Feb 08 04:23:28 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.