Details
-
Task
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
None
-
10.2.2-1, 10.2.2-2, 10.2.2-3
Description
There is feature that is there in Percona tree 5.5+ and also in MySQL/Enterprise
but not in MariaDB is priorization
- By default, connection that is in transaction gets a priority boost (which translated to
in transaction= low prio, outside of transaction => high prio). The first statement after "begin" does not need to be high prio (this transaction does not actually use any resources)
- We need to have a variable that will allow to specify per-connection or global scheduling behavior (auto = based on transaction status, high - always high prio, low- always low prio)
This way, it would be easy to return back to FIFO scheduling if someone prefers that (i.e set global threadpool_priority=high). Or for a connection to be "nice" (setting priority = low regardless of transaction status), if it does not unimportant work . This variable is akin to Percona's threadpool_high_prio_mode=[transactions|statements|none] , but with an understandable name.
- On Unix ,we'll use separate queues for high priority and low priority items. high priority queue is checked before low prio when an event is dequeued.
- On Windows threadpool, it is possible to 'yield' low prio work items.
- On Unix, to avoid starvation of the low prio items, periodically the old items in low prio queue need to be merged at the end of high prio queue. We can use 'thread_pool_kickup_timer' logic, like MySQL Enterprise is doing it.
- We won't be using 'throttling of low prio queue' like Percona, as restricting the number of active+waiting threads can easily lead to a deadlock (single global lock or table lock would suffice to check that)
Attachments
Activity
Field | Original Value | New Value |
---|---|---|
Assignee | Vladislav Vaintroub [ wlad ] |
Sprint | 10.2.2-1 [ 79 ] |
Rank | Ranked higher |
Sprint | 10.2.2-1 [ 79 ] | 10.2.2-1, 10.2.2-2 [ 79, 82 ] |
Sprint | 10.2.2-1, 10.2.2-2 [ 79, 82 ] | 10.2.2-1, 10.2.2-2, 10.2.2-3 [ 79, 82, 83 ] |
Description |
Main feature that is there in Percona tree 5.5+ and later an not in hours is
1. Priorization mode (in-transaction queries can be prioritized higher, nontransaction can be prioritized higher). https://www.percona.com/blog/2014/01/29/percona-server-thread-pool-improvements has benchmark details. 2. Some smaller stuff to play well with performance schema (idle waits for network), instrumentation for threads |
There is feature that is there in Percona tree 5.5+ and also in MySQL/Enterprise
but not in MariaDB is priorization - By default, connection that is in transaction gets a priority boost (which translated to in transaction= low prio, outside of transaction => high prio). The first statement after "begin" does not need to be high prio (this transaction does not actually use any resources) - We need to have a variable that will allow to specify per-connection or global scheduling behavior (auto = based on transaction status, high - always high prio, low- always low prio) This way, it would be easy to return back to FIFO scheduling if someone prefers that (i.e set global threadpool_priority=high). Or for a connection to be "nice" (setting priority = low regardless of transaction status), if it does not unimportant work . This variable is akin to Percona's threadpool_high_prio_mode=[transactions|statements|none] , but with an understandable name. - On Unix ,we'll use separate queues for high priority and low priority items. high priority queue is checked before low prio when an event is dequeued. - To avoid starvation of the low prio items, periodically the old items in low prio queue need to be merged at the end of high prio queue. We can use 'thread_pool_kickup_timer' logic, like MySQL Enterprise is doing it. - We won't be using 'throttling of low prio queue' like Percona, as restricting the number of active+waiting threads can easily lead to a deadlock (single global lock or table lock would suffice to check that) |
Summary | Port percona threadpool improvements to MariaDB tree | Threadpool : add priorization features |
Description |
There is feature that is there in Percona tree 5.5+ and also in MySQL/Enterprise
but not in MariaDB is priorization - By default, connection that is in transaction gets a priority boost (which translated to in transaction= low prio, outside of transaction => high prio). The first statement after "begin" does not need to be high prio (this transaction does not actually use any resources) - We need to have a variable that will allow to specify per-connection or global scheduling behavior (auto = based on transaction status, high - always high prio, low- always low prio) This way, it would be easy to return back to FIFO scheduling if someone prefers that (i.e set global threadpool_priority=high). Or for a connection to be "nice" (setting priority = low regardless of transaction status), if it does not unimportant work . This variable is akin to Percona's threadpool_high_prio_mode=[transactions|statements|none] , but with an understandable name. - On Unix ,we'll use separate queues for high priority and low priority items. high priority queue is checked before low prio when an event is dequeued. - To avoid starvation of the low prio items, periodically the old items in low prio queue need to be merged at the end of high prio queue. We can use 'thread_pool_kickup_timer' logic, like MySQL Enterprise is doing it. - We won't be using 'throttling of low prio queue' like Percona, as restricting the number of active+waiting threads can easily lead to a deadlock (single global lock or table lock would suffice to check that) |
There is feature that is there in Percona tree 5.5+ and also in MySQL/Enterprise
but not in MariaDB is priorization - By default, connection that is in transaction gets a priority boost (which translated to in transaction= low prio, outside of transaction => high prio). The first statement after "begin" does not need to be high prio (this transaction does not actually use any resources) - We need to have a variable that will allow to specify per-connection or global scheduling behavior (auto = based on transaction status, high - always high prio, low- always low prio) This way, it would be easy to return back to FIFO scheduling if someone prefers that (i.e set global threadpool_priority=high). Or for a connection to be "nice" (setting priority = low regardless of transaction status), if it does not unimportant work . This variable is akin to Percona's threadpool_high_prio_mode=[transactions|statements|none] , but with an understandable name. - On Unix ,we'll use separate queues for high priority and low priority items. high priority queue is checked before low prio when an event is dequeued. - On Windows threadpool, it is possible to 'yield' low prio work items. - On Unix, to avoid starvation of the low prio items, periodically the old items in low prio queue need to be merged at the end of high prio queue. We can use 'thread_pool_kickup_timer' logic, like MySQL Enterprise is doing it. - We won't be using 'throttling of low prio queue' like Percona, as restricting the number of active+waiting threads can easily lead to a deadlock (single global lock or table lock would suffice to check that) |
Status | Open [ 1 ] | In Progress [ 3 ] |
Assignee | Vladislav Vaintroub [ wlad ] | Sergei Golubchik [ serg ] |
Status | In Progress [ 3 ] | In Review [ 10002 ] |
Fix Version/s | 10.2 [ 14601 ] | |
Fix Version/s | 10.1 [ 16100 ] |
Assignee | Sergei Golubchik [ serg ] | Vladislav Vaintroub [ wlad ] |
Status | In Review [ 10002 ] | Stalled [ 10000 ] |
Component/s | OTHER [ 10125 ] | |
Fix Version/s | 10.2.2 [ 22013 ] | |
Fix Version/s | 10.2 [ 14601 ] | |
Resolution | Fixed [ 1 ] | |
Status | Stalled [ 10000 ] | Closed [ 6 ] |
Workflow | MariaDB v3 [ 76251 ] | MariaDB v4 [ 132908 ] |