[MDEV-18617] Optimization for thread group (asynchronous thread group submission) Created: 2019-02-18  Updated: 2019-04-20  Resolved: 2019-04-20

Status: Closed
Project: MariaDB Server
Component/s: Server
Affects Version/s: 10.2.22, 10.3.12
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: musazhang Assignee: Unassigned
Resolution: Incomplete Votes: 0
Labels: need_feedback


 Description   

When variable of thread_handling is assigned as pool-of-threads, there are at most thread_pool_size * thread_pool_oversubscribe working threads to deal with clients request. So when the concurrence of clients is larger than thread_pool_size * thread_pool_oversubscribe and some threads wait for binlog flush or redo log flush, working threads of thread group will decrease and as a result , the system throughput will decrease too.

To improve the throughput, we can let the thread pool worker thread continue to deal with client requests and have some other thread to deal with the waiting user thread.need



 Comments   
Comment by Vladislav Vaintroub [ 2019-02-18 ]

the "waiting" threads are not counted as active, and for that there is thd_wait_begin() / thd_wait_end() callbacks, to account for active vs waiting threads. There are at most thread_pool_size*thread_pool_oversubscribe active threads . The real maximum count worker of threads in pool is thread_pool_max_threads.

If you found a specific situation , where you think thd_wait_begin/end is missing, i.e where thread is counted as active and it is not, please provide a stacktrace of all threads, or pmp output

Comment by musazhang [ 2019-02-18 ]

when thread call log_write_up_to, there is no thd_wait_begin or thd_wait_end. isn't it ?

Comment by Vladislav Vaintroub [ 2019-02-18 ]

No there is not. The idea is to put thd_wait_begin/thd_wait_end around waiting parts, not around the whole functions.
Therefore I'm asking about providing the callstacks, to find out the waiting part, which I'd guess is os_event_wait. You can add thd_wait_xx around it,and then you can evaluate how much thd_wait_begin/end helps in your benchmarks, and then we can evaluate in ours. There is unfortunately some danger that adding thd_wait_xxx can lead to creating too many threads. Adding it for long waits is no problem, but for microsecond-wait it might be counterproductive.
.

Generated at Thu Feb 08 08:45:28 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.