[MDEV-14505] Threads_running becomes scalability bottleneck Created: 2017-11-27  Updated: 2019-01-17  Resolved: 2017-12-13

Status: Closed
Project: MariaDB Server
Component/s: Server
Affects Version/s: 5.5, 10.0, 10.1, 10.2, 10.3
Fix Version/s: 10.3.3

Type: Bug Priority: Major
Reporter: Sergey Vojtovich Assignee: Sergey Vojtovich
Resolution: Fixed Votes: 0
Labels: performance

Issue Links:
Blocks
is blocked by MDEV-18287 Status threads_running show wrong val... Closed
Epic Link: arm64 optimization

 Description   

thread_running is global shared variable, which is updated twice per query in dispatch_command() using atomic-add.

It started to appear on radar in OLTP RO benchmark. E.g here're numbers for 2 socket/20 cores/40 threads Intel Broadwell system:

   0,70%  mysqld               [.] dispatch_command
 
...
 
       │      inline void thread_safe_increment32(int32 *value)
 
       │      {
 
       │        (void) my_atomic_add32_explicit(value, 1, MY_MEMORY_ORDER_RELAXED);
 
  0,59 │ 680:   lea    thread_running,%rax
 
 16,07 │        lock   addl   $0x1,(%rax)

If we remove inc_thread_running() and dec_thread_running() we get slightly better throughput and dispatch_command() goes down in profiler:

   0,58%  mysqld               [.] dispatch_command
 
...
 
       │      /* increment query_id and return it.  */
 
       │      inline __attribute__((warn_unused_result)) query_id_t next_query_id()
 
       │      {
 
       │        return my_atomic_add64_explicit(&global_query_id, 1, MY_MEMORY_ORDER_RELAXED);
 
  0,49 │ 290:   mov    $0x1,%edx
 
 18,02 │        lock   xadd   %rdx,(%r14)

So bottleneck is shifted to global_query_id counter, which is subject for another bug.

I expect much higher scalability impact on more powerful hardware.

 Comments   
Comment by Sergey Vojtovich [ 2017-11-27 ]

ssethia, dthompson FYI: I added this under MDEV-14442, because it may have negative impact for ARM as well. Nevertheless it is generic scalability bottleneck, so feel free to move it out of scope of arm64 optimisations.

Comment by Sergey Vojtovich [ 2017-12-12 ]

Patch was approved by wlad.

Generated at Thu Feb 08 08:14:06 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.