[MDEV-10010] Recursive call to mysql_rwlock_rdlock for LOCK_system_variables_hash Created: 2016-04-29  Updated: 2016-08-11  Resolved: 2016-07-08

Status: Closed
Project: MariaDB Server
Component/s: Locking
Affects Version/s: 10.1
Fix Version/s: 5.5.51, 10.1.16, 10.0.27, 10.2.2

Type: Bug Priority: Major
Reporter: Vladislav Vaintroub Assignee: Sergey Vojtovich
Resolution: Fixed Votes: 0
Labels: None

Sprint: 10.1.15

 Description   

Application verifier reports an error in plugins_var test (recursive read lock on reader writer lock) for LOCK_system_variables_hash.

The lock is acquired in sql_show.cc , in fill_variables() function

 mysql_rwlock_rdlock(&LOCK_system_variables_hash);
  res= show_status_array(thd, wild, enumerate_sys_vars(thd, sorted_vars, scope),
                         scope, NULL, "", tables->table,
                         upper_case_names, partial_cond);
  mysql_rwlock_unlock(&LOCK_system_variables_hash);

however the same lock is acquired again deeper inside show_status array. appverifier points to this callstack

srw_rdlock+0xa [h:\work\server\mysys\thr_rwlock.c @ 89]
intern_sys_var_ptr+0x9b [h:\work\server\sql\sql_plugin.cc @ 2981]
sys_var_pluginvar::do_value_ptr+0x15 [h:\work\server\sql\sql_plugin.cc @ 3405]
sys_var::value_ptr+0x42 [h:\work\server\sql\set_var.cc @ 254]
show_status_array+0x33e [h:\work\server\sql\sql_show.cc @ 3285]
fill_variables+0x148 [h:\work\server\sql\sql_show.cc @ 7197]

On Windows, recursive shared lock acquire can cause a deadlock. Here is the explanation of the bug given by windbg command '!analyze -v'
APPLICATION_VERIFIER_SRWLOCK_RECURSIVE_ACQUIRE (253)
The SRW lock is being acquired recursively by the same thread.
This stop is generated if the SRW lock (Param1) is being acquired
recursively by the same thread.
This will result in a deadlock and the thread would block indefinitely.
Recursive acquisition of an SRW lock in exclusive mode will cause a deadlock.
Recursive acquisition of an SRW lock in shared mode will cause a deadlock when
there is a thread waiting for exclusive access. Consider the example below:

  • Thread A acquires the SRW lock in shared mode
  • Thread B tries to acruire the SRW lock in exclusive mode and waits
  • Thread A tries to acquire the SRW lock in shared mode recursively. This will
    be successful as long as there is no exclusive waiter (in this case B). Since
    SRW locks do not have writer starvation, thread A waits behind thread B.
    Now, Thread B is waiting for Thread A which is inturn waiting for Thread B
    causing a circular wait and hence a deadlock.


 Comments   
Comment by Sergey Vojtovich [ 2016-04-29 ]

See also https://jira.mariadb.org/browse/MDEV-5089?focusedCommentId=41621&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-41621

I agree it should be fixed, but it's not that trivial.

Comment by Vladislav Vaintroub [ 2016-04-29 ]

Thanks for the above link. It points to single unix spec But
http://pubs.opengroup.org/onlinepubs/007908799/xsh/pthread_rwlock_rdlock.html which I did not read prior, which says recursive rdlock must be possible. This is not exactly how Windows does it. If anything else fails, we could make Windows compliant to the spec, at the expense of introducing an atomic counter for readers

Comment by Vladislav Vaintroub [ 2016-04-29 ]

Thinking a little more, it may not be as easy to add recursion to SRW locks.... The last measure of going back to generic implementation with in thr_rwlock with mutex and condition variable, but maybe we can avoid it for this single case of read lock recursion.

Comment by Vladislav Vaintroub [ 2016-05-20 ]

On why windows reader-writer lock can't be used recursively see also https://blogs.msdn.microsoft.com/oldnewthing/20160506-00/?p=93416

Comment by Sergey Vojtovich [ 2016-06-27 ]

serg, please review fix for this bug.
wlad, do you think this patch will make Application Verifier happy?

Comment by Vladislav Vaintroub [ 2016-08-03 ]

elenst , I backported the fix to 5.5

Generated at Thu Feb 08 07:38:59 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.