Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
10.2(EOL), 10.3(EOL), 10.4(EOL), 10.5, 10.6, 10.7(EOL), 10.8(EOL)
-
None
Description
There are two failures here. They might turn out to be completely different bugs, but I can't separate them from each other, they happen interchangeable in the same scenario, which, apparently, involves a race condition upon installation of server_audit plugin in two different threads.
To reproduce:
git clone https://github.com/MariaDB/randgen --branch mdev22647 rqg-mdev22647
|
cd rqg-mdev22647
|
perl ./runall-new.pl --basedir=<basedir> --grammar=mdev22647.yy --threads=2 --skip-gendata --vardir=/dev/shm/var_mdev22647 --duration=30
|
Remember to modify the command line to point at the real basedir.
10.2 2c4a2f20 |
safe_mutex: Trying to lock mutex at /data/src/10.2/sql/sql_plugin.cc, line 2998, when the mutex was already locked at /data/src/10.2/sql/sys_vars_shared.h, line 56 in thread
|
T@20
|
200520 21:27:19 [ERROR] mysqld got signal 6 ;
|
|
#5 0x00007f36556d742a in __GI_abort () at abort.c:89
|
#6 0x0000564ea6f3240d in safe_mutex_lock (mp=0x564ea7af0e80 <LOCK_global_system_variables>, my_flags=0, file=0x564ea6ff1760 "/data/src/10.2/sql/sql_plugin.cc", line=2998) at /data/src/10.2/mysys/thr_mutex.c:264
|
#7 0x0000564ea6581266 in inline_mysql_mutex_lock (that=0x564ea7af0e80 <LOCK_global_system_variables>, src_file=0x564ea6ff1760 "/data/src/10.2/sql/sql_plugin.cc", src_line=2998) at /data/src/10.2/include/mysql/psi/mysql_thread.h:688
|
#8 0x0000564ea65895d9 in sync_dynamic_session_variables (thd=0x7f35dc000af0, global_lock=true) at /data/src/10.2/sql/sql_plugin.cc:2998
|
#9 0x0000564ea658989f in intern_sys_var_ptr (thd=0x7f35dc000af0, offset=104, global_lock=true) at /data/src/10.2/sql/sql_plugin.cc:3065
|
#10 0x0000564ea6589a0a in mysql_sys_var_str (thd=0x7f35dc000af0, offset=104) at /data/src/10.2/sql/sql_plugin.cc:3112
|
#11 0x00007f365439f173 in get_loc_info (thd=0x7f35dc000af0) at /data/src/10.2/plugin/server_audit/server_audit.c:950
|
#12 0x00007f36543a2fde in mark_always_logged (thd=0x7f35dc000af0) at /data/src/10.2/plugin/server_audit/server_audit.c:2592
|
#13 0x00007f36543a3b25 in update_logging (thd=0x7f35dc000af0, var=0x7f36545a7460 <mysql_sysvar_logging>, var_ptr=0x7f36545a8594 <logging>, save=0x7f35dc0111d0) at /data/src/10.2/plugin/server_audit/server_audit.c:2862
|
#14 0x0000564ea658ada4 in sys_var_pluginvar::global_update (this=0x7f35e80d4670, thd=0x7f35dc000af0, var=0x7f35dc0111b0) at /data/src/10.2/sql/sql_plugin.cc:3516
|
#15 0x0000564ea649e2af in sys_var::update (this=0x7f35e80d4670, thd=0x7f35dc000af0, var=0x7f35dc0111b0) at /data/src/10.2/sql/set_var.cc:208
|
#16 0x0000564ea64a0030 in set_var::update (this=0x7f35dc0111b0, thd=0x7f35dc000af0) at /data/src/10.2/sql/set_var.cc:837
|
#17 0x0000564ea649fc12 in sql_set_variables (thd=0x7f35dc000af0, var_list=0x7f35dc005470, free=true) at /data/src/10.2/sql/set_var.cc:738
|
#18 0x0000564ea657005e in mysql_execute_command (thd=0x7f35dc000af0) at /data/src/10.2/sql/sql_parse.cc:4591
|
#19 0x0000564ea6579e6f in mysql_parse (thd=0x7f35dc000af0, rawbuf=0x7f35dc011018 "SET /* QNO 13 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", length=59, parser_state=0x7f36545f4660, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:7740
|
#20 0x0000564ea65681c2 in dispatch_command (command=COM_QUERY, thd=0x7f35dc000af0, packet=0x7f35dc0087a1 "SET /* QNO 13 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", packet_length=59, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:1832
|
#21 0x0000564ea6566c3d in do_command (thd=0x7f35dc000af0) at /data/src/10.2/sql/sql_parse.cc:1385
|
#22 0x0000564ea66bbac9 in do_handle_one_connection (connect=0x564ea9058280) at /data/src/10.2/sql/sql_connect.cc:1336
|
#23 0x0000564ea66bb834 in handle_one_connection (arg=0x564ea9058280) at /data/src/10.2/sql/sql_connect.cc:1241
|
#24 0x00007f36576574a4 in start_thread (arg=0x7f36545f5700) at pthread_create.c:456
|
#25 0x00007f365578bd0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
|
All threads' stack trace attached as mdev22647_all_threads_mutex
10.2 2c4a2f20 |
mysqld: /data/src/10.2/sql/sql_audit.cc:115: void mysql_audit_acquire_plugins(THD*, ulong*): Assertion `!check_audit_mask(mysql_global_audit_mask, event_class_mask)' failed.
|
200520 21:45:53 [ERROR] mysqld got signal 6 ;
|
|
#7 0x00007fabcf613f12 in __GI___assert_fail (assertion=0x559c33648c68 "!check_audit_mask(mysql_global_audit_mask, event_class_mask)", file=0x559c33648c28 "/data/src/10.2/sql/sql_audit.cc", line=115, function=0x559c33648da0 <mysql_audit_acquire_plugins(THD*, unsigned long*)::__PRETTY_FUNCTION__> "void mysql_audit_acquire_plugins(THD*, ulong*)") at assert.c:101
|
#8 0x0000559c32ce313c in mysql_audit_acquire_plugins (thd=0x7fab58000af0, event_class_mask=0x7fabccddca20) at /data/src/10.2/sql/sql_audit.cc:115
|
#9 0x0000559c32ce3838 in mysql_audit_notify (thd=0x7fab58000af0, event_class=0, event=0x7fabccddca70) at /data/src/10.2/sql/sql_audit.cc:432
|
#10 0x0000559c32b49b62 in mysql_audit_general (thd=0x7fab58000af0, event_subtype=1, error_code=1193, msg=0x7fabccddce40 "Unknown system variable 'SERVER_AUDIT_LOGGING'") at /data/src/10.2/sql/sql_audit.h:197
|
#11 0x0000559c32b4c5f6 in THD::raise_condition (this=0x7fab58000af0, sql_errno=1193, sqlstate=0x559c3362eb25 "HY000", level=Sql_condition::WARN_LEVEL_ERROR, msg=0x7fabccddce40 "Unknown system variable 'SERVER_AUDIT_LOGGING'") at /data/src/10.2/sql/sql_class.cc:1047
|
#12 0x0000559c32aa5015 in my_message_sql (error=1193, str=0x7fabccddce40 "Unknown system variable 'SERVER_AUDIT_LOGGING'", MyFlags=0) at /data/src/10.2/sql/mysqld.cc:3639
|
#13 0x0000559c33548db6 in my_error (nr=1193, MyFlags=0) at /data/src/10.2/mysys/my_error.c:125
|
#14 0x0000559c32bb3f05 in find_sys_var_ex (thd=0x7fab58000af0, str=0x7fab58011108 "SERVER_AUDIT_LOGGING", length=20, throw_error=false, locked=false) at /data/src/10.2/sql/sql_plugin.cc:2840
|
#15 0x0000559c32bb3f75 in find_sys_var (thd=0x7fab58000af0, str=0x7fab58011108 "SERVER_AUDIT_LOGGING", length=20) at /data/src/10.2/sql/sql_plugin.cc:2847
|
#16 0x0000559c32d8efc1 in find_sys_var_null_base (thd=0x7fab58000af0, tmp=0x7fabccdde720) at /data/src/10.2/sql/sql_yacc.yy:394
|
#17 0x0000559c32db7052 in MYSQLparse (thd=0x7fab58000af0) at /data/src/10.2/sql/sql_yacc.yy:15373
|
#18 0x0000559c32ba9e2e in parse_sql (thd=0x7fab58000af0, parser_state=0x7fabccddf660, creation_ctx=0x0, do_pfs_digest=true) at /data/src/10.2/sql/sql_parse.cc:10029
|
#19 0x0000559c32ba4ce1 in mysql_parse (thd=0x7fab58000af0, rawbuf=0x7fab58011018 "SET /* QNO 765 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", length=60, parser_state=0x7fabccddf660, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:7693
|
#20 0x0000559c32b931c2 in dispatch_command (command=COM_QUERY, thd=0x7fab58000af0, packet=0x7fab580087a1 "SET /* QNO 765 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", packet_length=60, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:1832
|
#21 0x0000559c32b91c3d in do_command (thd=0x7fab58000af0) at /data/src/10.2/sql/sql_parse.cc:1385
|
#22 0x0000559c32ce6ac9 in do_handle_one_connection (connect=0x559c35714280) at /data/src/10.2/sql/sql_connect.cc:1336
|
#23 0x0000559c32ce6834 in handle_one_connection (arg=0x559c35714280) at /data/src/10.2/sql/sql_connect.cc:1241
|
#24 0x00007fabd159c4a4 in start_thread (arg=0x7fabccde0700) at pthread_create.c:456
|
#25 0x00007fabcf6d0d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
|
Reproducible on 10.2-10.5.
Couldn't reproduce on 10.1.
Didn't observe immediate effects on a non-debug build (no hang, either).
Attachments
Issue Links
- relates to
-
MDEV-19665 Server hangs on client connection with concurrent setting of server audit variable
-
- Open
-
-
MDEV-29557 Assertion failure or server crash in test_plugin_options upon server audit installation
-
- Open
-
-
MDEV-25248 SET GLOBAL server_audit_logging=ON deadlock due to LOCK_global_system_variables
-
- Confirmed
-
Activity
Field | Original Value | New Value |
---|---|---|
Attachment | mdev22647_all_threads_mutex [ 51823 ] | |
Attachment | mdev22647_all_threads_assert [ 51824 ] | |
Description | Stub, need JIRA number |
_There are two failures here. They might turn out to be completely different bugs, but I can't separate them from each other, they happen interchangeable in the same scenario, which, apparently, involves a race condition upon installation of {{server_audit}} plugin in two different threads._
To reproduce: {noformat} git clone https://github.com/MariaDB/randgen --branch mdev22647 rqg-mdev22647 cd rqg-mdev22647 perl ./runall-new.pl --basedir=<basedir> --grammar=mdev22647.yy --threads=2 --skip-gendata --vardir=/dev/shm/var_mdev22647 --duration=30 {noformat} Remember to modify the command line to point at the real basedir. {noformat:title=10.2 2c4a2f20} safe_mutex: Trying to lock mutex at /data/src/10.2/sql/sql_plugin.cc, line 2998, when the mutex was already locked at /data/src/10.2/sql/sys_vars_shared.h, line 56 in thread T@20 200520 21:27:19 [ERROR] mysqld got signal 6 ; #5 0x00007f36556d742a in __GI_abort () at abort.c:89 #6 0x0000564ea6f3240d in safe_mutex_lock (mp=0x564ea7af0e80 <LOCK_global_system_variables>, my_flags=0, file=0x564ea6ff1760 "/data/src/10.2/sql/sql_plugin.cc", line=2998) at /data/src/10.2/mysys/thr_mutex.c:264 #7 0x0000564ea6581266 in inline_mysql_mutex_lock (that=0x564ea7af0e80 <LOCK_global_system_variables>, src_file=0x564ea6ff1760 "/data/src/10.2/sql/sql_plugin.cc", src_line=2998) at /data/src/10.2/include/mysql/psi/mysql_thread.h:688 #8 0x0000564ea65895d9 in sync_dynamic_session_variables (thd=0x7f35dc000af0, global_lock=true) at /data/src/10.2/sql/sql_plugin.cc:2998 #9 0x0000564ea658989f in intern_sys_var_ptr (thd=0x7f35dc000af0, offset=104, global_lock=true) at /data/src/10.2/sql/sql_plugin.cc:3065 #10 0x0000564ea6589a0a in mysql_sys_var_str (thd=0x7f35dc000af0, offset=104) at /data/src/10.2/sql/sql_plugin.cc:3112 #11 0x00007f365439f173 in get_loc_info (thd=0x7f35dc000af0) at /data/src/10.2/plugin/server_audit/server_audit.c:950 #12 0x00007f36543a2fde in mark_always_logged (thd=0x7f35dc000af0) at /data/src/10.2/plugin/server_audit/server_audit.c:2592 #13 0x00007f36543a3b25 in update_logging (thd=0x7f35dc000af0, var=0x7f36545a7460 <mysql_sysvar_logging>, var_ptr=0x7f36545a8594 <logging>, save=0x7f35dc0111d0) at /data/src/10.2/plugin/server_audit/server_audit.c:2862 #14 0x0000564ea658ada4 in sys_var_pluginvar::global_update (this=0x7f35e80d4670, thd=0x7f35dc000af0, var=0x7f35dc0111b0) at /data/src/10.2/sql/sql_plugin.cc:3516 #15 0x0000564ea649e2af in sys_var::update (this=0x7f35e80d4670, thd=0x7f35dc000af0, var=0x7f35dc0111b0) at /data/src/10.2/sql/set_var.cc:208 #16 0x0000564ea64a0030 in set_var::update (this=0x7f35dc0111b0, thd=0x7f35dc000af0) at /data/src/10.2/sql/set_var.cc:837 #17 0x0000564ea649fc12 in sql_set_variables (thd=0x7f35dc000af0, var_list=0x7f35dc005470, free=true) at /data/src/10.2/sql/set_var.cc:738 #18 0x0000564ea657005e in mysql_execute_command (thd=0x7f35dc000af0) at /data/src/10.2/sql/sql_parse.cc:4591 #19 0x0000564ea6579e6f in mysql_parse (thd=0x7f35dc000af0, rawbuf=0x7f35dc011018 "SET /* QNO 13 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", length=59, parser_state=0x7f36545f4660, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:7740 #20 0x0000564ea65681c2 in dispatch_command (command=COM_QUERY, thd=0x7f35dc000af0, packet=0x7f35dc0087a1 "SET /* QNO 13 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", packet_length=59, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:1832 #21 0x0000564ea6566c3d in do_command (thd=0x7f35dc000af0) at /data/src/10.2/sql/sql_parse.cc:1385 #22 0x0000564ea66bbac9 in do_handle_one_connection (connect=0x564ea9058280) at /data/src/10.2/sql/sql_connect.cc:1336 #23 0x0000564ea66bb834 in handle_one_connection (arg=0x564ea9058280) at /data/src/10.2/sql/sql_connect.cc:1241 #24 0x00007f36576574a4 in start_thread (arg=0x7f36545f5700) at pthread_create.c:456 #25 0x00007f365578bd0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97 {noformat} All threads' stack trace attached as [^mdev22647_all_threads_mutex] {noformat:title=10.2 2c4a2f20} mysqld: /data/src/10.2/sql/sql_audit.cc:115: void mysql_audit_acquire_plugins(THD*, ulong*): Assertion `!check_audit_mask(mysql_global_audit_mask, event_class_mask)' failed. 200520 21:45:53 [ERROR] mysqld got signal 6 ; #7 0x00007fabcf613f12 in __GI___assert_fail (assertion=0x559c33648c68 "!check_audit_mask(mysql_global_audit_mask, event_class_mask)", file=0x559c33648c28 "/data/src/10.2/sql/sql_audit.cc", line=115, function=0x559c33648da0 <mysql_audit_acquire_plugins(THD*, unsigned long*)::__PRETTY_FUNCTION__> "void mysql_audit_acquire_plugins(THD*, ulong*)") at assert.c:101 #8 0x0000559c32ce313c in mysql_audit_acquire_plugins (thd=0x7fab58000af0, event_class_mask=0x7fabccddca20) at /data/src/10.2/sql/sql_audit.cc:115 #9 0x0000559c32ce3838 in mysql_audit_notify (thd=0x7fab58000af0, event_class=0, event=0x7fabccddca70) at /data/src/10.2/sql/sql_audit.cc:432 #10 0x0000559c32b49b62 in mysql_audit_general (thd=0x7fab58000af0, event_subtype=1, error_code=1193, msg=0x7fabccddce40 "Unknown system variable 'SERVER_AUDIT_LOGGING'") at /data/src/10.2/sql/sql_audit.h:197 #11 0x0000559c32b4c5f6 in THD::raise_condition (this=0x7fab58000af0, sql_errno=1193, sqlstate=0x559c3362eb25 "HY000", level=Sql_condition::WARN_LEVEL_ERROR, msg=0x7fabccddce40 "Unknown system variable 'SERVER_AUDIT_LOGGING'") at /data/src/10.2/sql/sql_class.cc:1047 #12 0x0000559c32aa5015 in my_message_sql (error=1193, str=0x7fabccddce40 "Unknown system variable 'SERVER_AUDIT_LOGGING'", MyFlags=0) at /data/src/10.2/sql/mysqld.cc:3639 #13 0x0000559c33548db6 in my_error (nr=1193, MyFlags=0) at /data/src/10.2/mysys/my_error.c:125 #14 0x0000559c32bb3f05 in find_sys_var_ex (thd=0x7fab58000af0, str=0x7fab58011108 "SERVER_AUDIT_LOGGING", length=20, throw_error=false, locked=false) at /data/src/10.2/sql/sql_plugin.cc:2840 #15 0x0000559c32bb3f75 in find_sys_var (thd=0x7fab58000af0, str=0x7fab58011108 "SERVER_AUDIT_LOGGING", length=20) at /data/src/10.2/sql/sql_plugin.cc:2847 #16 0x0000559c32d8efc1 in find_sys_var_null_base (thd=0x7fab58000af0, tmp=0x7fabccdde720) at /data/src/10.2/sql/sql_yacc.yy:394 #17 0x0000559c32db7052 in MYSQLparse (thd=0x7fab58000af0) at /data/src/10.2/sql/sql_yacc.yy:15373 #18 0x0000559c32ba9e2e in parse_sql (thd=0x7fab58000af0, parser_state=0x7fabccddf660, creation_ctx=0x0, do_pfs_digest=true) at /data/src/10.2/sql/sql_parse.cc:10029 #19 0x0000559c32ba4ce1 in mysql_parse (thd=0x7fab58000af0, rawbuf=0x7fab58011018 "SET /* QNO 765 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", length=60, parser_state=0x7fabccddf660, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:7693 #20 0x0000559c32b931c2 in dispatch_command (command=COM_QUERY, thd=0x7fab58000af0, packet=0x7fab580087a1 "SET /* QNO 765 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", packet_length=60, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:1832 #21 0x0000559c32b91c3d in do_command (thd=0x7fab58000af0) at /data/src/10.2/sql/sql_parse.cc:1385 #22 0x0000559c32ce6ac9 in do_handle_one_connection (connect=0x559c35714280) at /data/src/10.2/sql/sql_connect.cc:1336 #23 0x0000559c32ce6834 in handle_one_connection (arg=0x559c35714280) at /data/src/10.2/sql/sql_connect.cc:1241 #24 0x00007fabd159c4a4 in start_thread (arg=0x7fabccde0700) at pthread_create.c:456 #25 0x00007fabcf6d0d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97 {noformat} Reproducible on 10.2-10.5. Couldn't reproduce on 10.1. |
Priority | Major [ 3 ] | Minor [ 4 ] |
Description |
_There are two failures here. They might turn out to be completely different bugs, but I can't separate them from each other, they happen interchangeable in the same scenario, which, apparently, involves a race condition upon installation of {{server_audit}} plugin in two different threads._
To reproduce: {noformat} git clone https://github.com/MariaDB/randgen --branch mdev22647 rqg-mdev22647 cd rqg-mdev22647 perl ./runall-new.pl --basedir=<basedir> --grammar=mdev22647.yy --threads=2 --skip-gendata --vardir=/dev/shm/var_mdev22647 --duration=30 {noformat} Remember to modify the command line to point at the real basedir. {noformat:title=10.2 2c4a2f20} safe_mutex: Trying to lock mutex at /data/src/10.2/sql/sql_plugin.cc, line 2998, when the mutex was already locked at /data/src/10.2/sql/sys_vars_shared.h, line 56 in thread T@20 200520 21:27:19 [ERROR] mysqld got signal 6 ; #5 0x00007f36556d742a in __GI_abort () at abort.c:89 #6 0x0000564ea6f3240d in safe_mutex_lock (mp=0x564ea7af0e80 <LOCK_global_system_variables>, my_flags=0, file=0x564ea6ff1760 "/data/src/10.2/sql/sql_plugin.cc", line=2998) at /data/src/10.2/mysys/thr_mutex.c:264 #7 0x0000564ea6581266 in inline_mysql_mutex_lock (that=0x564ea7af0e80 <LOCK_global_system_variables>, src_file=0x564ea6ff1760 "/data/src/10.2/sql/sql_plugin.cc", src_line=2998) at /data/src/10.2/include/mysql/psi/mysql_thread.h:688 #8 0x0000564ea65895d9 in sync_dynamic_session_variables (thd=0x7f35dc000af0, global_lock=true) at /data/src/10.2/sql/sql_plugin.cc:2998 #9 0x0000564ea658989f in intern_sys_var_ptr (thd=0x7f35dc000af0, offset=104, global_lock=true) at /data/src/10.2/sql/sql_plugin.cc:3065 #10 0x0000564ea6589a0a in mysql_sys_var_str (thd=0x7f35dc000af0, offset=104) at /data/src/10.2/sql/sql_plugin.cc:3112 #11 0x00007f365439f173 in get_loc_info (thd=0x7f35dc000af0) at /data/src/10.2/plugin/server_audit/server_audit.c:950 #12 0x00007f36543a2fde in mark_always_logged (thd=0x7f35dc000af0) at /data/src/10.2/plugin/server_audit/server_audit.c:2592 #13 0x00007f36543a3b25 in update_logging (thd=0x7f35dc000af0, var=0x7f36545a7460 <mysql_sysvar_logging>, var_ptr=0x7f36545a8594 <logging>, save=0x7f35dc0111d0) at /data/src/10.2/plugin/server_audit/server_audit.c:2862 #14 0x0000564ea658ada4 in sys_var_pluginvar::global_update (this=0x7f35e80d4670, thd=0x7f35dc000af0, var=0x7f35dc0111b0) at /data/src/10.2/sql/sql_plugin.cc:3516 #15 0x0000564ea649e2af in sys_var::update (this=0x7f35e80d4670, thd=0x7f35dc000af0, var=0x7f35dc0111b0) at /data/src/10.2/sql/set_var.cc:208 #16 0x0000564ea64a0030 in set_var::update (this=0x7f35dc0111b0, thd=0x7f35dc000af0) at /data/src/10.2/sql/set_var.cc:837 #17 0x0000564ea649fc12 in sql_set_variables (thd=0x7f35dc000af0, var_list=0x7f35dc005470, free=true) at /data/src/10.2/sql/set_var.cc:738 #18 0x0000564ea657005e in mysql_execute_command (thd=0x7f35dc000af0) at /data/src/10.2/sql/sql_parse.cc:4591 #19 0x0000564ea6579e6f in mysql_parse (thd=0x7f35dc000af0, rawbuf=0x7f35dc011018 "SET /* QNO 13 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", length=59, parser_state=0x7f36545f4660, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:7740 #20 0x0000564ea65681c2 in dispatch_command (command=COM_QUERY, thd=0x7f35dc000af0, packet=0x7f35dc0087a1 "SET /* QNO 13 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", packet_length=59, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:1832 #21 0x0000564ea6566c3d in do_command (thd=0x7f35dc000af0) at /data/src/10.2/sql/sql_parse.cc:1385 #22 0x0000564ea66bbac9 in do_handle_one_connection (connect=0x564ea9058280) at /data/src/10.2/sql/sql_connect.cc:1336 #23 0x0000564ea66bb834 in handle_one_connection (arg=0x564ea9058280) at /data/src/10.2/sql/sql_connect.cc:1241 #24 0x00007f36576574a4 in start_thread (arg=0x7f36545f5700) at pthread_create.c:456 #25 0x00007f365578bd0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97 {noformat} All threads' stack trace attached as [^mdev22647_all_threads_mutex] {noformat:title=10.2 2c4a2f20} mysqld: /data/src/10.2/sql/sql_audit.cc:115: void mysql_audit_acquire_plugins(THD*, ulong*): Assertion `!check_audit_mask(mysql_global_audit_mask, event_class_mask)' failed. 200520 21:45:53 [ERROR] mysqld got signal 6 ; #7 0x00007fabcf613f12 in __GI___assert_fail (assertion=0x559c33648c68 "!check_audit_mask(mysql_global_audit_mask, event_class_mask)", file=0x559c33648c28 "/data/src/10.2/sql/sql_audit.cc", line=115, function=0x559c33648da0 <mysql_audit_acquire_plugins(THD*, unsigned long*)::__PRETTY_FUNCTION__> "void mysql_audit_acquire_plugins(THD*, ulong*)") at assert.c:101 #8 0x0000559c32ce313c in mysql_audit_acquire_plugins (thd=0x7fab58000af0, event_class_mask=0x7fabccddca20) at /data/src/10.2/sql/sql_audit.cc:115 #9 0x0000559c32ce3838 in mysql_audit_notify (thd=0x7fab58000af0, event_class=0, event=0x7fabccddca70) at /data/src/10.2/sql/sql_audit.cc:432 #10 0x0000559c32b49b62 in mysql_audit_general (thd=0x7fab58000af0, event_subtype=1, error_code=1193, msg=0x7fabccddce40 "Unknown system variable 'SERVER_AUDIT_LOGGING'") at /data/src/10.2/sql/sql_audit.h:197 #11 0x0000559c32b4c5f6 in THD::raise_condition (this=0x7fab58000af0, sql_errno=1193, sqlstate=0x559c3362eb25 "HY000", level=Sql_condition::WARN_LEVEL_ERROR, msg=0x7fabccddce40 "Unknown system variable 'SERVER_AUDIT_LOGGING'") at /data/src/10.2/sql/sql_class.cc:1047 #12 0x0000559c32aa5015 in my_message_sql (error=1193, str=0x7fabccddce40 "Unknown system variable 'SERVER_AUDIT_LOGGING'", MyFlags=0) at /data/src/10.2/sql/mysqld.cc:3639 #13 0x0000559c33548db6 in my_error (nr=1193, MyFlags=0) at /data/src/10.2/mysys/my_error.c:125 #14 0x0000559c32bb3f05 in find_sys_var_ex (thd=0x7fab58000af0, str=0x7fab58011108 "SERVER_AUDIT_LOGGING", length=20, throw_error=false, locked=false) at /data/src/10.2/sql/sql_plugin.cc:2840 #15 0x0000559c32bb3f75 in find_sys_var (thd=0x7fab58000af0, str=0x7fab58011108 "SERVER_AUDIT_LOGGING", length=20) at /data/src/10.2/sql/sql_plugin.cc:2847 #16 0x0000559c32d8efc1 in find_sys_var_null_base (thd=0x7fab58000af0, tmp=0x7fabccdde720) at /data/src/10.2/sql/sql_yacc.yy:394 #17 0x0000559c32db7052 in MYSQLparse (thd=0x7fab58000af0) at /data/src/10.2/sql/sql_yacc.yy:15373 #18 0x0000559c32ba9e2e in parse_sql (thd=0x7fab58000af0, parser_state=0x7fabccddf660, creation_ctx=0x0, do_pfs_digest=true) at /data/src/10.2/sql/sql_parse.cc:10029 #19 0x0000559c32ba4ce1 in mysql_parse (thd=0x7fab58000af0, rawbuf=0x7fab58011018 "SET /* QNO 765 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", length=60, parser_state=0x7fabccddf660, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:7693 #20 0x0000559c32b931c2 in dispatch_command (command=COM_QUERY, thd=0x7fab58000af0, packet=0x7fab580087a1 "SET /* QNO 765 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", packet_length=60, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:1832 #21 0x0000559c32b91c3d in do_command (thd=0x7fab58000af0) at /data/src/10.2/sql/sql_parse.cc:1385 #22 0x0000559c32ce6ac9 in do_handle_one_connection (connect=0x559c35714280) at /data/src/10.2/sql/sql_connect.cc:1336 #23 0x0000559c32ce6834 in handle_one_connection (arg=0x559c35714280) at /data/src/10.2/sql/sql_connect.cc:1241 #24 0x00007fabd159c4a4 in start_thread (arg=0x7fabccde0700) at pthread_create.c:456 #25 0x00007fabcf6d0d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97 {noformat} Reproducible on 10.2-10.5. Couldn't reproduce on 10.1. |
_There are two failures here. They might turn out to be completely different bugs, but I can't separate them from each other, they happen interchangeable in the same scenario, which, apparently, involves a race condition upon installation of {{server_audit}} plugin in two different threads._
To reproduce: {noformat} git clone https://github.com/MariaDB/randgen --branch mdev22647 rqg-mdev22647 cd rqg-mdev22647 perl ./runall-new.pl --basedir=<basedir> --grammar=mdev22647.yy --threads=2 --skip-gendata --vardir=/dev/shm/var_mdev22647 --duration=30 {noformat} Remember to modify the command line to point at the real basedir. {noformat:title=10.2 2c4a2f20} safe_mutex: Trying to lock mutex at /data/src/10.2/sql/sql_plugin.cc, line 2998, when the mutex was already locked at /data/src/10.2/sql/sys_vars_shared.h, line 56 in thread T@20 200520 21:27:19 [ERROR] mysqld got signal 6 ; #5 0x00007f36556d742a in __GI_abort () at abort.c:89 #6 0x0000564ea6f3240d in safe_mutex_lock (mp=0x564ea7af0e80 <LOCK_global_system_variables>, my_flags=0, file=0x564ea6ff1760 "/data/src/10.2/sql/sql_plugin.cc", line=2998) at /data/src/10.2/mysys/thr_mutex.c:264 #7 0x0000564ea6581266 in inline_mysql_mutex_lock (that=0x564ea7af0e80 <LOCK_global_system_variables>, src_file=0x564ea6ff1760 "/data/src/10.2/sql/sql_plugin.cc", src_line=2998) at /data/src/10.2/include/mysql/psi/mysql_thread.h:688 #8 0x0000564ea65895d9 in sync_dynamic_session_variables (thd=0x7f35dc000af0, global_lock=true) at /data/src/10.2/sql/sql_plugin.cc:2998 #9 0x0000564ea658989f in intern_sys_var_ptr (thd=0x7f35dc000af0, offset=104, global_lock=true) at /data/src/10.2/sql/sql_plugin.cc:3065 #10 0x0000564ea6589a0a in mysql_sys_var_str (thd=0x7f35dc000af0, offset=104) at /data/src/10.2/sql/sql_plugin.cc:3112 #11 0x00007f365439f173 in get_loc_info (thd=0x7f35dc000af0) at /data/src/10.2/plugin/server_audit/server_audit.c:950 #12 0x00007f36543a2fde in mark_always_logged (thd=0x7f35dc000af0) at /data/src/10.2/plugin/server_audit/server_audit.c:2592 #13 0x00007f36543a3b25 in update_logging (thd=0x7f35dc000af0, var=0x7f36545a7460 <mysql_sysvar_logging>, var_ptr=0x7f36545a8594 <logging>, save=0x7f35dc0111d0) at /data/src/10.2/plugin/server_audit/server_audit.c:2862 #14 0x0000564ea658ada4 in sys_var_pluginvar::global_update (this=0x7f35e80d4670, thd=0x7f35dc000af0, var=0x7f35dc0111b0) at /data/src/10.2/sql/sql_plugin.cc:3516 #15 0x0000564ea649e2af in sys_var::update (this=0x7f35e80d4670, thd=0x7f35dc000af0, var=0x7f35dc0111b0) at /data/src/10.2/sql/set_var.cc:208 #16 0x0000564ea64a0030 in set_var::update (this=0x7f35dc0111b0, thd=0x7f35dc000af0) at /data/src/10.2/sql/set_var.cc:837 #17 0x0000564ea649fc12 in sql_set_variables (thd=0x7f35dc000af0, var_list=0x7f35dc005470, free=true) at /data/src/10.2/sql/set_var.cc:738 #18 0x0000564ea657005e in mysql_execute_command (thd=0x7f35dc000af0) at /data/src/10.2/sql/sql_parse.cc:4591 #19 0x0000564ea6579e6f in mysql_parse (thd=0x7f35dc000af0, rawbuf=0x7f35dc011018 "SET /* QNO 13 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", length=59, parser_state=0x7f36545f4660, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:7740 #20 0x0000564ea65681c2 in dispatch_command (command=COM_QUERY, thd=0x7f35dc000af0, packet=0x7f35dc0087a1 "SET /* QNO 13 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", packet_length=59, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:1832 #21 0x0000564ea6566c3d in do_command (thd=0x7f35dc000af0) at /data/src/10.2/sql/sql_parse.cc:1385 #22 0x0000564ea66bbac9 in do_handle_one_connection (connect=0x564ea9058280) at /data/src/10.2/sql/sql_connect.cc:1336 #23 0x0000564ea66bb834 in handle_one_connection (arg=0x564ea9058280) at /data/src/10.2/sql/sql_connect.cc:1241 #24 0x00007f36576574a4 in start_thread (arg=0x7f36545f5700) at pthread_create.c:456 #25 0x00007f365578bd0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97 {noformat} All threads' stack trace attached as [^mdev22647_all_threads_mutex] {noformat:title=10.2 2c4a2f20} mysqld: /data/src/10.2/sql/sql_audit.cc:115: void mysql_audit_acquire_plugins(THD*, ulong*): Assertion `!check_audit_mask(mysql_global_audit_mask, event_class_mask)' failed. 200520 21:45:53 [ERROR] mysqld got signal 6 ; #7 0x00007fabcf613f12 in __GI___assert_fail (assertion=0x559c33648c68 "!check_audit_mask(mysql_global_audit_mask, event_class_mask)", file=0x559c33648c28 "/data/src/10.2/sql/sql_audit.cc", line=115, function=0x559c33648da0 <mysql_audit_acquire_plugins(THD*, unsigned long*)::__PRETTY_FUNCTION__> "void mysql_audit_acquire_plugins(THD*, ulong*)") at assert.c:101 #8 0x0000559c32ce313c in mysql_audit_acquire_plugins (thd=0x7fab58000af0, event_class_mask=0x7fabccddca20) at /data/src/10.2/sql/sql_audit.cc:115 #9 0x0000559c32ce3838 in mysql_audit_notify (thd=0x7fab58000af0, event_class=0, event=0x7fabccddca70) at /data/src/10.2/sql/sql_audit.cc:432 #10 0x0000559c32b49b62 in mysql_audit_general (thd=0x7fab58000af0, event_subtype=1, error_code=1193, msg=0x7fabccddce40 "Unknown system variable 'SERVER_AUDIT_LOGGING'") at /data/src/10.2/sql/sql_audit.h:197 #11 0x0000559c32b4c5f6 in THD::raise_condition (this=0x7fab58000af0, sql_errno=1193, sqlstate=0x559c3362eb25 "HY000", level=Sql_condition::WARN_LEVEL_ERROR, msg=0x7fabccddce40 "Unknown system variable 'SERVER_AUDIT_LOGGING'") at /data/src/10.2/sql/sql_class.cc:1047 #12 0x0000559c32aa5015 in my_message_sql (error=1193, str=0x7fabccddce40 "Unknown system variable 'SERVER_AUDIT_LOGGING'", MyFlags=0) at /data/src/10.2/sql/mysqld.cc:3639 #13 0x0000559c33548db6 in my_error (nr=1193, MyFlags=0) at /data/src/10.2/mysys/my_error.c:125 #14 0x0000559c32bb3f05 in find_sys_var_ex (thd=0x7fab58000af0, str=0x7fab58011108 "SERVER_AUDIT_LOGGING", length=20, throw_error=false, locked=false) at /data/src/10.2/sql/sql_plugin.cc:2840 #15 0x0000559c32bb3f75 in find_sys_var (thd=0x7fab58000af0, str=0x7fab58011108 "SERVER_AUDIT_LOGGING", length=20) at /data/src/10.2/sql/sql_plugin.cc:2847 #16 0x0000559c32d8efc1 in find_sys_var_null_base (thd=0x7fab58000af0, tmp=0x7fabccdde720) at /data/src/10.2/sql/sql_yacc.yy:394 #17 0x0000559c32db7052 in MYSQLparse (thd=0x7fab58000af0) at /data/src/10.2/sql/sql_yacc.yy:15373 #18 0x0000559c32ba9e2e in parse_sql (thd=0x7fab58000af0, parser_state=0x7fabccddf660, creation_ctx=0x0, do_pfs_digest=true) at /data/src/10.2/sql/sql_parse.cc:10029 #19 0x0000559c32ba4ce1 in mysql_parse (thd=0x7fab58000af0, rawbuf=0x7fab58011018 "SET /* QNO 765 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", length=60, parser_state=0x7fabccddf660, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:7693 #20 0x0000559c32b931c2 in dispatch_command (command=COM_QUERY, thd=0x7fab58000af0, packet=0x7fab580087a1 "SET /* QNO 765 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", packet_length=60, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:1832 #21 0x0000559c32b91c3d in do_command (thd=0x7fab58000af0) at /data/src/10.2/sql/sql_parse.cc:1385 #22 0x0000559c32ce6ac9 in do_handle_one_connection (connect=0x559c35714280) at /data/src/10.2/sql/sql_connect.cc:1336 #23 0x0000559c32ce6834 in handle_one_connection (arg=0x559c35714280) at /data/src/10.2/sql/sql_connect.cc:1241 #24 0x00007fabd159c4a4 in start_thread (arg=0x7fabccde0700) at pthread_create.c:456 #25 0x00007fabcf6d0d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97 {noformat} Reproducible on 10.2-10.5. Couldn't reproduce on 10.1. Didn't observe immediate effects on a non-debug build (no hang, either). |
Link | This issue relates to MDEV-19665 [ MDEV-19665 ] |
Assignee | Elena Stepanova [ elenst ] | Alexey Botchkov [ holyfoot ] |
Component/s | Plugins [ 10118 ] |
Priority | Minor [ 4 ] | Major [ 3 ] |
Link | This issue is duplicated by MDEV-25248 [ MDEV-25248 ] |
Status | Open [ 1 ] | Confirmed [ 10101 ] |
Fix Version/s | 10.6 [ 24028 ] | |
Affects Version/s | 10.6 [ 24028 ] |
Fix Version/s | 10.7 [ 24805 ] | |
Affects Version/s | 10.7 [ 24805 ] |
Workflow | MariaDB v3 [ 108956 ] | MariaDB v4 [ 144252 ] |
Fix Version/s | 10.8 [ 26121 ] | |
Affects Version/s | 10.8 [ 26121 ] |
Fix Version/s | 10.2 [ 14601 ] |
Summary | Assertion `!check_audit_mask(mysql_global_audit_mask, event_class_mask)' or Trying to lock mutex when the mutex was already locked | Assertion `!check_audit_mask(mysql_global_audit_mask, event_class_mask)' |
Fix Version/s | 10.3.37 [ 28404 ] | |
Fix Version/s | 10.4.27 [ 28405 ] | |
Fix Version/s | 10.5.18 [ 28421 ] | |
Fix Version/s | 10.6.11 [ 28441 ] | |
Fix Version/s | 10.7.7 [ 28442 ] | |
Fix Version/s | 10.8.6 [ 28443 ] | |
Fix Version/s | 10.9.4 [ 28444 ] | |
Fix Version/s | 10.3 [ 22126 ] | |
Fix Version/s | 10.4 [ 22408 ] | |
Fix Version/s | 10.5 [ 23123 ] | |
Fix Version/s | 10.6 [ 24028 ] | |
Fix Version/s | 10.7 [ 24805 ] | |
Fix Version/s | 10.8 [ 26121 ] | |
Assignee | Alexey Botchkov [ holyfoot ] | Sergei Golubchik [ serg ] |
Resolution | Fixed [ 1 ] | |
Status | Confirmed [ 10101 ] | Closed [ 6 ] |
Link | This issue is duplicated by MDEV-25248 [ MDEV-25248 ] |
Link | This issue relates to MDEV-25248 [ MDEV-25248 ] |
Link | This issue relates to MDEV-29557 [ MDEV-29557 ] |
I got this today on latest 10.5 code while running the full mtr test suite.
mariadbd: /my/maria-10.5/sql/sql_audit.cc:116: void mysql_audit_acquire_plugins(THD*, ulong*): Assertion `!check_audit_mask(mysql_global_audit_mask, event_class_mask)' failed.