Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-22647

Assertion `!check_audit_mask(mysql_global_audit_mask, event_class_mask)' or Trying to lock mutex when the mutex was already locked

    XMLWordPrintable

    Details

      Description

      There are two failures here. They might turn out to be completely different bugs, but I can't separate them from each other, they happen interchangeable in the same scenario, which, apparently, involves a race condition upon installation of server_audit plugin in two different threads.

      To reproduce:

      git clone https://github.com/MariaDB/randgen --branch mdev22647 rqg-mdev22647
      cd rqg-mdev22647
      perl ./runall-new.pl --basedir=<basedir> --grammar=mdev22647.yy --threads=2 --skip-gendata --vardir=/dev/shm/var_mdev22647 --duration=30
      

      Remember to modify the command line to point at the real basedir.

      10.2 2c4a2f20

      safe_mutex: Trying to lock mutex at /data/src/10.2/sql/sql_plugin.cc, line 2998, when the mutex was already locked at /data/src/10.2/sql/sys_vars_shared.h, line 56 in thread
       T@20
      200520 21:27:19 [ERROR] mysqld got signal 6 ;
       
      #5  0x00007f36556d742a in __GI_abort () at abort.c:89
      #6  0x0000564ea6f3240d in safe_mutex_lock (mp=0x564ea7af0e80 <LOCK_global_system_variables>, my_flags=0, file=0x564ea6ff1760 "/data/src/10.2/sql/sql_plugin.cc", line=2998) at /data/src/10.2/mysys/thr_mutex.c:264
      #7  0x0000564ea6581266 in inline_mysql_mutex_lock (that=0x564ea7af0e80 <LOCK_global_system_variables>, src_file=0x564ea6ff1760 "/data/src/10.2/sql/sql_plugin.cc", src_line=2998) at /data/src/10.2/include/mysql/psi/mysql_thread.h:688
      #8  0x0000564ea65895d9 in sync_dynamic_session_variables (thd=0x7f35dc000af0, global_lock=true) at /data/src/10.2/sql/sql_plugin.cc:2998
      #9  0x0000564ea658989f in intern_sys_var_ptr (thd=0x7f35dc000af0, offset=104, global_lock=true) at /data/src/10.2/sql/sql_plugin.cc:3065
      #10 0x0000564ea6589a0a in mysql_sys_var_str (thd=0x7f35dc000af0, offset=104) at /data/src/10.2/sql/sql_plugin.cc:3112
      #11 0x00007f365439f173 in get_loc_info (thd=0x7f35dc000af0) at /data/src/10.2/plugin/server_audit/server_audit.c:950
      #12 0x00007f36543a2fde in mark_always_logged (thd=0x7f35dc000af0) at /data/src/10.2/plugin/server_audit/server_audit.c:2592
      #13 0x00007f36543a3b25 in update_logging (thd=0x7f35dc000af0, var=0x7f36545a7460 <mysql_sysvar_logging>, var_ptr=0x7f36545a8594 <logging>, save=0x7f35dc0111d0) at /data/src/10.2/plugin/server_audit/server_audit.c:2862
      #14 0x0000564ea658ada4 in sys_var_pluginvar::global_update (this=0x7f35e80d4670, thd=0x7f35dc000af0, var=0x7f35dc0111b0) at /data/src/10.2/sql/sql_plugin.cc:3516
      #15 0x0000564ea649e2af in sys_var::update (this=0x7f35e80d4670, thd=0x7f35dc000af0, var=0x7f35dc0111b0) at /data/src/10.2/sql/set_var.cc:208
      #16 0x0000564ea64a0030 in set_var::update (this=0x7f35dc0111b0, thd=0x7f35dc000af0) at /data/src/10.2/sql/set_var.cc:837
      #17 0x0000564ea649fc12 in sql_set_variables (thd=0x7f35dc000af0, var_list=0x7f35dc005470, free=true) at /data/src/10.2/sql/set_var.cc:738
      #18 0x0000564ea657005e in mysql_execute_command (thd=0x7f35dc000af0) at /data/src/10.2/sql/sql_parse.cc:4591
      #19 0x0000564ea6579e6f in mysql_parse (thd=0x7f35dc000af0, rawbuf=0x7f35dc011018 "SET /* QNO 13 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", length=59, parser_state=0x7f36545f4660, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:7740
      #20 0x0000564ea65681c2 in dispatch_command (command=COM_QUERY, thd=0x7f35dc000af0, packet=0x7f35dc0087a1 "SET /* QNO 13 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", packet_length=59, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:1832
      #21 0x0000564ea6566c3d in do_command (thd=0x7f35dc000af0) at /data/src/10.2/sql/sql_parse.cc:1385
      #22 0x0000564ea66bbac9 in do_handle_one_connection (connect=0x564ea9058280) at /data/src/10.2/sql/sql_connect.cc:1336
      #23 0x0000564ea66bb834 in handle_one_connection (arg=0x564ea9058280) at /data/src/10.2/sql/sql_connect.cc:1241
      #24 0x00007f36576574a4 in start_thread (arg=0x7f36545f5700) at pthread_create.c:456
      #25 0x00007f365578bd0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
      

      All threads' stack trace attached as mdev22647_all_threads_mutex

      10.2 2c4a2f20

      mysqld: /data/src/10.2/sql/sql_audit.cc:115: void mysql_audit_acquire_plugins(THD*, ulong*): Assertion `!check_audit_mask(mysql_global_audit_mask, event_class_mask)' failed.
      200520 21:45:53 [ERROR] mysqld got signal 6 ;
       
      #7  0x00007fabcf613f12 in __GI___assert_fail (assertion=0x559c33648c68 "!check_audit_mask(mysql_global_audit_mask, event_class_mask)", file=0x559c33648c28 "/data/src/10.2/sql/sql_audit.cc", line=115, function=0x559c33648da0 <mysql_audit_acquire_plugins(THD*, unsigned long*)::__PRETTY_FUNCTION__> "void mysql_audit_acquire_plugins(THD*, ulong*)") at assert.c:101
      #8  0x0000559c32ce313c in mysql_audit_acquire_plugins (thd=0x7fab58000af0, event_class_mask=0x7fabccddca20) at /data/src/10.2/sql/sql_audit.cc:115
      #9  0x0000559c32ce3838 in mysql_audit_notify (thd=0x7fab58000af0, event_class=0, event=0x7fabccddca70) at /data/src/10.2/sql/sql_audit.cc:432
      #10 0x0000559c32b49b62 in mysql_audit_general (thd=0x7fab58000af0, event_subtype=1, error_code=1193, msg=0x7fabccddce40 "Unknown system variable 'SERVER_AUDIT_LOGGING'") at /data/src/10.2/sql/sql_audit.h:197
      #11 0x0000559c32b4c5f6 in THD::raise_condition (this=0x7fab58000af0, sql_errno=1193, sqlstate=0x559c3362eb25 "HY000", level=Sql_condition::WARN_LEVEL_ERROR, msg=0x7fabccddce40 "Unknown system variable 'SERVER_AUDIT_LOGGING'") at /data/src/10.2/sql/sql_class.cc:1047
      #12 0x0000559c32aa5015 in my_message_sql (error=1193, str=0x7fabccddce40 "Unknown system variable 'SERVER_AUDIT_LOGGING'", MyFlags=0) at /data/src/10.2/sql/mysqld.cc:3639
      #13 0x0000559c33548db6 in my_error (nr=1193, MyFlags=0) at /data/src/10.2/mysys/my_error.c:125
      #14 0x0000559c32bb3f05 in find_sys_var_ex (thd=0x7fab58000af0, str=0x7fab58011108 "SERVER_AUDIT_LOGGING", length=20, throw_error=false, locked=false) at /data/src/10.2/sql/sql_plugin.cc:2840
      #15 0x0000559c32bb3f75 in find_sys_var (thd=0x7fab58000af0, str=0x7fab58011108 "SERVER_AUDIT_LOGGING", length=20) at /data/src/10.2/sql/sql_plugin.cc:2847
      #16 0x0000559c32d8efc1 in find_sys_var_null_base (thd=0x7fab58000af0, tmp=0x7fabccdde720) at /data/src/10.2/sql/sql_yacc.yy:394
      #17 0x0000559c32db7052 in MYSQLparse (thd=0x7fab58000af0) at /data/src/10.2/sql/sql_yacc.yy:15373
      #18 0x0000559c32ba9e2e in parse_sql (thd=0x7fab58000af0, parser_state=0x7fabccddf660, creation_ctx=0x0, do_pfs_digest=true) at /data/src/10.2/sql/sql_parse.cc:10029
      #19 0x0000559c32ba4ce1 in mysql_parse (thd=0x7fab58000af0, rawbuf=0x7fab58011018 "SET /* QNO 765 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", length=60, parser_state=0x7fabccddf660, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:7693
      #20 0x0000559c32b931c2 in dispatch_command (command=COM_QUERY, thd=0x7fab58000af0, packet=0x7fab580087a1 "SET /* QNO 765 CON_ID 13 */ GLOBAL SERVER_AUDIT_LOGGING = ON", packet_length=60, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:1832
      #21 0x0000559c32b91c3d in do_command (thd=0x7fab58000af0) at /data/src/10.2/sql/sql_parse.cc:1385
      #22 0x0000559c32ce6ac9 in do_handle_one_connection (connect=0x559c35714280) at /data/src/10.2/sql/sql_connect.cc:1336
      #23 0x0000559c32ce6834 in handle_one_connection (arg=0x559c35714280) at /data/src/10.2/sql/sql_connect.cc:1241
      #24 0x00007fabd159c4a4 in start_thread (arg=0x7fabccde0700) at pthread_create.c:456
      #25 0x00007fabcf6d0d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
      

      Reproducible on 10.2-10.5.
      Couldn't reproduce on 10.1.
      Didn't observe immediate effects on a non-debug build (no hang, either).

        Attachments

        1. mdev22647_all_threads_assert
          38 kB
          Elena Stepanova
        2. mdev22647_all_threads_mutex
          36 kB
          Elena Stepanova

          Issue Links

            Activity

              People

              Assignee:
              holyfoot Alexey Botchkov
              Reporter:
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated: