Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-19918

Server hangs or crashes while trying to lock mutex when the mutex was already locked upon startup with server_audit and orphan records in mysql.plugin

    Details

    • Type: Bug
    • Status: In Progress (View Workflow)
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 10.4, 10.4.7, 10.4.8
    • Fix Version/s: 10.4
    • Component/s: Plugin - Audit, Plugins
    • Labels:
      None

      Description

      When the server is started with --plugin-load-add=server_audit and it has records in mysql.plugin referring non-existent libraries, it hangs on a release build or crashes on a debug build.

      10.4 9e0ed0fa

      Trying to lock mutex at /data/src/10.4/sql/sql_plugin.cc, line 2399, when the mutex was alread
      y locked at /data/src/10.4/sql/sql_plugin.cc, line 1854 in thread T@1
      190701 20:58:06 [ERROR] mysqld got signal 6 ;
       
      #5  0x00007f1adb13042a in __GI_abort () at abort.c:89
      #6  0x00005620a7381932 in safe_mutex_lock (mp=0x5620a80e95e0 <LOCK_plugin>, my_flags=0, file=0x5620a747d330 "/data/src/10.4/sql/sql_plugin.cc", line=2399) at /data/src/10.4/mysys/thr_mutex.c:265
      #7  0x00005620a6889ef5 in inline_mysql_mutex_lock (that=0x5620a80e95e0 <LOCK_plugin>, src_file=0x5620a747d330 "/data/src/10.4/sql/sql_plugin.cc", src_line=2399) at /data/src/10.4/include/mysql/psi/mysql_thread.h:710
      #8  0x00005620a6890a99 in plugin_foreach_with_mask (thd=0x5620ab294120, func=0x5620a69e269f <acquire_plugins(THD*, plugin_ref, void*)>, type=5, state_mask=8, arg=0x7ffee2db8700) at /data/src/10.4/sql/sql_plugin.cc:2399
      #9  0x00005620a69e28a1 in mysql_audit_acquire_plugins (thd=0x5620ab294120, event_class_mask=0x7ffee2db8700) at /data/src/10.4/sql/sql_audit.cc:120
      #10 0x00005620a69e2f19 in mysql_audit_notify (thd=0x5620ab294120, event_class=0, event=0x7ffee2db8770) at /data/src/10.4/sql/sql_audit.cc:417
      #11 0x00005620a67fd6d7 in mysql_audit_general (thd=0x5620ab294120, event_subtype=1, error_code=1126, msg=0x7ffee2db8fb0 "Can't open shared library '/data/bld/10.4-debug/lib/plugin/bar.so' (errno: 0, cannot open shared object file: No such file or directory)") at /data/src/10.4/sql/sql_audit.h:191
      #12 0x00005620a68004e1 in THD::raise_condition (this=0x5620ab294120, sql_errno=1126, sqlstate=0x5620a74935dd "HY000", level=Sql_state_errno_level::WARN_LEVEL_ERROR, ucid=..., msg=0x7ffee2db8fb0 "Can't open shared library '/data/bld/10.4-debug/lib/plugin/bar.so' (errno: 0, cannot open shared object file: No such file or directory)") at /data/src/10.4/sql/sql_class.cc:1076
      #13 0x00005620a674e464 in THD::raise_condition (this=0x5620ab294120, sql_errno=1126, sqlstate=0x0, level=Sql_state_errno_level::WARN_LEVEL_ERROR, msg=0x7ffee2db8fb0 "Can't open shared library '/data/bld/10.4-debug/lib/plugin/bar.so' (errno: 0, cannot open shared object file: No such file or directory)") at /data/src/10.4/sql/sql_class.h:4354
      #14 0x00005620a67413ff in my_message_sql (error=1126, str=0x7ffee2db8fb0 "Can't open shared library '/data/bld/10.4-debug/lib/plugin/bar.so' (errno: 0, cannot open shared object file: No such file or directory)", MyFlags=64) at /data/src/10.4/sql/mysqld.cc:3371
      #15 0x00005620a736d933 in my_error (nr=1126, MyFlags=64) at /data/src/10.4/mysys/my_error.c:125
      #16 0x00005620a688b4ce in plugin_dl_add (dl=0x7ffee2db9b50, MyFlags=64) at /data/src/10.4/sql/sql_plugin.cc:753
      #17 0x00005620a688c4ff in plugin_add (tmp_root=0x7ffee2dba5c0, if_not_exists=false, name=0x7ffee2db9b40, dl=0x7ffee2db9b50, MyFlags=64) at /data/src/10.4/sql/sql_plugin.cc:1100
      #18 0x00005620a688ed03 in plugin_load (tmp_root=0x7ffee2dba5c0) at /data/src/10.4/sql/sql_plugin.cc:1855
      #19 0x00005620a688e641 in plugin_init (argc=0x5620a80e5fd8 <remaining_argc>, argv=0x5620aa5f3a48, flags=0) at /data/src/10.4/sql/sql_plugin.cc:1734
      #20 0x00005620a6744cce in init_server_components () at /data/src/10.4/sql/mysqld.cc:5202
      #21 0x00005620a6745e99 in mysqld_main (argc=12, argv=0x5620aa5f3a48) at /data/src/10.4/sql/mysqld.cc:5725
      #22 0x00005620a673a730 in main (argc=12, argv=0x7ffee2dbb008) at /data/src/10.4/sql/main.cc:25
      

      The test case below is artificial, but there is a quite real-life scenario which represents the problem.

      • on Debian/Ubuntu, install mariadb-server and mariadb-plugin-spider packages;
      • add plugin-load-add=server_audit at the end of /etc/mysql/my.cnf;
      • uninstall mariadb-plugin-spider
      • restart the service => it hangs upon restart

      Run the test case below with --plugin-load-add=server_audit
      It seems important, installing it via INSTALL SONAME doesn't cause the same effect.

      insert into mysql.plugin values ('foo','bar.so');
       
      --source include/restart_mysqld.inc
       
      # Cleanup
      delete from mysql.plugin where name = 'foo';
      

      Couldn't reproduce on 10.2-10.3.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                holyfoot Alexey Botchkov
                Reporter:
                elenst Elena Stepanova
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated: