Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-16676

Using malloc-lib=jemalloc in MariaDB 10.2 causes non-critical error about missing mysql_config on startup

Details

    Description

      When using a custom memory allocator, e.g.

      [mysqld_safe]
      malloc-lib=jemalloc
      

      When starting MariaDB, the mysqld_safe script attempts to learn which lib directories to use by executing mysql_config command.

      However this is only provided by the MariaDB-devel package, and so isn't installed on most production servers.

      As such, we get an error each time we start MariaDB, in the form of:

      180703 12:52:39 mysqld_safe Can not run mysql_config --variable=pkglibdir from '/usr/bin/mysql_config'
      

      MariaDB still starts OK because the mysqld_safe script also looks in the normal lib directories, where jemalloc is installed to by CentOS.

      I do not think we need to see this error, as it will be common for mysql_config not to be installed. Perhaps only show the error if the allocator library isn't found in any of the normal library directories?

      Attachments

        Issue Links

          Activity

            tomp Tom Parrott created issue -
            serg Sergei Golubchik made changes -
            Field Original Value New Value
            Issue Type Task [ 3 ] Bug [ 1 ]
            serg Sergei Golubchik made changes -
            Affects Version/s 5.5 [ 15800 ]
            Affects Version/s 10.0 [ 16000 ]
            Affects Version/s 10.1 [ 16100 ]
            Affects Version/s 10.2 [ 14601 ]
            Affects Version/s 10.3 [ 22126 ]
            serg Sergei Golubchik made changes -
            Fix Version/s 5.5 [ 15800 ]
            Fix Version/s 10.0 [ 16000 ]
            Fix Version/s 10.1 [ 16100 ]
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.3 [ 22126 ]
            serg Sergei Golubchik made changes -
            Status Open [ 1 ] Confirmed [ 10101 ]
            serg Sergei Golubchik made changes -
            Fix Version/s 10.4 [ 22408 ]
            jb-boin Jean Weisbuch added a comment -

            The mysqld_safe function "set_malloc_lib()" is still broken in multiple ways in 10.5.

            The first issue is that if "malloc-lib" is set to "tcmalloc" or "jemalloc", it is calling the "get_mysql_config()" function which is also mostly broken.

            The "get_mysql_config()" function is trying to execute "mysql_config" on the same path as the "mysqld_safe" script which won't work on most systems as it's now included on a separate package ("libmariadb-dev-compat" on Debian) and should ideally call it by it's new name "mariadb_config".

            And even if the "get_mysql_config()" function is working, it will then fail as "set_malloc_lib()" is trying to call "mariadb_config" with a parameter that does not exist anymore ("--variable=pkglibdir") :

            # mariadb_config --variable=pkglibdir
            mariadb_config: unrecognized option '--variable=pkglibdir'
            

            So the whole "get_mysql_config()" function should probably be removed as it's only used here and not functional at all.
            –

            Another issue (at least on modern Debian/Ubuntu) is that without finding the "$pkglibdir", it can't find the library as it's not looking in the right path (/usr/lib/x86_64-linux-gnu) in the "for libdir in ..." loop, forcing the whole mysqld_safe script to do a exit 1.
            –

            Yet another issue with how the function work is that if there are multiple libraries matching the regular expression on :

            tmp=`echo "$libdir/lib$malloc_lib.so".[0-9]`
            

            It will return the multiple filenames separated by spaces on the $where variable making the test [ -n "$where" ] fail, forcing the whole mysqld_safe script to do a exit 1.

            jb-boin Jean Weisbuch added a comment - The mysqld_safe function " set_malloc_lib() " is still broken in multiple ways in 10.5. The first issue is that if " malloc-lib " is set to " tcmalloc " or " jemalloc ", it is calling the " get_mysql_config() " function which is also mostly broken. The " get_mysql_config() " function is trying to execute " mysql_config " on the same path as the " mysqld_safe " script which won't work on most systems as it's now included on a separate package (" libmariadb-dev-compat " on Debian) and should ideally call it by it's new name " mariadb_config ". And even if the " get_mysql_config() " function is working, it will then fail as " set_malloc_lib() " is trying to call " mariadb_config " with a parameter that does not exist anymore (" --variable=pkglibdir ") : # mariadb_config --variable=pkglibdir mariadb_config: unrecognized option '--variable=pkglibdir' So the whole " get_mysql_config() " function should probably be removed as it's only used here and not functional at all. – Another issue (at least on modern Debian/Ubuntu) is that without finding the " $pkglibdir ", it can't find the library as it's not looking in the right path ( /usr/lib/x86_64-linux-gnu ) in the " for libdir in ... " loop, forcing the whole mysqld_safe script to do a exit 1 . – Yet another issue with how the function work is that if there are multiple libraries matching the regular expression on : tmp=`echo "$libdir/lib$malloc_lib.so".[0-9]` It will return the multiple filenames separated by spaces on the $where variable making the test [ -n "$where" ] fail, forcing the whole mysqld_safe script to do a exit 1 .
            serg Sergei Golubchik made changes -
            Labels beginner-friendly
            serg Sergei Golubchik made changes -
            Assignee Anel Husakovic [ anel ]
            anel Anel Husakovic made changes -
            Description When using a custom memory allocator, e.g.

            [mysqld_safe]
            malloc-lib=jemalloc

            When starting MariaDB, the mysqld_safe script attempts to learn which lib directories to use by executing mysql_config command.

            However this is only provided by the MariaDB-devel package, and so isn't installed on most production servers.

            As such, we get an error each time we start MariaDB, in the form of:

            {code}
            180703 12:52:39 mysqld_safe Can not run mysql_config --variable=pkglibdir from '/usr/bin/mysql_config'
            {code}

            MariaDB still starts OK because the mysqld_safe script also looks in the normal lib directories, where jemalloc is installed to by CentOS.

            I do not think we need to see this error, as it will be common for mysql_config not to be installed. Perhaps only show the error if the allocator library isn't found in any of the normal library directories?

            When using a custom memory allocator, e.g.
            {code:noformat}
            [mysqld_safe]
            malloc-lib=jemalloc
            {code}
            When starting {{MariaDB}}, the {{mysqld_safe}} script attempts to learn which lib directories to use by executing {{mysql_config}} command.

            However this is only provided by the {{MariaDB-devel}} package, and so isn't installed on most production servers.

            As such, we get an error each time we start {{MariaDB}}, in the form of:

            {code}
            180703 12:52:39 mysqld_safe Can not run mysql_config --variable=pkglibdir from '/usr/bin/mysql_config'
            {code}

            {{MariaDB}} still starts OK because the {{mysqld_safe}} script also looks in the normal lib directories, where {{jemalloc}} is installed to by {{CentOS}}.

            I do not think we need to see this error, as it will be common for {{mysql_config}} not to be installed. Perhaps only show the error if the allocator library isn't found in any of the normal library directories?

            anel Anel Husakovic made changes -
            Status Confirmed [ 10101 ] In Progress [ 3 ]

            danblack please review patch

            anel Anel Husakovic added a comment - danblack please review patch
            anel Anel Husakovic made changes -
            Assignee Anel Husakovic [ anel ] Daniel Black [ danblack ]
            Status In Progress [ 3 ] In Review [ 10002 ]

            danblack, jb-boin here is a new patch 1b7e953ae5bd14 based on jb-boin observation of using ldconfig -, thanks jb-boin.

            anel Anel Husakovic added a comment - danblack , jb-boin here is a new patch 1b7e953ae5bd14 based on jb-boin observation of using ldconfig - , thanks jb-boin .
            anel Anel Husakovic made changes -
            Fix Version/s 5.5 [ 15800 ]
            Fix Version/s 10.1 [ 16100 ]
            anel Anel Husakovic made changes -
            Fix Version/s 10.0 [ 16000 ]
            anel Anel Husakovic added a comment - - edited

            New patch 09f46e87d25799f9b addressing the location of ldconfig and review from previous commit.

            anel Anel Husakovic added a comment - - edited New patch 09f46e87d25799f9b addressing the location of ldconfig and review from previous commit.

            Pushed to 10.2 with 95bb3cb886cb64be3ee5a

            anel Anel Husakovic added a comment - Pushed to 10.2 with 95bb3cb886cb64be3ee5a
            anel Anel Husakovic made changes -
            Fix Version/s 10.2.35 [ 25022 ]
            Fix Version/s 10.3.26 [ 25021 ]
            Fix Version/s 10.4.16 [ 25020 ]
            Fix Version/s 10.5.7 [ 25019 ]
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.3 [ 22126 ]
            Fix Version/s 10.4 [ 22408 ]
            Assignee Daniel Black [ danblack ] Anel Husakovic [ anel ]
            Resolution Fixed [ 1 ]
            Status In Review [ 10002 ] Closed [ 6 ]
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            anel Anel Husakovic made changes -
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 88205 ] MariaDB v4 [ 154620 ]

            People

              anel Anel Husakovic
              tomp Tom Parrott
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.