Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-4890

Valgrind warnings on shutdown on a build with openSSL

Details

    • Bug
    • Status: Closed (View Workflow)
    • Minor
    • Resolution: Fixed
    • 10.0.4, 5.5.32, 5.1.67, 5.3.12
    • 10.0.5, 5.5.34, 5.1.73, 5.3.13
    • None
    • Ubuntu 12.04 x86_64

    Description

      I built the server with openSSL (WITH_SSL=system instead of bundled) and started getting valgrind warnings on server shutdown:

      perl ./mtr alias --valgrind-mysqld
      		 

      ==29019== 24 bytes in 1 blocks are still reachable in loss record 1 of 600
      		 
      ==29019==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
      		 
      ==29019==    by 0x5DCE7F3: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
      		 
      ==29019==    by 0x5E4928A: lh_insert (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
      		 
      ==29019==    by 0x5DD149B: OBJ_NAME_add (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
      		 
      ==29019==    by 0x5E53714: EVP_add_cipher (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
      		 
      ==29019==    by 0x5B54690: SSL_library_init (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
      		 
      ==29019==    by 0xEBE487: check_ssl_init (viosslfactories.c:155)
      		 
      ==29019==    by 0xEBE5AA: new_VioSSLFd (viosslfactories.c:189)
      		 
      ==29019==    by 0xEBEB66: new_VioSSLAcceptorFd (viosslfactories.c:336)
      		 
      ==29019==    by 0x5841CF: init_ssl() (mysqld.cc:4412)
      		 
      ==29019==    by 0x585888: mysqld_main(int, char**) (mysqld.cc:5224)
      		 
      ==29019==    by 0x57C4C3: main (main.cc:25)
      		 
      ==29019== 
      ==29019== 24 bytes in 1 blocks are still reachable in loss record 2 of 600
      		 
      ==29019==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
      		 
      ==29019==    by 0x5DCE7F3: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
      		 
      ==29019==    by 0x5E4928A: lh_insert (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
      		 
      ==29019==    by 0x5DD149B: OBJ_NAME_add (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
      		 
      ==29019==    by 0x5B54690: SSL_library_init (in /lib/x86_64-linux-gnu/libssl.so.1.0.0)
      		 
      ==29019==    by 0xEBE487: check_ssl_init (viosslfactories.c:155)
      		 
      ==29019==    by 0xEBE5AA: new_VioSSLFd (viosslfactories.c:189)
      		 
      ==29019==    by 0xEBEB66: new_VioSSLAcceptorFd (viosslfactories.c:336)
      		 
      ==29019==    by 0x5841CF: init_ssl() (mysqld.cc:4412)
      		 
      ==29019==    by 0x585888: mysqld_main(int, char**) (mysqld.cc:5224)
      		 
      ==29019==    by 0x57C4C3: main (main.cc:25)
      		 
      ==29019== 
      ...

      I'm not sure whether they're essential or not, and if they are, whether it's our fault, maybe it's a bad valgrind version or a 3rd-party library bug.

      ldd ../sql/mysqld
      		 
      	linux-vdso.so.1 =>  (0x00007fff7578d000)
      		 
      	libodbc.so.1 => /usr/lib/x86_64-linux-gnu/libodbc.so.1 (0x00007f1731474000)
      		 
      	libaio.so.1 => /lib/x86_64-linux-gnu/libaio.so.1 (0x00007f1731272000)
      		 
      	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f173105a000)
      		 
      	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f1730e52000)
      		 
      	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1730c35000)
      		 
      	libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f17309fb000)
      		 
      	libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f173079d000)
      		 
      	libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f17303d5000)
      		 
      	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f17301d0000)
      		 
      	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f172fed0000)
      		 
      	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f172fbd4000)
      		 
      	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f172f9bd000)
      		 
      	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f172f5fe000)
      		 
      	libltdl.so.7 => /usr/lib/x86_64-linux-gnu/libltdl.so.7 (0x00007f172f3f4000)
      		 
      	/lib64/ld-linux-x86-64.so.2 (0x00007f17316ed000)
      		 

      valgrind --version
      		 
      valgrind-3.7.0

      Attachments

        Activity

          Ascending order - Click to sort in descending order
          View 4 older comments
          pivanof Pavel Ivanov added a comment -

          The change in vio_end() made here http://bazaar.launchpad.net/~maria-captains/maria/10.0/revision/2502.565.54 causes mysqltest_embedded to hang on exit. So in 10.0.5 tarball testing of embedded server doesn't work at all when it's compiled with openssl.

          I think the problem is that mysql_server_end() in case of compilation with embedded server calls both end_embedded_server() (which calls clean_up() which calls vio_end() ) and vio_end(). But apparently ERR_remove_state(0) can be called only once. On the second call it hangs inside openssl trying to lock the mutex.

          pivanof Pavel Ivanov added a comment - The change in vio_end() made here http://bazaar.launchpad.net/~maria-captains/maria/10.0/revision/2502.565.54 causes mysqltest_embedded to hang on exit. So in 10.0.5 tarball testing of embedded server doesn't work at all when it's compiled with openssl. I think the problem is that mysql_server_end() in case of compilation with embedded server calls both end_embedded_server() (which calls clean_up() which calls vio_end() ) and vio_end(). But apparently ERR_remove_state(0) can be called only once. On the second call it hangs inside openssl trying to lock the mutex.
          elenst Elena Stepanova added a comment -

          Hi Alexander,

          Re-opening to draw your attention to the above. I think for the sake of changelog history and such, it will be better to open another bug report and close this one again, but I'll leave it to you to decide.

          elenst Elena Stepanova added a comment - Hi Alexander, Re-opening to draw your attention to the above. I think for the sake of changelog history and such, it will be better to open another bug report and close this one again, but I'll leave it to you to decide.
          bar Alexander Barkov added a comment - - edited

          Pavel, can you please post how-to-repeat instructions?

          I can't repeat the hang on my FC18 64bit box.
          I tried both debug and release builds:

          cmake . -DCMAKE_INSTALL_PREFIX=/opt/mariadb-10.0 -DWITH_SSL=system -DWITH_EMBEDDED_SERVER=yes

          and

          cmake . -DCMAKE_INSTALL_PREFIX=/opt/mariadb-10.0 -DWITH_SSL=system -DWITH_EMBEDDED_SERVER=yes -DCMAKE_BUILD_TYPE=Debug

          It does not hang.

          Note, I can confirm that there is a problem in the code with vio_end() executed twice.
          But it would be nice to repeat the hang anyway.
          Thanks.

          bar Alexander Barkov added a comment - - edited Pavel, can you please post how-to-repeat instructions? I can't repeat the hang on my FC18 64bit box. I tried both debug and release builds: cmake . -DCMAKE_INSTALL_PREFIX=/opt/mariadb-10.0 -DWITH_SSL=system -DWITH_EMBEDDED_SERVER=yes and cmake . -DCMAKE_INSTALL_PREFIX=/opt/mariadb-10.0 -DWITH_SSL=system -DWITH_EMBEDDED_SERVER=yes -DCMAKE_BUILD_TYPE=Debug It does not hang. Note, I can confirm that there is a problem in the code with vio_end() executed twice. But it would be nice to repeat the hang anyway. Thanks.
          pivanof Pavel Ivanov added a comment -

          Hm... Sorry, it turned out that I was linking with a locally built openssl library that included some hacks which were actually causing the hang. I can't reproduce with Ubuntu's standard openssl library either.

          pivanof Pavel Ivanov added a comment - Hm... Sorry, it turned out that I was linking with a locally built openssl library that included some hacks which were actually causing the hang. I can't reproduce with Ubuntu's standard openssl library either.
          bar Alexander Barkov added a comment -

          Thanks for checking! I opened a separate issue for this:
          https://mariadb.atlassian.net/browse/MDEV-5311
          The double call for vio_end() still needs to be fixed.

          bar Alexander Barkov added a comment - Thanks for checking! I opened a separate issue for this: https://mariadb.atlassian.net/browse/MDEV-5311 The double call for vio_end() still needs to be fixed.

          People

            bar Alexander Barkov
            elenst Elena Stepanova
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.