Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Cannot Reproduce
-
10.1(EOL)
-
None
Description
When test test-init.c from CONC-336 is linked with `mysql_config --cflags --libmysqld-libs` then second run cause deadlock in inline_mysql_cond_destroy.
$ gcc -W -Wall -O2 -g test-init.c -o test-init `mysql_config --cflags --libmysqld-libs`
|
$ gdb --args ./test-init 2 127.0.0.1 3306 pali pali
|
...
|
(gdb) r
|
...
|
^C
|
Program received signal SIGINT, Interrupt.
|
__lll_lock_wait_private () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:95
|
95 ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S: Adresár alebo súbor neexistuje.
|
(gdb) bt
|
#0 __lll_lock_wait_private () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:95
|
#1 0x00007ffff6184eb5 in __pthread_cond_destroy (cond=cond@entry=0x7ffff7568ae0 <COND_thread_count>) at pthread_cond_destroy.c:34
|
#2 0x00007ffff67436c0 in inline_mysql_cond_destroy (that=0x7ffff7568ae0 <COND_thread_count>) at ./include/mysql/psi/mysql_thread.h:1131
|
#3 clean_up_mutexes () at ./libmysqld/../sql/mysqld.cc:2264
|
#4 0x00007ffff6748974 in end_embedded_server () at ./libmysqld/lib_sql.cc:648
|
#5 0x00007ffff674c7ec in mysql_server_end () at ./libmysql/libmysql.c:213
|
#6 0x00005555555549e4 in main (argc=<optimized out>, argv=<optimized out>) at test-init.c:49
|
(gdb)
|
This is from MariaDB version 10.1.26, which is available in current Debian stable.
So problem is in function mysql_server_end which cause that deadlock.
Originally reported in comment: https://jira.mariadb.org/browse/CONC-336?focusedCommentId=111921&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-111921
Attachments
Issue Links
- relates to
-
-
- Closed
-
Activity
Now I take MariaDB version 10.1.26 from the sources: https://archive.mariadb.org/mariadb-10.1.26/source/mariadb-10.1.26.tar.gz
Then I used same steps for compiling as in the previous post but into cmake I added another parameter -DWITH_SSL=bundled because such old MariaDB version failed to compile with new OpenSSL.
And testing application passed without any crash and without any deadlock.
So I have feeling that this issue is related to some setup or something other. Becuase for sure the version from the description (10.1.26) caused this issue and I put into description backtrace where it hanged.
So can be end_embedded_server() or its init function affected by some SSL library? Or some system / additional plugin? Or something other? If you have some idea or what can I test then please let me know. I think I do not have original machine with original system on which it was crashing.
I see, thanks. Yes, it could be an SSL library, could be a specific plugin — there could be many things that don't like being initialized and deinitialized repeatedly. I mean, it's not a given that some piece of code can deinitialize itself back to its initial state.
Still, it's likely some external library. I suspect your recent test most probably had exactly the same set of plugins of exactly same versions as your earlier tests, that exhibited the crash.
Now I tried another thing: I took Debian version of MariaDB 10.1.26 (which has patches) from archive:
https://snapshot.debian.org/package/mariadb-10.1/10.1.26-1/
- https://snapshot.debian.org/archive/debian/20170824T160430Z/pool/main/m/mariadb-10.1/mariadb-10.1_10.1.26-1.debian.tar.xz
- https://snapshot.debian.org/archive/debian/20170824T160430Z/pool/main/m/mariadb-10.1/mariadb-10.1_10.1.26-1.dsc
- https://snapshot.debian.org/archive/debian/20170824T160430Z/pool/main/m/mariadb-10.1/mariadb-10.1_10.1.26.orig.tar.gz
Unpacked it via:
$ dpkg-source -x mariadb-10.1_10.1.26-1.dsc
|
And then applied same build steps as before. But results are same. It is working.
I do not know what else I can try...
pali, I'll close it as Can't repeat then. If you'll stumble upon it again, please, just add a comment here and I'll reopen
Hello, sorry for a longer delay. Now I have tested it with new 10.11.6 version (last 10 released at this time of testing) and the result is that this version is not affected by this bug. I'm going to test also other versions to figure out which are broken and which works. Because something must have been fixed in this 10.x version branch as I'm really sure that it was crashing.
Here are steps which I used for future reproducing:
$ wget https://archive.mariadb.org/mariadb-10.11.6/source/mariadb-10.11.6.tar.gz$ tar -xf mariadb-10.11.6.tar.gz$ cd mariadb-10.11.6/$ mkdir build$ cd build/$ cmake -DWITH_EMBEDDED_SERVER=ON -DWITH_UNIT_TESTS=OFF ..$ cd libmysqld/$ make -j8$ wget https://jira.mariadb.org/secure/attachment/45689/test-init.c$ gcc -W -Wall -O2 -g test-init.c -o test-init -I../../libmariadb/include/ -I../libmariadb/include/ ./libmysqld.so -Wl,-rpath=.$ ./test-init 2 127.0.0.1 3306 test ''(MariaDB server is running on localhost at port 3306 and user named test can be used without password)
Please check the steps if are correct and if I did not some mistake there.
To ensure that correct library (libmysql*d*.so is symlink to libmariadb*d*.so) was used here is output from ldd:
$ ldd ./test-initlinux-vdso.so.1 (0x00007ffc949ef000)libmariadbd.so.19 => ./libmariadbd.so.19 (0x00007fc8630cc000)libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc862ec7000)libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007fc862e8d000)libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fc862e6f000)libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007fc862ddc000)libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007fc862af0000)libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fc862ae9000)libaio.so.1 => /usr/lib/x86_64-linux-gnu/libaio.so.1 (0x00007fc862ae4000)libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fc862960000)libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc8627dd000)libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fc8627c3000)libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc8627a2000)/lib64/ld-linux-x86-64.so.2 (0x00007fc8650b7000)And cmake showed following features details (in case some feature may affect this problem):
The following features have been enabled:* WSREP, Server plugin STATIC* ARCHIVE, Storage Engine MODULE* BLACKHOLE, Storage Engine MODULE* CONNECT_VCT, Support for VCT in the CONNECT storage engine* CONNECT_LIBXML2, Support for LIBXML2 in the CONNECT storage engine* CONNECT_ODBC, Support for ODBC in the CONNECT storage engine* CONNECT_ZIP, Support for ZIP in the CONNECT storage engine* CONNECT_REST, Support for REST API in the CONNECT storage engine* CONNECT_XMAP, Support for index file mapping in the CONNECT storage engine* CONNECT, Storage Engine MODULE* CSV, Storage Engine STATIC* EXAMPLE, Storage Engine MODULE* FEDERATED, Storage Engine MODULE* FEDERATEDX, Storage Engine MODULE* HEAP, Storage Engine STATIC* INNODB_AHI, InnoDB Adaptive Hash Index* INNODB_ROOT_GUESS, Cache index root block descriptors in InnoDB* INNOBASE, Storage Engine STATIC* MARIABACKUP, MariaDB Backup Utility* ARIA, Storage Engine STATIC* S3, Storage Engine MODULE* MROONGA, Storage Engine MODULE* MYISAM, Storage Engine STATIC* MYISAMMRG, Storage Engine STATIC* PERFSCHEMA, Storage Engine STATIC* ROCKSDB, Storage Engine MODULE* ROCKSDB_LZ4, LZ4 Compression in the RocksDB storage engine* ROCKSDB_BZip2, BZip2 Compression in the RocksDB storage engine* ROCKSDB_Snappy, Snappy Compression in the RocksDB storage engine* ROCKSDB_ZSTD, ZSTD Compression in the RocksDB storage engine* ROCKSDB_ZLIB, zlib Compression in the RocksDB storage engine* SEQUENCE, Storage Engine STATIC* SPHINX, Storage Engine MODULE* SPIDER, Storage Engine MODULE* TEST_SQL_DISCOVERY, Storage Engine MODULE* AUDIT_NULL, Server plugin MODULE* AUTH_ED25519, Server plugin MODULE* DIALOG_EXAMPLES, Server plugin MODULE* AUTH_TEST_PLUGIN, Server plugin MODULE* QA_AUTH_INTERFACE, Server plugin MODULE* QA_AUTH_SERVER, Server plugin MODULE* QA_AUTH_CLIENT, Server plugin MODULE* AUTH_0X0100, Server plugin MODULE* AUTH_SOCKET, Server plugin STATIC* DAEMON_EXAMPLE, Server plugin MODULE* DEBUG_KEY_MANAGEMENT, Server plugin MODULE* DISKS, Server plugin MODULE* EXAMPLE_KEY_MANAGEMENT, Server plugin MODULE* FEEDBACK, Server plugin STATIC* FILE_KEY_MANAGEMENT, Server plugin MODULE* FTEXAMPLE, Server plugin MODULE* FUNC_TEST, Server plugin MODULE* HANDLERSOCKET, Server plugin MODULE* HASHICORP_KEY_MANAGEMENT, Hashicorp Key Management Plugin* LOCALES, Server plugin MODULE* METADATA_LOCK_INFO, Server plugin MODULE* PASSWORD_REUSE_CHECK, Server plugin MODULE* PROVIDER_BZIP2, Server plugin MODULE* PROVIDER_LZMA, Server plugin MODULE* PROVIDER_LZO, Server plugin MODULE* QUERY_CACHE_INFO, Server plugin MODULE* QUERY_RESPONSE_TIME, Server plugin MODULE* SERVER_AUDIT, Server plugin MODULE* SIMPLE_PASSWORD_CHECK, Server plugin MODULE* SQL_ERRLOG, Server plugin MODULE* TEST_SQL_SERVICE, Server plugin MODULE* TYPE_GEOM, Server plugin STATIC* TYPE_INET, Server plugin STATIC* TYPE_MYSQL_JSON, Server plugin MODULE* TYPE_MYSQL_TIMESTAMP, Server plugin MODULE* TYPE_TEST, Server plugin MODULE* TYPE_UUID, Server plugin STATIC* USER_VARIABLES, Server plugin STATIC* USERSTAT, Server plugin STATIC* TEST_VERSIONING, Server plugin MODULE* WSREP_INFO, Server plugin MODULE* THREAD_POOL_INFO, Server plugin STATIC* PARTITION, Storage Engine STATIC* SQL_SEQUENCE, Storage Engine STATIC* EMBEDDED_SERVER, Embedded MariaDB Server Library-- The following OPTIONAL packages have been found:* Git* LibXml2* CURL* Boost (required version >= 1.40.0)Required for the OQGraph storage engine* BZip2* LibLZMA* LZO* BISON (required version >= 2.4)-- The following RECOMMENDED packages have been found:* OpenSSL-- The following REQUIRED packages have been found:* Curses* Threads-- The following features have been disabled:* LIBWRAP, Support for tcp wrappers* SYSTEMD, Systemd scripts and notification support* CONNECT_JDBC, Support for JDBC in the CONNECT storage engine* CONNECT_MONGODB, Support for MongoDB in the CONNECT storage engine* NUMA, NUMA memory allocation policy* INNODB_EXTRA_DEBUG, Extra InnoDB debug checks* AWS_KEY_MANAGEMENT, AWS Encryption Key Management Plugin-- The following OPTIONAL packages have not been found:* Java (required version >= 1.6)Required for the CONNECT_JDBC feature* JNIRequired for the CONNECT_JDBC feature* PMEM* JudyRequired for the OQGraph storage engine* GSSAPI* LZ4 (required version >= 1.6)* Snappy