[MDEV-7177] Server hangs on shutdown after InnoDB error (main.plugin_loaderr fails in buildbot) Created: 2014-11-23  Updated: 2015-02-05  Resolved: 2015-02-05

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB, Tests
Affects Version/s: 10.1
Fix Version/s: 10.1.3

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Sergey Vojtovich
Resolution: Fixed Votes: 0
Labels: buildbot, tests
Environment:

CentOS 5 32-bit


Issue Links:
Blocks
blocks MDEV-7172 Fix buildbot failures in 10.1 tree Closed

 Description   

http://buildbot.askmonty.org/buildbot/builders/kvm-bintar-centos5-x86/builds/1892/steps/test/logs/stdio

main.plugin_loaderr                      w2 [ fail ]  timeout after 900 seconds
        Test ended at 2014-11-23 09:21:08
 
Test case timeout after 900 seconds
 
...

The problem is reproducible on CentOS 5.9 32-bit, but I am not getting it on Wheezy 32 bit.

What happens there is that the test shuts down the server, then attempts to start it with a wrong set of options, the attempt is expected to fail and abort, and then the server is restarted properly to finish the test.
On CentOS 5 32-bit the "bad" attempt indeed fails, too, but the server hangs upon shutdown.
It can be well enough reproducible by just running the command line

/home/elenst/mariadb-10.1.2-linux-i686/bin/mysqld --basedir=/home/elenst/mariadb-10.1.2-linux-i686 --innodb_read_io_threads=2 --innodb=force --innodb-page-size=6000

The hang does not happen every time, but was happening often enough for me when I was trying it (~9 times out of 10).

Stack trace from the hanging server:

Thread 2 (Thread 0xb73ffb90 (LWP 4091)):
#0  0x00f3d410 in __kernel_vsyscall ()
#1  0x004d1ff2 in pthread_cond_timedwait@@GLIBC_2.3.2 ()
   from /lib/libpthread.so.0
#2  0x0886e8aa in inline_mysql_cond_timedwait (arg=0x0)
    at /home/buildbot/buildbot/build/include/mysql/psi/mysql_thread.h:1202
#3  timer_handler (arg=0x0)
    at /home/buildbot/buildbot/build/mysys/thr_timer.c:309
#4  0x004cd912 in start_thread () from /lib/libpthread.so.0
#5  0x0031f7ce in clone () from /lib/libc.so.6
 
Thread 1 (Thread 0xb7f706f0 (LWP 4090)):
#0  0x00f3d410 in __kernel_vsyscall ()
#1  0x004d1ff2 in pthread_cond_timedwait@@GLIBC_2.3.2 ()
   from /lib/libpthread.so.0
#2  0x0886e5b9 in inline_mysql_cond_timedwait ()
    at /home/buildbot/buildbot/build/include/mysql/psi/mysql_thread.h:1202
#3  end_thr_timer () at /home/buildbot/buildbot/build/mysys/thr_timer.c:123
#4  0x08135146 in clean_up (print_message=true)
    at /home/buildbot/buildbot/build/sql/mysqld.cc:2107
#5  0x08135980 in unireg_abort (exit_code=1)
    at /home/buildbot/buildbot/build/sql/mysqld.cc:2023
#6  0x0813b61f in init_server_components ()
    at /home/buildbot/buildbot/build/sql/mysqld.cc:5026
#7  0x0813c69a in mysqld_main (argc=6, argv=0xb741a280)
    at /home/buildbot/buildbot/build/sql/mysqld.cc:5607
#8  0x081306a2 in main (argc=) at /home/buildbot/buildbot/build/sql/main.cc:25

Stdout

[elenst@localhost mariadb-10.1.2-linux-i686]$ /home/elenst/mariadb-10.1.2-linux-i686/bin/mysqld  --basedir=/home/elenst/mariadb-10.1.2-linux-i686  --innodb_read_io_threads=2 --innodb=force --innodb-page-size=6000
InnoDB: Warning: innodb_page_size has been changed from default value 16384 to 6000d. (###EXPERIMENTAL### operation)
141126 20:13:43 [ERROR] InnoDB: Invalid page size=6000.
 
141126 20:13:43 [ERROR] Plugin 'InnoDB' init function returned error.
141126 20:13:43 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
141126 20:13:43 [ERROR] Failed to initialize plugins.
141126 20:13:43 [ERROR] Aborting



 Comments   
Comment by Jan Lindström (Inactive) [ 2014-11-27 ]

Both threads are not inside InnoDB/XtraDB, someone with more knowledge on that part of server should have a look of this. Based on stack, InnoDB/XtraDB has already shut down.

Comment by Elena Stepanova [ 2014-11-27 ]

svoj,
Could you please take a look? If not, please reassign it back to me.

Comment by Sergei Golubchik [ 2015-01-10 ]

Doesn't seem to happen anymore.

Comment by Sergey Vojtovich [ 2015-01-14 ]

Happened again tonight:
http://buildbot.askmonty.org/buildbot/builders/kvm-bintar-centos5-amd64/builds/1001/steps/test/logs/stdio

Comment by Sergey Vojtovich [ 2015-02-02 ]

serg, please review fix for this bug.

Generated at Thu Feb 08 07:17:36 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.