[MDEV-18106] Upgrade from mariadb 10.2.19 to 10.2.20 hangs (debian) Created: 2018-12-29  Updated: 2019-01-27  Resolved: 2019-01-27

Status: Closed
Project: MariaDB Server
Component/s: Packaging
Affects Version/s: 10.2.20
Fix Version/s: N/A

Type: Bug Priority: Critical
Reporter: yoch Assignee: Unassigned
Resolution: Incomplete Votes: 0
Labels: need_feedback
Environment:

Debian 9.6


 Description   

While upgrading mariadb using `apt upgrade`, apt hangs on this line with 200% CPU usage:

Setting up mariadb-server-10.2 (10.2.20...

Investigating with ps show that install script is blocked on bootstrap step:

/usr/sbin/mysqld --lc-messages-dir=/usr/share/mysql/english/.. --bootstrap --silent-startup --basedir=/usr --datadir=/var/lib/mysql --log-warnings=0 --enforce-storage-engine= --plugin-dir=/usr/lib/mysql/plugin --disable-log-bin --user=mysql --max_allowed_packet=8M --net_buffer_length=16K

At this state, strace simply shows:

futex(0x56132668fe44, FUTEX_WAIT_PRIVATE, 1, NULL

After one hour, I need to stop the install and start mysqld manually.Now it works, but my dpkg state is corrupted. I tried to reinstall, but the script hang again.

 Comments   
Comment by Elena Stepanova [ 2018-12-29 ]

Did you run bootstrap manually as well, and it worked, or did you just start mysqld? (bootstrap isn't really needed on upgrade, so it can well be omitted, but it would be helpful for us to know if the problem is on the install scripts side or the server).

If it's reproducible, would you be able to get a stack trace from a hanging process? You might have to install dbgsym packages to make it meaningful.

Comment by yoch [ 2018-12-29 ]

I also tried to run bootstrap by myself, and it hangs as well.

Here some backtrace (I don't know if I installed dbgsym correctly, because apt isn't working):

pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
 
185     ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S: No such file or directory.
 
(gdb) backtrace
 
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
 
#1  0x00005586de77a5fb in ?? ()
 
#2  0x00005586de77d70a in mysqld_main(int, char**) ()
 
#3  0x00007fac42ad62e1 in __libc_start_main (main=0x5586de75d0c0 <main>, argc=13, argv=0x7ffff29b9838, init=<optimized out>, fini=<optimized out>,
 
    rtld_fini=<optimized out>, stack_end=0x7ffff29b9828) at ../csu/libc-start.c:291
 
#4  0x00005586de77040a in _start ()

Comment by Elena Stepanova [ 2018-12-30 ]

I don't know if you're aware, so just to make sure: when the server runs in bootstrap, it expects some input to be fed to it. If nothing is given, it keeps waiting, which does look like hanging. The installation script passes over (or at least is supposed to pass over) to it the contents of several files through a pipe, so you don't see i in the processlist, but when you run it manually, you need to do it yourself.

So, now I'm wondering if the feeding doesn't happen on your installation on some reason. I tried upgrade on one of our lab machines, it seems to work okay for me.

You can try something simple first, like

$ cat 1.sql
 
use mysql;

If it works, then we'll need to try with real SQL scripts. It can probably be done by running sudo mysql_install_db.

Generated at Thu Feb 08 08:41:32 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.