[MDEV-7659] buildbot may leave stale mysqld Created: 2015-03-04  Updated: 2015-03-20  Resolved: 2015-03-20

Status: Closed
Project: MariaDB Server
Component/s: Tests
Affects Version/s: 5.5, 10.0, 10.1
Fix Version/s: 5.5.43, 10.0.18, 10.1.4

Type: Bug Priority: Major
Reporter: Sergey Vojtovich Assignee: Sergei Golubchik
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-7448 mtr may leave stale mysqld Closed

 Description   

There was an attempt to fix this in MDEV-7448, but it didn't work.

When bb slave looses connection to master it seem to send SIGKILL to process group:

2015-03-02 19:03:15-0500 [Broker,client] lost remote
 
2015-03-02 19:03:15-0500 [Broker,client] lost remote step
 
2015-03-02 19:03:15-0500 [Broker,client] stopCommand: halting current command <buildslave.commands.shell.SlaveShellCommand instance at 0x10004609998>
 
2015-03-02 19:03:15-0500 [Broker,client] command interrupted, attempting to kill
 
2015-03-02 19:03:15-0500 [Broker,client] trying to kill process group 17399
 
2015-03-02 19:03:15-0500 [Broker,client]  signal 9 sent successfully
 
2015-03-02 19:03:15-0500 [Broker,client] Lost connection to buildbot.askmonty.org:9989

mysqld is most probably detached from that process group. At least mysql-test/lib/My/SafeProcess/safe_process.cc calls setpgid(). As a result signal won't reach mysqld.

This means that intermediate processes must catch SIGKILL and forward it to mysqld. But SIGKILL cannot be caught.

I can think of 3 ways of fixing this problem:

  • let bb use different signal when killing process group (SIGINT, SIGHUP)
  • don't detach mysqld from bb process group
  • somehow let bb record and kill mysqld process

 Comments   
Comment by Sergey Vojtovich [ 2015-03-05 ]

Just for the record: there seem to be bb slave interruptSignal option. It should likely be possible to specify it on a per-step basis on master.

OTOH we could probably use SIGTERM: it should be fine for build step too.

There're no stale mysqld after "killall -g -s SIGTERM mtr", but if I use SIGKILL there is.

Comment by Sergey Vojtovich [ 2015-03-05 ]

Ok to push.

Generated at Thu Feb 08 07:21:18 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.