[MDEV-21385] PAM v2 plugin produces lots of zombie processes Created: 2019-12-23  Updated: 2022-04-20  Resolved: 2020-07-10

Status: Closed
Project: MariaDB Server
Component/s: Authentication and Privilege System, Plugin - pam
Affects Version/s: 10.4
Fix Version/s: 10.4.14, 10.5.5

Type: Bug Priority: Critical
Reporter: Florian Bezdeka Assignee: Alexey Botchkov
Resolution: Fixed Votes: 5
Labels: None
Environment:

OS: CentOS 8

Issue Links:
Blocks
Problem/Incident
is caused by MDEV-7032 new pam plugin with a suid wrapper Closed
is caused by MDEV-15473 Isolate/sandbox PAM modules, so that ... Closed

 Description   

The authentication using the PAM v2 module works, but produces a lot of zombie processes.

$ ps aux | grep auth_pam_tool
 
...
 
 
 
root      1637  0.0  0.0      0     0 ?        Z    Dez22   0:00 [auth_pam_tool] <defunct>
 
root      1638  0.0  0.0      0     0 ?        Z    Dez22   0:00 [auth_pam_tool] <defunct>
 
root      1639  0.0  0.0      0     0 ?        Z    Dez21   0:00 [auth_pam_tool] <defunct>
 
root      1640  0.0  0.0      0     0 ?        Z    Dez21   0:00 [auth_pam_tool] <defunct>
 
root      1641  0.0  0.0      0     0 ?        Z    Dez22   0:00 [auth_pam_tool] <defunct>
 
root      1642  0.0  0.0      0     0 ?        Z    08:21   0:00 [auth_pam_tool] <defunct>
 
...

Configuration details:

  • Using sssd with ldap backend for mariadb service
  • OS: CentOS 8

Logs (to make clear authentifaction itself works):

Dez 23 14:30:59 web1 mysqld[13717]: pam_sss(mariadb:auth): authentication success; logname= uid=27 euid=27 tty= ruser= rhost= user=web_<removed>
 
Dez 23 14:31:01 web1 mysqld[13717]: pam_sss(mariadb:auth): authentication success; logname= uid=27 euid=27 tty= ruser= rhost= user=web_<removed>
 
Dez 23 14:31:01 web1 mysqld[13717]: pam_sss(mariadb:auth): authentication success; logname= uid=27 euid=27 tty= ruser= rhost= user=web_<removed>

/etc/pam.d/mariadb

auth    required pam_sss.so domains=mariadb
 
account required pam_sss.so domains=mariadb



 Comments   
Comment by Elena Stepanova [ 2019-12-23 ]

fbezdeka,

Is there anything specific needed to be done to get the zombie processes?

Comment by Florian Bezdeka [ 2019-12-24 ]

I don't think so. I added the content of /etc/pam.d/mariadb to make clear how the pam to sssd routing is done. Let me know if something else is needed.

As workarround I switched back to the PAM v1 plugin. That temporary solved this problem.

Comment by Dazz Knowles [ 2020-01-21 ]

I'm seeing this too on Centos 7 with 10.4

Comment by Janne Nykopp [ 2020-04-06 ]

I have the same problem when running MariaDB official Docker image. Initially I thought it was due to the docker container not running an init that'd reap orphaned child processes so I installed dumb-init and ran that as the PID 1. That did not solve the zombie problem, but it did clarify that the zombie processes are owned by mysqld process.

Looking at https://raw.githubusercontent.com/MariaDB/server/10.4/plugin/auth_pam/auth_pam.c, there are a couple of things that to me look a bit strange.

First, on line 53, if fork() fails and returns -1 as proc_id, the waitpid() on line 191 would wait for any child process. Since other waits use exact pids, it could mess the logic up (but now in practice it won't, since waitpid() uses WNOHANG).

This leads to the second question; since WNOHANG is used, could it be that the parent process calls the waitpid() before child process has had the opportunity to call exit()? That'd result in a zombie.

Comment by Sergei Golubchik [ 2020-04-07 ]

Makes sense. Perhaps WNOHANG is the reason, indeed.

Comment by Alexey Botchkov [ 2020-07-10 ]

https://github.com/MariaDB/server/commit/99c03aa6abe77382253d1657057901de6f3e20da

Comment by Sergei Golubchik [ 2020-07-10 ]

https://github.com/MariaDB/server/commit/86063549042c7cd303de9e04c05ad199f7b55a4c is ok to push

thanks!

Comment by Kim Henriksen [ 2020-07-19 ]

I also have tons of auth_pam_tool zombies on Ubuntu 20.04 with 10.5.4+maria~focal

Generated at Thu Feb 08 09:06:44 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.