Hello,
You made a mistake when you released the version 10.3.22 (and all others also I guess, but I didn't tested)
The file pam_user_map.so is not in the good directory 
the log :
Feb 03 17:48:29 pmaria36-par mysqld[18993]: PAM unable to dlopen(/usr/lib64/security/pam_user_map.so): /usr/lib64/security/pam_user_map.so: cannot open shared object file: No such file or directory
Feb 03 17:48:29 pmaria36-par mysqld[18993]: PAM adding faulty module: /usr/lib64/security/pam_user_map.so
Feb 03 17:48:32 pmaria36-par collectd[5489]: write_graphite plugin: Connection closed after 0.000 seconds.
Feb 03 17:48:34 pmaria36-par mysqld[18993]: pam_ldap(mariadb:auth): unexpected response from failed conversation function
Feb 03 17:48:34 pmaria36-par mysqld[18993]: pam_ldap(mariadb:auth): conversation failed
Feb 03 17:48:34 pmaria36-par mysqld[18993]: pam_ldap(mariadb:auth): failed to get password: Authentication token manipulation error
Feb 03 17:49:48 pmaria36-par consul[2830]: 2020/02/03 17:49:48 [INFO] memberlist: Marking qtfckq81301e8.kubes02.par.criteo.preprod as failed, suspect timeout reached (2 peer confirmations)
Feb 03 17:49:48 pmaria36-par consul[2830]: 2020/02/03 17:49:48 [INFO] serf: EventMemberFailed: qtfckq81301e8.kubes02.par.criteo.preprod 10.236.74.12
Feb 03 17:49:48 pmaria36-par consul[2830]: 2020/02/03 17:49:48 [INFO] serf: EventMemberJoin: qtfckq81301e8.kubes02.par.criteo.preprod 10.236.74.12
Feb 03 17:49:58 pmaria36-par mysqld[18993]: pam_ldap(mariadb:auth): unexpected response from failed conversation function
Feb 03 17:49:58 pmaria36-par mysqld[18993]: pam_ldap(mariadb:auth): conversation failed
Feb 03 17:49:58 pmaria36-par mysqld[18993]: pam_ldap(mariadb:auth): failed to get password: Authentication token manipulation error
Feb 03 17:49:58 pmaria36-par mysqld[18993]: pam_user_map(mariadb:auth): Opening file '/etc/security/user_map.conf'.
Feb 03 17:49:58 pmaria36-par mysqld[18993]: pam_user_map(mariadb:auth): Incoming username 'a.lequoy'.
For Centos :
The file should be placed in /usr/lib64/security/pam_user_map.so like all other about pam, and not in : /usr/lib/security/pam_user_map.so
with cp -a /usr/lib/security/pam_user_map.so /usr/lib64/security/
it's working perfectly :
Feb 03 18:04:11 pmaria36-par mysqld[18993]: pam_ldap(mariadb:auth): nslcd authentication; user=a.lequoy
Feb 03 18:04:11 pmaria36-par mysqld[18993]: pam_ldap(mariadb:auth): authentication succeeded
Feb 03 18:04:11 pmaria36-par mysqld[18993]: pam_user_map(mariadb:auth): Opening file '/etc/security/user_map.conf'.
Feb 03 18:04:11 pmaria36-par mysqld[18993]: pam_user_map(mariadb:auth): Incoming username 'a.lequoy'.
Feb 03 18:04:12 pmaria36-par mysqld[18993]: pam_user_map(mariadb:auth): User belongs to 59 groups [***].
Feb 03 18:04:12 pmaria36-par mysqld[18993]: pam_user_map(mariadb:auth): Check if user is in group 'gu-dba': YES
Feb 03 18:04:12 pmaria36-par mysqld[18993]: pam_user_map(mariadb:auth): User mapped as 'adminprod'
https://github.com/MariaDB/server/commit/1b04e854531fd5634defaffa30fed82e3329436d