When FIPS is enabled (/proc/sys/crypto/fips_enabled is 1), OpenSSL disabled MD5, so my_md5() — that uses OpenSSL — doesn't work. It causes numerous failures and crashes as md5 is used internally for various purposes (e.g. checksums of views, MD5() SQL function, etc).
The fix is to tell OpenSSL that MariaDB needs MD5 even if FIPS disables it. This is fine as long as it's not used for cryptographic purposes.