Random Communication Link failures seen with MariaDB SSL enabled users on Rocky Linux 9.4

We are seeing random communication link failure issues during heavy traffic with SSL enabled users in Mariadb 10.5.18 installed on rocky linux 9.4.
OpenSSL version: 3.0.7 (default for rocky 9.4)

When disabling the ssl on those users, link failure issues are not seen. We recently migrated to rockylinux 9.4 from centos 7 which had open ssl 1.0 and there weren't any link failures issues in centos.

We even tried custom openssl 3.3.2 installation on rocky linux updated mariadb service file to point to that openssl version to see if it fixes the issue. Still we are seeing those link failures too often.

Is there any config on mariadb that we are missing for ssl or any way to resolve this issue?