Currently the build flag WITH_SSL can be used to either use the system OpenSSL library, a custom OpenSSL from an alternative path or the embedded WolfSSL. There is currently no option to use the system WolfSSL.
In e.g. Debian WolfSSL has been packaged and available in the distro since 2015.
Please implement option WITH_SSL=system-wolfssl
I already did a stub at https://salsa.debian.org/mariadb-team/mariadb-10.4/-/commit/d540460bd807b47dba031744e630a450d4c39951 but I don't have enough skills (or time) to figure out all details and how to finish it, so I wish somebody else would do this.
NOTE! If it turns out that WolfSSL in Debian somehow is not suitable for MariaDB builds, please file bugs for the package in Debian. The maintainer seems very responsive and happy to modify the package so that it's consumers are happy.
PS. Nowadays the option should be WITH_TLS but that is another issue.