symbol conflict with libsodium causes crashes

This problem was reported to PowerDNS as https://github.com/PowerDNS/pdns/issues/10340 - please see there if anything below is unclear.

I picked version 3.1.12 in the 'Affects Version/s' dropdown because I do not know what CONC versions corresponds to libmariadb3 version '1:10.3.27-0+deb10u1' as shipped by Debian Buster; please correct that, if you would be so kind!

A (PowerDNS) user has created a MariaDB user with ed25519-based authentication (create user pdnstest@'%' identified via ed25519 using password('pdnstest');). When PowerDNS tries to connect to MariaDB with these credentials, PowerDNS crashes due to a bad pointer dereference.

PowerDNS uses libsodium. MariaDB's client_ed25519.so contains code that is related to libsodium, but, critically, the crypto_sign function has a different prototype. The cause of the crash is that the client_ed25519 plugin manages to call libsodium's crypto_sign instead of its own version.

Quoted from the PowerDNS ticket:

1. https://github.com/mariadb-corporation/mariadb-connector-c/blob/3.1/plugins/auth/ed25519.c#L110-L111 calls crypto_sign(signature, packet, NONCE_BYTES, (unsigned char*)mysql->passwd, strlen(mysql->passwd));
2. it wants to call https://github.com/mariadb-corporation/mariadb-connector-c/blob/3.1/plugins/auth/ref10/crypto_sign.h#L5-L9 (int crypto_sign( unsigned char *sm, const unsigned char *m, unsigned long long len, const unsigned char *pw, unsigned long long pwlen);
3. but because pdns links libsodium, it calls int crypto_sign(unsigned char *sm, unsigned long long *smlen_p, const unsigned char *m, unsigned long long mlen, const unsigned char *sk) _attribute_ ((nonnull(1, 3, 5)));

To simplify things for you (so you do not need to build PowerDNS), I've put a small reproducer at https://github.com/PowerDNS/libmariadb-libsodium-ed25519-crash . On Debian Buster, make should suffice. Then, given the user created earlier, call ./problem 127.0.0.1 pdnstest pdnstest pdnstest. A segmentation fault is the result.

LD_PRELOADing client_ed25519.so avoids the crash, but we only get away with that (I think) because PowerDNS does not use crypto_sign.

I'm not well versed in linkers, symbol resolution, etc., so I don't know if the problem is as simple as some build flag that is wrong, or that the fix would necessarily involve renaming the functions so they don't clash. I'm surprised, intuitively, that client_ed25519 would end up calling the symbol with the same name in a different lib instead of its own.

(I saw https://jira.mariadb.org/browse/MDEV-19217 which is somewhat related, but clearly a different problem.)