The library used by the ed25519 authentication plugin exhibits undefined behavior as reported by UBSAN.

This is edited example output from UBSAN that shows that negative values are used in left shift operations.

ref10/fe_sq.c:121:76: runtime error: left shift of negative value -46510040

    #0 0x7f17794c866d in ref10_fe_sq ref10/fe_sq.c:121

    #1 0x7f17794a4700 in ref10_ge_frombytes_negate_vartime ref10/ge_frombytes.c:21

    #2 0x7f17794a18c2 in crypto_sign_open ref10/open.c:24

Analyzing the code in the MariaDB server, it is seen that it also uses signed integers with both left and right shift operations. The latter is not as big of a problem (I assume) than the former as right shift of negative integers is only implementation defined whereas a left shift is undefined behavior.

I believe marko can confirm that he has also seen this when running with UBSAN.