[MDEV-31379] Undefined behavior in the reference Ed25519 implementation Created: 2023-06-01  Updated: 2023-10-23

Status: Open
Project: MariaDB Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: markus makela Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Duplicate
is duplicated by MDEV-31801 Undefined behavior in ed25519 Open

 Description   

This is from the MaxScale sources but the same undefined behavior that UBSAN reports should be in MariaDB as well:

/home/timofey_turenko_mariadb_com/MaxScale/server/modules/authenticator/Ed25519/ref10/fe_sq.c:121:76: runtime error: left shift of negative value -46510040
 
    #0 0x7f0921c31b8b in mxs_ed25519_ref10_fe_sq /home/timofey_turenko_mariadb_com/MaxScale/server/modules/authenticator/Ed25519/ref10/fe_sq.c:121
 
    #1 0x7f0921c0dc63 in mxs_ed25519_ref10_ge_frombytes_negate_vartime /home/timofey_turenko_mariadb_com/MaxScale/server/modules/authenticator/Ed25519/ref10/ge_frombytes.c:21
 
    #2 0x7f0921c0addd in crypto_sign_open /home/timofey_turenko_mariadb_com/MaxScale/server/modules/authenticator/Ed25519/ref10/open.c:24

It seems this problem has also been found by others:
https://github.com/hyperledger/iroha-ed25519/commit/b61a1e77af5dc458ed6a5aee395d5b22775a4917
https://github.com/randombit/botan/issues/1372

 Comments   
Comment by markus makela [ 2023-06-01 ]

According to marko this wasn't seen in a recent run with UBSAN enabled.

Comment by Marko Mäkelä [ 2023-06-01 ]

I have observed this error in my local builds at least since 2020. I had commented about it in MDEV-29332.

For some reason, the https://buildbot.mariadb.net/buildbot/builders/kvm-ubsan/ builder does not report this. It compiles with GCC 9.3.0, while I would use something newer.

Generated at Thu Feb 08 10:23:25 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.