[MDEV-26302] Unittest failures with OpenSSL-3: aes & mf_iocache Created: 2021-08-04  Updated: 2022-01-20  Resolved: 2022-01-20

Status: Closed
Project: MariaDB Server
Component/s: Compiling, SSL
Affects Version/s: None
Fix Version/s: 10.2.42, 10.3.33, 10.4.23, 10.5.14, 10.6.6, 10.7.2, 10.8.1

Type: Bug Priority: Major
Reporter: Thomas Deutschmann Assignee: Sergei Golubchik
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
PartOf
is part of MDEV-25785 Add support for OpenSSL 3.0 Closed

 Description   

With OpenSSL 3.0.0-beta2, I am getting the following two unittest failures:

97% tests passed, 2 tests failed out of 66
 
Total Test time (real) =  25.31 sec
 
The following tests FAILED:
         21 - aes (Failed)
         27 - mf_iocache (Failed)
Errors while running CTest

Backtrace:

$ gdb -- mf_iocache-t
1..277
# temp io_cache with encryption
ok 1 - open_cached_file, pos_in_file = 0, pos_in_mem = 0
ok 2 - small write, pos_in_file = 0, pos_in_mem = 100
 
 
Program received signal SIGSEGV, Segmentation fault.
evp_cipher_init_internal (ctx=0x0, cipher=0x7ffff7f64f60 <aesni_128_gcm>, impl=0x0,
    key=0x5555558e4fc8 "\027\231\311\362\312V1\201:\216\200\060\240<W\231", iv=0x7fffffff9ec0 "", enc=1, params=0x0)
    at crypto/evp/evp_enc.c:100
100     crypto/evp/evp_enc.c: No such file or directory.
(gdb) bt
#0  evp_cipher_init_internal (ctx=0x0, cipher=0x7ffff7f64f60 <aesni_128_gcm>, impl=0x0,
    key=0x5555558e4fc8 "\027\231\311\362\312V1\201:\216\200\060\240<W\231", iv=0x7fffffff9ec0 "", enc=1, params=0x0)
    at crypto/evp/evp_enc.c:100
#1  0x00007ffff7d59e9b in EVP_CipherInit_ex (ctx=<optimized out>, cipher=<optimized out>, impl=impl@entry=0x0,
    key=<optimized out>, iv=iv@entry=0x7fffffff9ec0 "", enc=<optimized out>) at crypto/evp/evp_enc.c:398
#2  0x0000555555589e02 in MyCTX::init (ivlen=16, iv=0x7fffffff9ec0 "", klen=<optimized out>, key=<optimized out>,
    encrypt=<optimized out>, cipher=<optimized out>, this=0x7fffffff9ae0)
    at /var/tmp/portage/dev-db/mariadb-10.5.11/work/mysql/mysys_ssl/my_crypt.cc:67
#3  MyCTX_gcm::init (this=0x7fffffff9ae0, cipher=<optimized out>, encrypt=<optimized out>, key=<optimized out>,
    klen=<optimized out>, iv=0x7fffffff9ec0 "", ivlen=16)
    at /var/tmp/portage/dev-db/mariadb-10.5.11/work/mysql/mysys_ssl/my_crypt.cc:203
#4  0x000055555557998a in encryption_crypt (src=src@entry=0x5555558e0fa8 'Z' <repeats 200 times>...,
    slen=slen@entry=16384, dst=dst@entry=0x5555558e4fe0 "", dlen=dlen@entry=0x7fffffff9e6c,
    key=key@entry=0x5555558e4fc8 "\027\231\311\362\312V1\201:\216\200\060\240<W\231", iv=iv@entry=0x7fffffff9ec0 "",
    flags=1, key_id=2, key_version=1, ivlen=16, klen=16)
    at /var/tmp/portage/dev-db/mariadb-10.5.11/work/mysql/include/mysql/service_encryption.h:117
#5  0x000055555557a123 in my_b_encr_write (info=0x5555558c5fc0 <info>,
    Buffer=0x5555558e0fa8 'Z' <repeats 200 times>..., Count=16384)
    at /var/tmp/portage/dev-db/mariadb-10.5.11/work/mysql/sql/mf_iocache_encr.cc:191
#6  0x000055555557c992 in my_b_flush_io_cache (info=info@entry=0x5555558c5fc0 <info>, need_append_buffer_lock=0,
    need_append_buffer_lock@entry=1) at /var/tmp/portage/dev-db/mariadb-10.5.11/work/mysql/mysys/mf_iocache.c:1728
#7  0x000055555557d072 in _my_b_write (info=info@entry=0x5555558c5fc0 <info>,
    Buffer=0x7fffffffdf9c 'Z' <repeats 200 times>..., Count=300)
    at /var/tmp/portage/dev-db/mariadb-10.5.11/work/mysql/mysys/mf_iocache.c:559
#8  0x0000555555577fec in my_b_write (info=0x5555558c5fc0 <info>, Count=16584,
    Buffer=0x7fffffffa000 'Z' <repeats 200 times>...)
    at /var/tmp/portage/dev-db/mariadb-10.5.11/work/mysql/include/my_sys.h:521
#9  temp_io_cache () at /var/tmp/portage/dev-db/mariadb-10.5.11/work/mysql/unittest/sql/mf_iocache-t.cc:132
#10 0x0000555555577884 in main (argc=<optimized out>, argv=<optimized out>)
    at /var/tmp/portage/dev-db/mariadb-10.5.11/work/mysql/unittest/sql/mf_iocache-t.cc:455

and

(gdb) bt
#0  evp_cipher_init_internal (ctx=0x0, cipher=0x7ffff7f67760 <aesni_128_ecb>, impl=0x0,
    key=0x7fffffffd470 "\001\002\003\004\005\006\a\b\t", iv=0x7fffffffd480 "\002\003\004\005\006\a\b\t", enc=1,
    params=0x0) at crypto/evp/evp_enc.c:100
#1  0x00007ffff7d59e9b in EVP_CipherInit_ex (ctx=<optimized out>, cipher=<optimized out>, impl=impl@entry=0x0,
    key=<optimized out>, iv=<optimized out>, enc=<optimized out>) at crypto/evp/evp_enc.c:398
#2  0x00005555555831bd in MyCTX::init (this=<optimized out>, cipher=<optimized out>, encrypt=<optimized out>,
    key=<optimized out>, klen=<optimized out>, iv=<optimized out>, ivlen=16)
    at /var/tmp/portage/dev-db/mariadb-10.5.11/work/mysql/mysys_ssl/my_crypt.cc:67
#3  0x00005555555830bf in my_aes_crypt (mode=mode@entry=MY_AES_ECB, flags=flags@entry=1,
    src=src@entry=0x7fffffffd4a0 '.' <repeats 200 times>..., slen=slen@entry=200,
    dst=dst@entry=0x7fffffffdc80 "\020\335\377\377\377\177", dlen=dlen@entry=0x7fffffffd468,
    key=0x7fffffffd470 "\001\002\003\004\005\006\a\b\t", klen=16, iv=0x7fffffffd480 "\002\003\004\005\006\a\b\t",
    ivlen=16) at /var/tmp/portage/dev-db/mariadb-10.5.11/work/mysql/mysys_ssl/my_crypt.cc:313
#4  0x00005555555777e6 in main (argc=<optimized out>, argv=<optimized out>)
    at /var/tmp/portage/dev-db/mariadb-10.5.11/work/mysql/unittest/mysys/aes-t.c:75



 Comments   
Comment by Daniel Black [ 2021-08-04 ]

I had a look. I'm trying to work out how ctx got a null pointer at the top of the stack on evp_cipher_init_internal when the ctx is allocated in MyCTX

Generated at Thu Feb 08 09:44:17 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.