[MDEV-30919] Test unit.aes failed on signal 11 Created: 2023-03-23  Updated: 2023-10-07  Resolved: 2023-09-19

Status: Closed
Project: MariaDB Server
Component/s: Server, SSL
Affects Version/s: 10.11
Fix Version/s: 10.4.31, 10.5.22, 10.6.15, 10.9.8, 10.10.6, 10.11.5, 11.0.3, 11.1.2

Type: Bug Priority: Major
Reporter: Otto Kekäläinen Assignee: Daniel Black
Resolution: Fixed Votes: 0
Labels: None


 Description   

While testing 10.11.2 I noticed the s390x build post test run in https://launchpadlibrarian.net/656785756/buildlog_ubuntu-lunar-s390x.mariadb_1%3A10.11.2-2~ubuntu23.04.1~1679287945.48be213fe75.dev.otto_BUILDING.txt.gz failed on:

unit.aes                                 w3 [ fail ]
 
        Test ended at 2023-03-20 15:57:18
 
 
 
CURRENT_TEST: unit.aes
 
1..122
 
ok 1 - encrypt MY_AES_ECB 200 pad
 
ok 2 - my_aes_get_size
 
ok 3 - md5 len
 
ok 4 - md5
 
ok 5 - decrypt MY_AES_ECB 208
 
ok 6 - memcmp
 
ok 7 - encrypt MY_AES_ECB 128 pad
 
ok 8 - my_aes_get_size
 
ok 9 - md5 len
 
ok 10 - md5
 
ok 11 - decrypt MY_AES_ECB 144
 
ok 12 - memcmp
 
ok 13 - encrypt MY_AES_CBC 159 pad
 
ok 14 - my_aes_get_size
 
ok 15 - md5 len
 
ok 16 - md5
 
ok 17 - decrypt MY_AES_CBC 160
 
ok 18 - memcmp
 
ok 19 - encrypt MY_AES_CBC 192 pad
 
ok 20 - my_aes_get_size
 
ok 21 - md5 len
 
ok 22 - md5
 
ok 23 - decrypt MY_AES_CBC 208
 
ok 24 - memcmp
 
ok 25 - encrypt MY_AES_ECB 200 nopad
 
Bail out! Signal 11 thrown
 
# 122 tests planned,  0 failed,  25 was last executed
 
 
 
mysqltest failed with unexpected return code 255



 Comments   
Comment by Ehsan Kiani Far [ 2023-06-29 ]

Hi, I faced the same issue when I was building on ubuntu20.10-s390x. In my case the issue was with the "libcrypto.so.3" file and fixed the issue with installing the `gnutls-dev` package instead of `libssl-dev`. Just wanted to share this with you.

Comment by Daniel Black [ 2023-06-29 ]

ehsankianifar can you run gdb --args ./unittest/mysys/aes-t and share bt -frame-arguments all full on the openssl

Test 26 at least on x86_64 is "26 - encrypt bytewise MY_AES_ECB 200"

Appears to be openssl 3.0 related as it doesn't fail in RHEL7/8, or Ubuntu 20.04. Only later versions.

Comment by Ehsan Kiani Far [ 2023-06-30 ]

Sure,
This is the output on ubuntu 22.04:

(gdb) bt -frame-arguments all full
 
#0  0x000003fffdc86b8e in ?? () from /lib/s390x-linux-gnu/libcrypto.so.3
 
No symbol table info available.
 
#1  0x000003fffdc87414 in EVP_CipherInit_ex () from /lib/s390x-linux-gnu/libcrypto.so.3
 
No symbol table info available.
 
#2  0x000002aa0003d4c6 in MyCTX::init (ivlen=<optimized out>, iv=0x3ffffffe060 "\002\003\004\005\006\a\b\t", 
    klen=<optimized out>, key=0x3ffffffe050 "\001\002\003\004\005\006\a\b\t", encrypt=1, cipher=0x3fffde8b788, 
    this=0x3ffffffdca9) at /home/ubuntu/server/mysys_ssl/my_crypt.cc:63
 
No locals.
 
#3  MyCTX_nopad::init (this=0x3ffffffdca9, cipher=0x3fffde8b788, encrypt=<optimized out>, 
    key=0x3ffffffe050 "\001\002\003\004\005\006\a\b\t", klen=<optimized out>, 
    iv=0x3ffffffe060 "\002\003\004\005\006\a\b\t", ivlen=16) at /home/ubuntu/server/mysys_ssl/my_crypt.cc:117
 
        res = <optimized out>
 
#4  0x000002aa0003cc46 in my_aes_crypt_init (ctx=ctx@entry=0x3ffffffdca9, mode=<optimized out>, 
    flags=<optimized out>, key=0x3ffffffe050 "\001\002\003\004\005\006\a\b\t", 
    key@entry=0x3ffffffe060 "\002\003\004\005\006\a\b\t", klen=klen@entry=16, 
    iv=0x3ffffffe060 "\002\003\004\005\006\a\b\t", ivlen=16) at /home/ubuntu/server/mysys_ssl/my_crypt.cc:301
 
No locals.
 
#5  0x000002aa00030468 in aes_crypt_bytewise (mode=mode@entry=MY_AES_ECB, 
    flags=<error reading variable: value has been optimized out>, 
    src=src@entry=0x3ffffffe080 '.' <repeats 200 times>, slen=slen@entry=200, 
    dst=dst@entry=0x3ffffffec9c "\375\371O\"", dlen=0x3ffffffe048, 
    key=0x3ffffffe050 "\001\002\003\004\005\006\a\b\t", iv=0x3ffffffe060 "\002\003\004\005\006\a\b\t", ivlen=16, 
    klen=16) at /home/ubuntu/server/unittest/mysys/aes-t.c:39
 
        ctx = 0x3ffffffdca9
 
        res1 = <optimized out>
 
        res2 = <optimized out>
 
        d1 = 0
 
        d2 = 0
 
        i = <optimized out>
 
#6  0x000002aa00030670 in do_test (mode=mode@entry=0, mode_str=<optimized out>, 
    mode_str@entry=0x2aa0008438e "MY_AES_ECB", nopad=nopad@entry=2, slen=slen@entry=200, fill=fill@entry=46 '.', 
    dlen=200, hash=0x2aa000843ee "\277\354C\321f\215\001\255:%\356\246=\306\304h")
 
    at /home/ubuntu/server/unittest/mysys/aes-t.c:88
 
        key = "\001\002\003\004\005\006\a\b\t\000\001\002\003\004\005\006"
 
        iv = "\002\003\004\005\006\a\b\t\000\001\002\003\004\005\006\a"
 
        src = '.' <repeats 200 times>...
 
        dst = "\276\006c\"\226\066\f\252Zu`\206\372i\200\373\276\006c\"\226\066\f\252Zu`\206\372i\200\373\276\006c\"\226\066\f\252Zu`\206\372i\200\373\276\006c\"\226\066\f\252Zu`\206\372i\200\373\276\006c\"\226\066\f\252Zu`\206\372i\2--Type <RET> for more, q to quit, c to continue without paging--
 
00\373\276\006c\"\226\066\f\252Zu`\206\372i\200\373\276\006c\"\226\066\f\252Zu`\206\372i\200\373\276\006c\"\226\066\f\252Zu`\206\372i\200\373\276\006c\"\226\066\f\252Zu`\206\372i\200\373\276\006c\"\226\066\f\252Zu`\206\372i\200\373\276\006c\"\226\066\f\252Zu`\206\372i\200\373\276\006c\"\226\066\f\252Zu`\206\372i\200\373\022\325)\311\352\301X="...
 
        dst2 = "\375\371O\"\000\000\002\252\000K\256\240\000\000\003\377\375\377\067@\000\000\002\252\000K\256\360\000\000\002\252\000>\267\370\000\000\002\252\000K\256\270\000\000\002\252\000K\256\260\000\000\003\377\377\377\354x\000\000\003\377\375\373\017\220\000\000\003\377\375\373\020\070\000\000\002\252\000\003$\366\000\000\003\377\377\377\354xapxe3 dnyb-2k et\377\377\377\377\377\377\377\377\000\000\002\252\000\031\275\260\000U\307\377\254\022\303T\000\000\000\000\000\000\000\000", '\377' <repeats 16 times>, "\000\000\002\252\000\003\t\340\000\000\000\000\020\000\000\000st/mysys\000\000\000\000\000\000\000\020\000\000\003\377\375\370\t`\000\000\003\377"...
 
        ddst = '@' <repeats 192 times>, "........"...
 
        md5 = "T\304u\035\377\340\366\200\360\205\273\213\332\a!\027"
 
        src_len = 200
 
        dst_len = 200
 
        dst_len2 = 0
 
        ddst_len = 192
 
        result = <optimized out>
 
#7  0x000002aa00030008 in main (argc=<optimized out>, argv=<optimized out>)
 
    at /home/ubuntu/server/unittest/mysys/aes-t.c:138
 
No locals.

For reference, I installed `git gcc g++ make which wget tar gzip cmake libssl-dev ncurses-dev bison build-essential libboost-all-dev gawk libpam0g-dev patch libnghttp2-dev` then used `cmake` then `make` to build the package.
In the `aes` test all the pad (DO_TEST_P) tests pass but all the nopad (DO_TEST_N) tests fail with `segmentation fault`.
Seemingly my_aes_crypt_init triggers the fault.

Comment by Sergei Golubchik [ 2023-07-08 ]

Try the next release (planned to go out at the end of the month), supposedly commit 9c0e91a27cc should fix it.

Or, if you'd like, you can try packages from the CI, they're built for every push.

Comment by Daniel Black [ 2023-09-19 ]

tested on 9c0e91a27cc and functions without crash.

9c0e91a27cc has only merged to 11.2 so far, it still has bb failures in 11.3 currently.

Comment by Otto Kekäläinen [ 2023-10-07 ]

I confirm that this seems to be fixed in 10.11.5 as the failure is no longer visible on any tested architecture.

Comment by Daniel Black [ 2023-10-07 ]

Thanks otto for confirming.

Generated at Thu Feb 08 10:19:53 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.