[MDEV-13698] stack overflow (OpenSSL on Windows) Created: 2017-09-01  Updated: 2017-09-18  Resolved: 2017-09-18

Status: Closed
Project: MariaDB Server
Component/s: Platform Windows, SSL
Affects Version/s: 10.2
Fix Version/s: 10.2.9

Type: Bug Priority: Major
Reporter: Georg Richter Assignee: Sergei Golubchik
Resolution: Fixed Votes: 0
Labels: None
Environment:

Windows



 Description   

The CRYPTO_set_mem_functions call in function check_openssl_compatibility causes a stack overflow when freeing the memory:

mysys_ssl/openssl.c

if (!CRYPTO_set_mem_functions(coc_malloc, CRYPTO_realloc, CRYPTO_free))
    return 1;

OpenSSL evp_enc.c

void CRYPTO_free(void *str, const char *file, int line)
{
    if (free_impl != NULL && free_impl != &CRYPTO_free) {
        free_impl(str, file, line);
        return;
    }

function pointers

  &CRYPTO_free	0x5da87f00 {libcrypto-1_1.dll!CRYPTO_free
  free_impl	0x0138b1de {mysqld.exe!_CRYPTO_free}	

Suggested fix:

if (!CRYPTO_set_mem_functions(coc_malloc, NULL, NULL))
    return 1;


Generated at Thu Feb 08 08:07:40 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.