fix: multiple OpenSSL 3.x memory leaks#75
Conversation
… construction
Three distinct memory leak fixes for OpenSSL >= 3.0.0 code paths:
1. _get_key_parameters(): EVP_PKEY_get_bn_param() allocates new BIGNUMs
(unlike pre-3.x getters which return internal pointers). cor_bn2sv()
duplicates them via BN_dup() but the originals were never freed,
leaking 8 BIGNUMs on every call.
2. verify(): XSRETURN_NO/XSRETURN_YES returned immediately, bypassing
EVP_MD_free(md) and EVP_PKEY_CTX_free(ctx) cleanup. Restructured to
capture verify result, free resources, then switch on the result.
3. _new_key_from_parameters():
- EVP_PKEY_CTX (pctx) was never freed on any path (success or error)
- EVP_PKEY_CTX from EVP_PKEY_check() (test_ctx) was never freed
- OSSL_PARAM_BLD and OSSL_PARAM were not freed in the else branch
(public-key-only path)
Moved pctx/params_build/params declarations to PREINIT for proper
scope, added cleanup on both success and error paths.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
looks like there is a newer version of this one in another PR |
|
@timlegge — correct. PR #75 has been superseded by individual, more focused PRs:
However, one fix from this PR is not yet covered elsewhere: the This PR can be closed once that remaining leak is addressed in its own PR. |
| EVP_PKEY_CTX *pctx = EVP_PKEY_CTX_new_from_name(NULL, "RSA", NULL); | ||
| pctx = EVP_PKEY_CTX_new_from_name(NULL, "RSA", NULL); | ||
| CHECK_OPEN_SSL(pctx != NULL); | ||
| CHECK_OPEN_SSL(EVP_PKEY_fromdata_init(pctx) > 0); |
There was a problem hiding this comment.
Should this be THROW(EVP_PKEY_fromdata_init(pctx) > 0);
| BN_free(dmp1); | ||
| BN_free(dmq1); | ||
| BN_free(iqmp); | ||
| #endif |
timlegge
left a comment
There was a problem hiding this comment.
LGTM - The code duplication with the switch should be sorted later though
Summary
Fixes three distinct memory leak families in OpenSSL >= 3.0.0 code paths:
_get_key_parameters():EVP_PKEY_get_bn_param()allocates new BIGNUMs (unlike pre-3.x getters which return internal const pointers), butcor_bn2sv()duplicates them viaBN_dup()without freeing the originals — 8 BIGNUMs leaked per callverify():XSRETURN_NO/XSRETURN_YESreturned immediately, bypassingEVP_MD_free()andEVP_PKEY_CTX_free()cleanup — leaked on every verify call_new_key_from_parameters():EVP_PKEY_CTX(pctx) was never freed on any path,test_ctxfromEVP_PKEY_check()was never freed, andOSSL_PARAM_BLD/OSSL_PARAMwere not freed in the public-key-only branchRoot cause
The OpenSSL 3.x migration changed ownership semantics: pre-3.x getter functions return internal pointers (caller must not free), while 3.x functions like
EVP_PKEY_get_bn_param()allocate new objects (caller must free). The original migration missed several of these ownership changes.Test plan
_get_key_parameters())🤖 Generated with Claude Code