Skip to content

fix: BIO resource leak in extractBioString() error paths#145

Merged
timlegge merged 3 commits into
cpan-authors:mainfrom
toddr-bot:koan.toddr.bot/fix-extractbiostring-leak
Apr 3, 2026
Merged

fix: BIO resource leak in extractBioString() error paths#145
timlegge merged 3 commits into
cpan-authors:mainfrom
toddr-bot:koan.toddr.bot/fix-extractbiostring-leak

Conversation

@toddr-bot
Copy link
Copy Markdown
Contributor

@toddr-bot toddr-bot commented Mar 27, 2026

What

Fix BIO memory leak in extractBioString() when BIO_flush() or BIO_get_mem_data() fails.

Why

CHECK_OPEN_SSL calls croakSsl() which longjmps out of the function, skipping the BIO_free() at the end. This is the same pattern that was fixed in rsa_crypt(), sign(), verify(), and generate_key()extractBioString() was the last holdout.

Also, BIO_get_mem_data() can return -1 on error. When passed as size_t to newSVpv(), this would be reinterpreted as a huge positive value.

How

  • Replace CHECK_OPEN_SSL with explicit checks that BIO_free() before croakSsl()
  • Add guard on BIO_get_mem_data() return value
  • Drop the BIO_set_close() error check — it just sets a flag before the immediately following BIO_free() and cannot meaningfully fail on a mem BIO

Testing

All 507 tests pass on OpenSSL 3.5.1.

🤖 Generated with Claude Code


Quality Report

Changes: 1 file changed, 10 insertions(+), 3 deletions(-)

Code scan: clean

Tests: passed (OK)

Branch hygiene: clean

Generated by Kōan post-mission quality pipeline

Copy link
Copy Markdown
Member

@timlegge timlegge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine prevents memory leak in error case

@timlegge timlegge marked this pull request as ready for review March 29, 2026 01:15
@atoomic
Copy link
Copy Markdown
Collaborator

atoomic commented Apr 3, 2026

@toddr-bot rebase

@toddr-bot
Copy link
Copy Markdown
Contributor Author

Rebase: fix: BIO resource leak in extractBioString() error paths

Branch koan.toddr.bot/fix-extractbiostring-leak rebased onto main and force-pushed.

Diff: 1 file changed, 10 insertions(+), 3 deletions(-)

Review feedback was analyzed and applied.

Actions

  • Already-solved check: negative (confidence=high, reasoning=The recent commits fixed leaks in rsa_crypt(), sign(), verify(), generate_key(), and _new_key_from_p)
  • Warning: pending (unsubmitted) review comments detected
  • Rebased koan.toddr.bot/fix-extractbiostring-leak onto upstream/main
  • Force-pushed koan.toddr.bot/fix-extractbiostring-leak to origin
  • CI check enqueued (async)

CI

CI will be checked asynchronously.


Automated by Kōan

@toddr-bot toddr-bot force-pushed the koan.toddr.bot/fix-extractbiostring-leak branch from 835febe to 5562c11 Compare April 3, 2026 04:37
@atoomic
Copy link
Copy Markdown
Collaborator

atoomic commented Apr 3, 2026

view comment from #145 (comment)
@toddr-bot rebase and refactor the code mentioned above

toddr-bot added a commit to toddr-bot/Crypt-OpenSSL-RSA that referenced this pull request Apr 3, 2026
Refactored `extractBioString()` to use the `THROW`/`goto err` cleanup pattern per reviewer request:

- Replaced duplicated `BIO_free(); croakSsl()` error handling in two error paths with `THROW()` macro + centralized `err:` cleanup label
- This follows the same pattern used by `rsa_crypt()`, `sign()`, `verify()`, `generate_key()`, and `_new_key_from_params()` throughout `RSA.xs`
- Added `int error = 0` (required by the `THROW` macro)
- Retained the `BIO_get_mem_data() < 0` guard and the unchecked `BIO_set_close()` from the original fix
- The `err:` label frees the BIO then calls `CHECK_OPEN_SSL(0)` which croaks with the OpenSSL error, matching the idiom in `rsa_crypt()`
@toddr-bot
Copy link
Copy Markdown
Contributor Author

Rebase: fix: BIO resource leak in extractBioString() error paths

Branch koan.toddr.bot/fix-extractbiostring-leak rebased onto main and force-pushed.

Diff: 1 file changed, 11 insertions(+), 3 deletions(-)

Review feedback was analyzed and applied.

Changes

Refactored extractBioString() to use the THROW/goto err cleanup pattern per reviewer request:

  • Replaced duplicated BIO_free(); croakSsl() error handling in two error paths with THROW() macro + centralized err: cleanup label
  • This follows the same pattern used by rsa_crypt(), sign(), verify(), generate_key(), and _new_key_from_params() throughout RSA.xs
  • Added int error = 0 (required by the THROW macro)
  • Retained the BIO_get_mem_data() < 0 guard and the unchecked BIO_set_close() from the original fix
  • The err: label frees the BIO then calls CHECK_OPEN_SSL(0) which croaks with the OpenSSL error, matching the idiom in rsa_crypt()

Actions

  • Already-solved check: negative (confidence=high, reasoning=The recent commits fixed leaks in rsa_crypt(), sign(), verify(), generate_key(), and _new_key_from_p)
  • Warning: pending (unsubmitted) review comments detected
  • Rebased koan.toddr.bot/fix-extractbiostring-leak onto upstream/main
  • Applied review feedback
  • Force-pushed koan.toddr.bot/fix-extractbiostring-leak to origin
  • CI check enqueued (async)

CI

CI will be checked asynchronously.


Automated by Kōan

Comment thread RSA.xs Outdated
if (datasize < 0) {
BIO_free(p_stringBio);
croakSsl(__FILE__, __LINE__);
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could probably be simplified to read by using a macro for BIO_free & croakSsl so we do avoid introducing boilerplate and complexity to read / maintain

@atoomic atoomic requested a review from timlegge April 3, 2026 11:22
@atoomic
Copy link
Copy Markdown
Collaborator

atoomic commented Apr 3, 2026

@timlegge please review one more time after that small refactor to make the code easier to read / maintain using a shared block for the exception

Copy link
Copy Markdown
Member

@timlegge timlegge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@atoomic approved with comments

Comment thread RSA.xs

datasize = BIO_get_mem_data(p_stringBio, &datap);
THROW(datasize >= 0);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

strictly speaking this should be > 0 - 0 is returned if p_stringBio is null but 299 should fail in that case

Comment thread RSA.xs
sv = newSVpv(datap, datasize);

CHECK_OPEN_SSL(BIO_set_close(p_stringBio, BIO_CLOSE) == 1);
BIO_set_close(p_stringBio, BIO_CLOSE);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We lost the success check here - should wrap probably it in a throw but the p_stringBio was successfully set and the BIO is freed next so likely okay

@atoomic
Copy link
Copy Markdown
Collaborator

atoomic commented Apr 3, 2026

View comment from #145 (comment) and adjust accordingly
@toddr-bot rebase

toddr-bot and others added 3 commits April 3, 2026 13:57
CHECK_OPEN_SSL calls croakSsl() which longjmps, skipping BIO_free().
Replace with explicit error checks that free the BIO before croaking.

Also add a guard on BIO_get_mem_data() return value — a negative return
would be misinterpreted as a huge size_t by newSVpv().

The BIO_set_close() check is dropped since it just sets a flag before
the immediately following BIO_free(), and cannot meaningfully fail on
a mem BIO.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Refactored `extractBioString()` to use the `THROW`/`goto err` cleanup pattern per reviewer request:

- Replaced duplicated `BIO_free(); croakSsl()` error handling in two error paths with `THROW()` macro + centralized `err:` cleanup label
- This follows the same pattern used by `rsa_crypt()`, `sign()`, `verify()`, `generate_key()`, and `_new_key_from_params()` throughout `RSA.xs`
- Added `int error = 0` (required by the `THROW` macro)
- Retained the `BIO_get_mem_data() < 0` guard and the unchecked `BIO_set_close()` from the original fix
- The `err:` label frees the BIO then calls `CHECK_OPEN_SSL(0)` which croaks with the OpenSSL error, matching the idiom in `rsa_crypt()`
Done. Here's the summary for the commit message:

- Changed `BIO_get_mem_data()` guard from `datasize >= 0` to `datasize > 0` per @timlegge's review: `BIO_get_mem_data()` returns 0 when the BIO is NULL, which should be treated as an error (the NULL case would already be caught by the `BIO_flush()` check above, but `> 0` is strictly more correct)
@toddr-bot
Copy link
Copy Markdown
Contributor Author

Rebase: fix: BIO resource leak in extractBioString() error paths

Branch koan.toddr.bot/fix-extractbiostring-leak rebased onto main and force-pushed.

Diff: 1 file changed, 11 insertions(+), 3 deletions(-)

Review feedback was analyzed and applied.

Changes

Done. Here's the summary for the commit message:

  • Changed BIO_get_mem_data() guard from datasize >= 0 to datasize > 0 per @timlegge's review: BIO_get_mem_data() returns 0 when the BIO is NULL, which should be treated as an error (the NULL case would already be caught by the BIO_flush() check above, but > 0 is strictly more correct)

Actions

  • Already-solved check: negative (confidence=high, reasoning=The recent commits fixed leaks in rsa_crypt(), generate_key(), _new_key_from_parameters(), and get)
  • Rebased koan.toddr.bot/fix-extractbiostring-leak onto upstream/main
  • Applied review feedback
  • Force-pushed koan.toddr.bot/fix-extractbiostring-leak to origin
  • CI check enqueued (async)

CI

CI will be checked asynchronously.


Automated by Kōan

@toddr-bot toddr-bot force-pushed the koan.toddr.bot/fix-extractbiostring-leak branch from 6c6bc97 to 67513ea Compare April 3, 2026 13:57
@timlegge timlegge merged commit fc9d5b5 into cpan-authors:main Apr 3, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants