Skip to content

capturing stderr#7292

Merged
dogancanbakir merged 1 commit intodevfrom
5350-code-execution-error-logging
Mar 24, 2026
Merged

capturing stderr#7292
dogancanbakir merged 1 commit intodevfrom
5350-code-execution-error-logging

Conversation

@Mzack9999
Copy link
Member

@Mzack9999 Mzack9999 commented Mar 23, 2026

Proposed changes

Close #5350

Checklist

  • Pull request is created against the dev branch
  • All checks passed (lint, unit/integration/regression tests etc.) with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Summary by CodeRabbit

  • Improvements

    • Enhanced error visibility with warning logs when code execution fails
    • Added detailed debug logging for stderr output to improve troubleshooting
  • Tests

    • Expanded test coverage for code execution scenarios including stderr capture, mixed output handling, and exit code behavior

@auto-assign auto-assign bot requested a review from dwisiswant0 March 23, 2026 23:12
@neo-by-projectdiscovery-dev
Copy link

neo-by-projectdiscovery-dev bot commented Mar 23, 2026

Neo - PR Security Review

No security issues found

Highlights

  • Adds stderr capture to code protocol execution results
  • Exposes stderr in data map for template matchers/extractors
  • Includes comprehensive test coverage for stderr scenarios

Comment @pdneo help for available commands. · Open in Neo

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 23, 2026

Walkthrough

The PR adds warning-level logging for code execution failures and enhanced debug output for stderr capture in the code protocol handler. Concurrently, test infrastructure is refactored with helper functions to reduce duplication, and comprehensive test cases are added covering successful execution, stderr handling, and mixed output scenarios.

Changes

Cohort / File(s) Summary
Code Execution Error Handling
pkg/protocols/code/code.go
Added warning log emission when code execution fails (err != nil), and enhanced debug logging to output captured stderr content during result printing.
Test Infrastructure & Coverage
pkg/protocols/code/code_test.go
Refactored test setup into newTestRequest and executeRequest helper functions to reduce duplication. Added test cases validating stdout response capture, stderr capture, non-zero exit code handling, and mixed stdout/stderr output scenarios.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Poem

🐰 A hop, a skip, through error logs we go,
Stderr whispers now what failures show,
Tests refactored clean with helpers so neat,
Code execution woes find their defeat! 🥕✨

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning The PR partially addresses issue #5350 by capturing stderr and adding logging, but does not implement exit status checking or error generation as required. Implement exit status checking to generate errors when executed code returns non-zero status, as explicitly required in issue #5350's expected behavior.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'capturing stderr' is directly related to the main change; the PR adds stderr capture functionality and logging for code execution failures.
Out of Scope Changes check ✅ Passed All changes are in-scope: logging enhancements in code.go and comprehensive test coverage in code_test.go directly support stderr capture objectives.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 5350-code-execution-error-logging

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
pkg/protocols/code/code_test.go (1)

67-74: Assert execution error metadata in the failing-script case.

Given the PR objective, this test should also validate that failure context is surfaced in the event payload (e.g., gotEvent["error"] is non-empty), not just stderr content.

✅ Suggested test assertion
 	require.Contains(t, gotEvent["stderr"], "fail-message")
 	require.Empty(t, gotEvent["response"], "stdout should be empty when script only writes to stderr")
+	require.NotEmpty(t, gotEvent["error"], "execution error should be surfaced in event payload")
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/protocols/code/code_test.go` around lines 67 - 74, Update
TestCodeProtocolFailingScript to also assert that the failure metadata is
present in the event payload: after calling executeRequest (used with
newTestRequest) verify that gotEvent["error"] is non-empty (or contains the
expected error substring) in addition to the existing stderr/response checks so
the test confirms the protocol surfaces execution error context.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pkg/protocols/code/code.go`:
- Around line 280-282: The handler currently only logs execution errors but
doesn't include them in the event payload; update the error branch in the
execute code path (the block checking if err != nil in
pkg/protocols/code/code.go around request.options.TemplateID) to set the payload
map entry data["error"] (or the existing result/event payload map variable) to
the error information (e.g., err.Error() or the error object) so downstream
consumers can detect failures; keep the existing gologger.Warning() call but
ensure data["error"] is populated whenever err != nil.

---

Nitpick comments:
In `@pkg/protocols/code/code_test.go`:
- Around line 67-74: Update TestCodeProtocolFailingScript to also assert that
the failure metadata is present in the event payload: after calling
executeRequest (used with newTestRequest) verify that gotEvent["error"] is
non-empty (or contains the expected error substring) in addition to the existing
stderr/response checks so the test confirms the protocol surfaces execution
error context.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: bc0caeee-4c37-46c9-b662-dd587ad447f7

📥 Commits

Reviewing files that changed from the base of the PR and between e4edd5a and 3a004d1.

📒 Files selected for processing (2)
  • pkg/protocols/code/code.go
  • pkg/protocols/code/code_test.go

Comment on lines +280 to +282
if err != nil {
gologger.Warning().Msgf("[%s] Could not execute code: %s", request.options.TemplateID, err)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Propagate execution failures into the event payload, not only logs.

This currently logs the failure but does not set data["error"], so downstream result/event consumers still can’t reliably distinguish execution failure from non-match.

💡 Proposed fix
-	if err != nil {
-		gologger.Warning().Msgf("[%s] Could not execute code: %s", request.options.TemplateID, err)
-	}
+	execError := ""
+	if err != nil {
+		execError = err.Error()
+		gologger.Warning().Msgf("[%s] Could not execute code: %s", request.options.TemplateID, err)
+	}
@@
 	data["template-path"] = request.options.TemplatePath
 	data["template-id"] = request.options.TemplateID
 	data["template-info"] = request.options.TemplateInfo
+	if execError != "" {
+		data["error"] = execError
+	}
 	if gOutput.Stderr.Len() > 0 {
 		data["stderr"] = fmtStdout(gOutput.Stderr.String())
 	}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/protocols/code/code.go` around lines 280 - 282, The handler currently
only logs execution errors but doesn't include them in the event payload; update
the error branch in the execute code path (the block checking if err != nil in
pkg/protocols/code/code.go around request.options.TemplateID) to set the payload
map entry data["error"] (or the existing result/event payload map variable) to
the error information (e.g., err.Error() or the error object) so downstream
consumers can detect failures; keep the existing gologger.Warning() call but
ensure data["error"] is populated whenever err != nil.

@dogancanbakir dogancanbakir merged commit a86ee99 into dev Mar 24, 2026
21 checks passed
@dogancanbakir dogancanbakir deleted the 5350-code-execution-error-logging branch March 24, 2026 18:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Lack of error when code execution fails

2 participants