Skip to content

Agentcore optimization nys#1722

Merged
akshseh merged 19 commits into
awslabs:mainfrom
BharathiSrini:agentcore-optimization-nys
Jun 22, 2026
Merged

Agentcore optimization nys#1722
akshseh merged 19 commits into
awslabs:mainfrom
BharathiSrini:agentcore-optimization-nys

Conversation

@BharathiSrini

Copy link
Copy Markdown
Collaborator

Amazon Bedrock AgentCore Samples Pull Request

Adds insights sample to the optimize-your-agent section, because
AgentCore Insights is now publicly available and there was no end-to-end
code sample showing how to use it.

  • insights.py: runs FailureAnalysis, UserIntent, and ExecutionSummary
    against an HR Assistant agent using the public SDK (bedrock-agentcore
    1.15.0, boto3 1.43.32). Supports one-shot batch runs and recurring
    daily OnlineEvaluationConfig via --online flag.
  • requirements.txt: updated to public SDK versions, removed private
    WHL references.
  • README: documents the SDK workflow, CLI workflow (agentcore-cli
    v0.20.1+), the FailureAnalysis signal taxonomy with all 7 category
    strings, per-session detail structure (explanation, fixType,
    recommendation, failureSpans), and the triage-to-optimization flow.
    Insights is positioned as Step 3 in the CLI workflow, after deploy
    and baseline evaluation.

User experience

After this change, a user can pip install -r requirements.txt, deploy the HR Assistant with deploy.py, and run python insights.py --name
--generate-traces to get a full failure analysis with clustered root causes, signal categories, per-session span-level evidence, and
fix recommendations — all using the public SDK. The README also shows the equivalent CLI workflow with agentcore run insights and how to
chain insights into a system prompt recommendation.

Checklist

If your change doesn't seem to apply, please leave them unchecked.

  • I have reviewed the contributing guidelines
  • Add your name to CONTRIBUTORS.md
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • Are you uploading a dataset?
  • Have you documented Introduction, Architecture Diagram, Prerequisites, Usage, Sample Prompts, and Clean Up steps in your example README?
  • I agree to resolve any issues created for this example in the future.
  • I have performed a self-review of this change
  • Changes have been tested
  • Changes are documented

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.

- Add insights.py: runs FailureAnalysis, UserIntent, and ExecutionSummary
  batch insight jobs on the HR Assistant agent. Supports --generate-traces
  to send curated failure-mode sessions, --online to create a recurring
  daily OnlineEvaluationConfig, and --insight to select individual insight
  types. Uses both aws/spans and the runtime log group as data sources.

- Update README with a full Failure Insights section covering all three
  insight types, data source requirements, CLI examples, and how to chain
  insights into a system prompt recommendation.
@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

- Add encoding="utf-8" to file read/write calls
- Wrap long lines to stay within 100-char limit
- Add pylint disable comments for intentional broad-exception-caught
- Rename loop variable to avoid module-scope naming false positive
- Remove f-string prefix from string literals without placeholders (ruff fix)
@github-actions

github-actions Bot commented Jun 18, 2026

Copy link
Copy Markdown

Latest scan for commit: efeed70 | Updated: 2026-06-22 18:01:13 UTC

Security Scan Results

Scan Metadata

  • Project: ASH
  • Scan executed: 2026-06-22T18:00:53+00:00
  • ASH version: 3.0.0

Summary

Scanner Results

The table below shows findings by scanner, with status based on severity thresholds and dependencies:

Column Explanations:

Severity Levels (S/C/H/M/L/I):

  • Suppressed (S): Security findings that have been explicitly suppressed/ignored and don't affect the scanner's pass/fail status
  • Critical (C): The most severe security vulnerabilities requiring immediate remediation (e.g., SQL injection, remote code execution)
  • High (H): Serious security vulnerabilities that should be addressed promptly (e.g., authentication bypasses, privilege escalation)
  • Medium (M): Moderate security risks that should be addressed in normal development cycles (e.g., weak encryption, input validation issues)
  • Low (L): Minor security concerns with limited impact (e.g., information disclosure, weak recommendations)
  • Info (I): Informational findings for awareness with minimal security risk (e.g., code quality suggestions, best practice recommendations)

Other Columns:

  • Time: Duration taken by each scanner to complete its analysis
  • Action: Total number of actionable findings at or above the configured severity threshold that require attention

Scanner Results:

  • PASSED: Scanner found no security issues at or above the configured severity threshold - code is clean for this scanner
  • FAILED: Scanner found security vulnerabilities at or above the threshold that require attention and remediation
  • MISSING: Scanner could not run because required dependencies/tools are not installed or available
  • SKIPPED: Scanner was intentionally disabled or excluded from this scan
  • ERROR: Scanner encountered an execution error and could not complete successfully

Severity Thresholds (Thresh Column):

  • CRITICAL: Only Critical severity findings cause scanner to fail
  • HIGH: High and Critical severity findings cause scanner to fail
  • MEDIUM (MED): Medium, High, and Critical severity findings cause scanner to fail
  • LOW: Low, Medium, High, and Critical severity findings cause scanner to fail
  • ALL: Any finding of any severity level causes scanner to fail

Threshold Source: Values in parentheses indicate where the threshold is configured:

  • (g) = global: Set in the global_settings section of ASH configuration
  • (c) = config: Set in the individual scanner configuration section
  • (s) = scanner: Default threshold built into the scanner itself

Statistics calculation:

  • All statistics are calculated from the final aggregated SARIF report
  • Suppressed findings are counted separately and do not contribute to actionable findings
  • Scanner status is determined by comparing actionable findings to the threshold
Scanner S C H M L I Time Action Result Thresh
bandit 0 0 0 0 0 0 738ms 0 PASSED MED (g)
cdk-nag 0 0 0 0 0 0 6.2s 0 PASSED MED (g)
cfn-nag 0 0 0 0 0 0 8ms 0 PASSED MED (g)
checkov 0 0 0 0 0 0 5.1s 0 PASSED MED (g)
detect-secrets 0 0 0 0 0 0 715ms 0 PASSED MED (g)
grype 0 0 0 0 0 0 47.2s 0 PASSED MED (g)
npm-audit 0 0 0 0 0 0 197ms 0 PASSED MED (g)
opengrep 0 0 0 0 0 0 <1ms 0 SKIPPED MED (g)
semgrep 0 0 0 0 0 0 <1ms 0 MISSING MED (g)
syft 0 0 0 0 0 0 2.1s 0 PASSED MED (g)

@BharathiSrini BharathiSrini force-pushed the agentcore-optimization-nys branch from 2696476 to d592f27 Compare June 18, 2026 20:29

@akshseh akshseh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updates communicated

@github-actions github-actions Bot added the 03-AgentCore-identity 01-tutorials/03-AgentCore-identity label Jun 18, 2026
@BharathiSrini BharathiSrini requested a review from akshseh June 18, 2026 20:49
@BharathiSrini BharathiSrini force-pushed the agentcore-optimization-nys branch from 5dfbb1c to 15cea49 Compare June 18, 2026 21:14
Comment thread 06-workshops/12-AgentCore-optimization/optimization_tutorial.ipynb Outdated
@akshseh akshseh merged commit a444af4 into awslabs:main Jun 22, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

03-AgentCore-identity 01-tutorials/03-AgentCore-identity 06-workshops

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants