From d5fdbcd0d88453a270640286be6b79f299318b16 Mon Sep 17 00:00:00 2001 From: notgitika Date: Thu, 18 Jun 2026 11:39:21 -0400 Subject: [PATCH 1/5] docs: cover payments, policies/guardrails, insights, harness, datasets, web-search MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The README and the vended AGENTS.md (copied into every agentcore create project as the AI-assistant context file) had grown stale relative to the feature surface. Several capabilities had docs in docs/ but no mention in the README, and the AGENTS.md customers see still listed only the original handful of resource types. README.md - Broaden 'add'/'remove' descriptions to enumerate the actual resource types (15+ now, was 'agents, memory, credentials, evaluators, targets') - Add command-table sections: Knowledge Bases, Insights ([preview]), Harness, Policies & Guardrails, Payments, Datasets, Web Search Gateway Targets — each linking to docs/ where one already exists - Refresh Capabilities to list the actual feature set - Group Documentation links into Reference / Resources & features / Evaluation & quality / Operations and add the missing entries (payments, container-builds, transaction_search, telemetry, tui-harness, PERMISSIONS, TESTING) src/assets/agents/AGENTS.md - AgentCoreProjectSpec resource list now reflects every top-level array in the schema (was missing onlineInsightsConfigs, knowledgeBases, harnesses, policyEngines, policies, payments, configBundles, datasets, runtimeEndpoints) - Key Types covers the new resource shapes - Common Enum Values adds PaymentProvider, PolicyEnforcementMode, GuardrailContentFilter (the spelled-out 'INSULTS' enum so AI assistants emit the canonical name) - CLI Commands replaced with grouped tables (lifecycle / resources / jobs / config bundles & datasets / observability / utilities) so the customer-vended file matches what the CLI actually exposes --- README.md | 139 ++++++++++++++++-- .../assets.snapshot.test.ts.snap | 86 +++++++++-- src/assets/agents/AGENTS.md | 86 +++++++++-- 3 files changed, 274 insertions(+), 37 deletions(-) diff --git a/README.md b/README.md index 32b3ab3dc..84eec0531 100644 --- a/README.md +++ b/README.md @@ -91,10 +91,10 @@ agentcore invoke ### Resource Management -| Command | Description | -| -------- | ---------------------------------------------------- | -| `add` | Add agents, memory, credentials, evaluators, targets | -| `remove` | Remove resources from project | +| Command | Description | +| -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `add` | Add agents, memory, credentials, gateways and gateway-targets, evaluators, online evals, online insights, knowledge bases, config bundles, datasets, harnesses, policy engines and policies, payment managers and payment connectors, runtime endpoints | +| `remove` | Remove any of the above resources from the project | > **Note**: Run `agentcore deploy` after `add` or `remove` to update resources in AWS. @@ -145,6 +145,88 @@ agentcore invoke | `promote ab-test` | Apply the winning variant to `agentcore.json` | | `archive ab-test` | Delete the test on the service and clear local history | +### Knowledge Bases + +| Command | Description | +| -------------------- | ------------------------------------------------------- | +| `add knowledge-base` | Add a managed Bedrock Knowledge Base wired to a gateway | + +> See [Knowledge Bases](docs/knowledge-bases.md) for ingestion, vectorization, and retrieval setup. + +### Insights — `[preview]` + +Failure-pattern analysis across agent sessions. Insights configs run continuously alongside online evals and surface +clusters of bad outcomes. + +| Command | Description | +| ------------------------ | ----------------------------------------------------------- | +| `add online-insights` | Add a continuous insights config bound to a runtime | +| `run insights` | Run on-demand failure analysis across recent sessions | +| `view insights` | List insights jobs or view one in detail | +| `pause online-insights` | Pause a deployed online insights config | +| `resume online-insights` | Resume a paused online insights config | +| `archive insights` | Delete an insights job on the service + clear local history | + +### Harness + +A harness bundles a runtime, model, tools, skills, memory, and observability into one declarative config. Use it when +you want infra without writing agent code. + +| Command | Description | +| --------------------- | ------------------------------------------------------------------------- | +| `add harness` | Add a harness resource (runtime + tools + skills + memory) | +| `add harness --tool` | Add a tool to an existing harness | +| `add harness --skill` | Add a skill to an existing harness | +| `export harness` | Export a harness config to a deployable Strands Python agent under `app/` | + +> After `export harness`, **read `app//EXPORT_NOTES.md`** before running `deploy` — it lists any manual +> follow-up the exporter could not automate. + +### Policies & Guardrails + +Policy engines apply Cedar-based pre/post-call policies to agent invocations — including Bedrock content filters +(`VIOLENCE`, `HATE`, `SEXUAL`, `MISCONDUCT`, `INSULTS`), prompt-attack detection, and sensitive-information redaction. + +| Command | Description | +| ------------------- | -------------------------------------------------------------------- | +| `add policy-engine` | Add a Cedar policy engine to the project | +| `add policy` | Add a policy to a policy engine (form-based guardrails or raw Cedar) | + +### Payments + +Pay-per-call agent transactions via the [x402 protocol](https://www.x402.org/). When a tool call returns +`402 Payment Required`, the payments system signs and submits payment then retries automatically. + +| Command | Description | +| ----------------------- | ---------------------------------------------------------------------------- | +| `add payment-manager` | Add a payment manager (orchestrates payment sessions for the agent) | +| `add payment-connector` | Add a payment connector with provider credentials (CoinbaseCDP, StripePrivy) | + +> See [Payments](docs/payments.md) for the full setup including instrument creation and tool allowlists. + +### Datasets + +Curated session datasets for batch evaluation and recommendation runs. + +| Command | Description | +| ------------------------- | ------------------------------------------------------------------------- | +| `add dataset` | Add a dataset resource (session list, ground-truth file, or trace filter) | +| `dataset download` | Download a dataset version locally | +| `dataset publish-version` | Publish a new dataset version | +| `dataset remove-version` | Remove a dataset version | + +### Web Search Gateway Targets + +Add a managed web-search target to a gateway: + +```bash +agentcore add gateway-target --type connector --connector web-search \ + --gateway --name +# Optional: --exclude-domains "example.com,foo.org" +``` + +See [Gateway](docs/gateway.md) for full target setup including Lambda, MCP, OpenAPI, Smithy, and API Gateway. + ### Utilities | Command | Description | @@ -181,31 +263,62 @@ my-project/ Projects use JSON schema files in the `agentcore/` directory: -- `agentcore.json` - Agent specifications, memory, credentials, evaluators, online evals +- `agentcore.json` - Project resources (agents, memory, credentials, gateways, evaluators, online evals/insights, + knowledge bases, harnesses, policy engines and policies, payment managers and connectors, config bundles, datasets, + runtime endpoints) - `deployed-state.json` - Runtime state in agentcore/.cli/ (auto-managed) - `aws-targets.json` - Deployment targets (account, region) ## Capabilities - **Runtime** - Managed execution environment for deployed agents -- **Memory** - Semantic, summarization, and user preference strategies -- **Credentials** - Secure API key management via Secrets Manager -- **Evaluations** - LLM-as-a-Judge for on-demand and continuous agent quality monitoring +- **Memory** - Semantic, summarization, user-preference, and episodic strategies +- **Credentials** - Secure API key + OAuth credential management via Secrets Manager +- **Gateways** - MCP gateways with Lambda / MCP server / OpenAPI / Smithy / API Gateway / **web-search** / + **knowledge-base** targets +- **Evaluations** - LLM-as-a-Judge for on-demand, batch, and continuous agent quality monitoring +- **Recommendations** - Auto-optimize system prompts and tool descriptions from real session traces +- **A/B Tests** - Traffic-split between config-bundle or target-based variants and promote the winner +- **Insights** _[preview]_ - Failure-pattern analysis and clustering across agent sessions +- **Knowledge Bases** - Managed Bedrock Knowledge Bases auto-wired to gateways +- **Harness** - Declarative agent: bundle runtime + tools + skills + memory + observability without writing agent code +- **Policies & Guardrails** - Cedar pre/post-call policies including Bedrock content filters, prompt-attack detection, + and sensitive-information redaction +- **Payments** - x402-protocol microtransactions for pay-per-call tools and APIs +- **Config Bundles** - Versioned runtime configurations as a separately-deployable resource ## Documentation +**Reference** + - [CLI Commands Reference](docs/commands.md) - Full command reference for scripting and CI/CD - [Configuration](docs/configuration.md) - Schema reference for config files +- [Frameworks](docs/frameworks.md) - Supported frameworks and model providers +- [PERMISSIONS](docs/PERMISSIONS.md) - IAM permissions required to deploy + +**Resources & features** + +- [Memory](docs/memory.md) - Memory strategies and sharing +- [Gateway](docs/gateway.md) - Gateway setup, targets, and authentication +- [Knowledge Bases](docs/knowledge-bases.md) - Managed Bedrock Knowledge Bases wired to gateways +- [Payments](docs/payments.md) - x402-protocol microtransactions for paid tools/APIs +- [Config Bundles](docs/config-bundles.md) - Versioned runtime configurations +- [Container Builds](docs/container-builds.md) - Container build types and Dockerfile setup + +**Evaluation & quality** + - [Evaluations](docs/evals.md) - Evaluators, on-demand evals, and online monitoring - [Batch Evaluation](docs/batch-evaluation.md) - Run evaluators across sessions at scale - [Recommendations](docs/recommendations.md) - Optimize prompts and tool descriptions - [A/B Tests](docs/ab-tests.md) - Split traffic between variants and promote the winner -- [Config Bundles](docs/config-bundles.md) - Versioned runtime configurations -- [Frameworks](docs/frameworks.md) - Supported frameworks and model providers -- [Gateway](docs/gateway.md) - Gateway setup, targets, and authentication -- [Knowledge Bases](docs/knowledge-bases.md) - Managed Bedrock Knowledge Bases wired to gateways -- [Memory](docs/memory.md) - Memory strategies and sharing + +**Operations** + - [Local Development](docs/local-development.md) - Dev server and debugging +- [Transaction Search](docs/transaction_search.md) - Trace + log search across agent invocations +- [Telemetry](docs/telemetry.md) - CLI usage telemetry — what's collected and how to opt out +- [TUI Harness](docs/tui-harness.md) - Programmatic TUI driver for testing +- [Testing](docs/TESTING.md) - Unit, integration, and e2e test infrastructure - [Feedback](docs/feedback.md) - Submit feedback from your terminal ## Examples diff --git a/src/assets/__tests__/__snapshots__/assets.snapshot.test.ts.snap b/src/assets/__tests__/__snapshots__/assets.snapshot.test.ts.snap index 4b42f93a5..22242c4ec 100644 --- a/src/assets/__tests__/__snapshots__/assets.snapshot.test.ts.snap +++ b/src/assets/__tests__/__snapshots__/assets.snapshot.test.ts.snap @@ -7148,13 +7148,21 @@ file maps to a JSON config file and includes validation constraints as comments ### Key Types -- **AgentCoreProjectSpec**: Root config with \`runtimes\`, \`memories\`, \`credentials\`, \`agentCoreGateways\`, \`evaluators\`, \`onlineEvalConfigs\`, \`policyEngines\` arrays +- **AgentCoreProjectSpec**: Root config with \`runtimes\`, \`memories\`, \`credentials\`, \`agentCoreGateways\`, \`evaluators\`, \`onlineEvalConfigs\`, \`onlineInsightsConfigs\`, \`knowledgeBases\`, \`harnesses\`, \`policyEngines\`, \`policies\`, \`payments\` (managers + connectors), \`configBundles\`, \`datasets\`, \`runtimeEndpoints\` arrays - **AgentEnvSpec**: Agent configuration (build type, entrypoint, code location, runtime version, network mode) - **Memory**: Memory resource with strategies (SEMANTIC, SUMMARIZATION, USER_PREFERENCE, EPISODIC) and expiry - **Credential**: API key or OAuth credential provider -- **AgentCoreGateway**: MCP gateway with targets (Lambda, MCP server, OpenAPI, Smithy, API Gateway) +- **AgentCoreGateway**: MCP gateway with targets (Lambda, MCP server, OpenAPI, Smithy, API Gateway, web-search, knowledge-base) - **Evaluator**: LLM-as-a-Judge or code-based evaluator - **OnlineEvalConfig**: Continuous evaluation pipeline bound to an agent +- **OnlineInsightsConfig** _[preview]_: Continuous failure-pattern analysis bound to an agent +- **KnowledgeBase**: Managed Bedrock Knowledge Base auto-wired to a gateway +- **Harness**: Declarative agent — runtime + tools + skills + memory + observability without writing agent code +- **PolicyEngine** + **Policy**: Cedar policy engine with form-based guardrails (Bedrock content filters, prompt-attack, sensitive-info) or raw Cedar policies +- **PaymentManager** + **PaymentConnector**: x402-protocol payment orchestration with provider credentials (CoinbaseCDP, StripePrivy) +- **ConfigBundle**: Versioned runtime configuration as a separately-deployable resource +- **Dataset**: Curated session dataset for batch evaluation and recommendation runs +- **RuntimeEndpoint**: Named endpoint (e.g. \`PROMPT_V1\`) targeting a specific runtime version ### Common Enum Values @@ -7162,8 +7170,11 @@ file maps to a JSON config file and includes validation constraints as comments - **NetworkMode**: \`'PUBLIC'\` | \`'VPC'\` - **RuntimeVersion**: \`'PYTHON_3_10'\` | \`'PYTHON_3_11'\` | \`'PYTHON_3_12'\` | \`'PYTHON_3_13'\` | \`'PYTHON_3_14'\` | \`'NODE_18'\` | \`'NODE_20'\` | \`'NODE_22'\` - **MemoryStrategyType**: \`'SEMANTIC'\` | \`'SUMMARIZATION'\` | \`'USER_PREFERENCE'\` | \`'EPISODIC'\` -- **GatewayTargetType**: \`'lambda'\` | \`'mcpServer'\` | \`'openApiSchema'\` | \`'smithyModel'\` | \`'apiGateway'\` | \`'lambdaFunctionArn'\` +- **GatewayTargetType**: \`'lambda'\` | \`'mcpServer'\` | \`'openApiSchema'\` | \`'smithyModel'\` | \`'apiGateway'\` | \`'lambdaFunctionArn'\` | \`'connector'\` (web-search, bedrock-knowledge-bases, bedrock-agentic-retrieve) - **ModelProvider**: \`'Bedrock'\` | \`'Gemini'\` | \`'OpenAI'\` | \`'Anthropic'\` +- **PaymentProvider**: \`'CoinbaseCDP'\` | \`'StripePrivy'\` +- **PolicyEnforcementMode**: \`'ACTIVE'\` | \`'PASSIVE'\` +- **GuardrailContentFilter**: \`'VIOLENCE'\` | \`'HATE'\` | \`'SEXUAL'\` | \`'MISCONDUCT'\` | \`'INSULTS'\` ### Build Types @@ -7229,22 +7240,73 @@ cat app//EXPORT_NOTES.md # read this before touching anyt ## CLI Commands +Run \`agentcore --help\` or \`agentcore --help\` for full flags. Commonly used: + +**Project lifecycle** + | Command | Description | | --- | --- | | \`agentcore create\` | Create a new project | -| \`agentcore add \` | Add agent, memory, credential, gateway, evaluator, policy | -| \`agentcore remove \` | Remove a resource | -| \`agentcore export harness\` | Export a harness to a Strands runtime agent | | \`agentcore dev\` | Run agent locally with hot-reload | | \`agentcore deploy\` | Deploy to AWS | -| \`agentcore status\` | Show deployment status | | \`agentcore invoke\` | Invoke agent (local or deployed) | -| \`agentcore logs\` | View agent logs | -| \`agentcore traces\` | View agent traces | -| \`agentcore eval\` | Run evaluations against an agent | -| \`agentcore package\` | Package agent artifacts | +| \`agentcore status\` | Show deployment status | | \`agentcore validate\` | Validate configuration | -| \`agentcore pause\` / \`resume\` | Pause or resume a deployed agent | +| \`agentcore package\` | Package agent artifacts | +| \`agentcore import\` | Import resources from a Bedrock AgentCore Starter Toolkit project | + +**Resources** + +| Command | Description | +| --- | --- | +| \`agentcore add \` | Add agent, memory, credential, gateway, gateway-target, evaluator, online-eval, online-insights, knowledge-base, harness, policy-engine, policy, payment-manager, payment-connector, config-bundle, dataset, runtime-endpoint | +| \`agentcore remove \` | Remove any resource | +| \`agentcore export harness\` | Export a harness to a Strands runtime agent under \`app//\` | + +**Jobs (run, view, archive, lifecycle)** + +| Command | Description | +| --- | --- | +| \`agentcore run eval\` | Run on-demand evaluation against agent traces | +| \`agentcore run batch-evaluation\` | Run evaluators across all sessions at scale | +| \`agentcore run recommendation\` | Optimize prompts or tool descriptions from real traces | +| \`agentcore run insights\` _[preview]_ | Run failure-pattern analysis across sessions | +| \`agentcore run ab-test\` | Start an A/B test (config-bundle or target-based) | +| \`agentcore run ingest\` | Ingest dataset rows for evaluation | +| \`agentcore view \` | List or view jobs (recommendation, batch-evaluation, ab-test, insights) | +| \`agentcore archive \` | Delete a job on the service + clear local history | +| \`agentcore stop \` | Stop a running batch-evaluation or ab-test | +| \`agentcore promote ab-test\` | Apply the winning variant to \`agentcore.json\` | +| \`agentcore pause \` / \`agentcore resume \` | Pause/resume a deployed online-eval, online-insights, or ab-test | + +**Config bundles & datasets** + +| Command | Description | +| --- | --- | +| \`agentcore config-bundle versions\` (alias \`cb versions\`) | List version history for a bundle | +| \`agentcore config-bundle diff\` | Diff two versions of a bundle | +| \`agentcore config-bundle create-branch\` | Create a new branch on an existing bundle | +| \`agentcore dataset download\` | Download a dataset version locally | +| \`agentcore dataset publish-version\` | Publish a new dataset version | +| \`agentcore dataset remove-version\` | Remove a dataset version | + +**Observability & history** + +| Command | Description | +| --- | --- | +| \`agentcore logs\` | Stream/search agent runtime logs | +| \`agentcore logs evals\` | Stream/search online-eval logs | +| \`agentcore traces list\` / \`agentcore traces get\` | List recent traces or download one to JSON | +| \`agentcore evals history\` | View past on-demand eval results | + +**Utilities** + +| Command | Description | +| --- | --- | +| \`agentcore fetch access\` | Fetch access info for deployed gateway or agent | +| \`agentcore feedback\` | Send feedback (with optional screenshot) to the AgentCore team | +| \`agentcore update\` | Check for and install CLI updates | +| \`agentcore telemetry\` | View or change telemetry preferences | " `; diff --git a/src/assets/agents/AGENTS.md b/src/assets/agents/AGENTS.md index 431ce6cc3..d60ba4446 100644 --- a/src/assets/agents/AGENTS.md +++ b/src/assets/agents/AGENTS.md @@ -56,13 +56,21 @@ file maps to a JSON config file and includes validation constraints as comments ### Key Types -- **AgentCoreProjectSpec**: Root config with `runtimes`, `memories`, `credentials`, `agentCoreGateways`, `evaluators`, `onlineEvalConfigs`, `policyEngines` arrays +- **AgentCoreProjectSpec**: Root config with `runtimes`, `memories`, `credentials`, `agentCoreGateways`, `evaluators`, `onlineEvalConfigs`, `onlineInsightsConfigs`, `knowledgeBases`, `harnesses`, `policyEngines`, `policies`, `payments` (managers + connectors), `configBundles`, `datasets`, `runtimeEndpoints` arrays - **AgentEnvSpec**: Agent configuration (build type, entrypoint, code location, runtime version, network mode) - **Memory**: Memory resource with strategies (SEMANTIC, SUMMARIZATION, USER_PREFERENCE, EPISODIC) and expiry - **Credential**: API key or OAuth credential provider -- **AgentCoreGateway**: MCP gateway with targets (Lambda, MCP server, OpenAPI, Smithy, API Gateway) +- **AgentCoreGateway**: MCP gateway with targets (Lambda, MCP server, OpenAPI, Smithy, API Gateway, web-search, knowledge-base) - **Evaluator**: LLM-as-a-Judge or code-based evaluator - **OnlineEvalConfig**: Continuous evaluation pipeline bound to an agent +- **OnlineInsightsConfig** _[preview]_: Continuous failure-pattern analysis bound to an agent +- **KnowledgeBase**: Managed Bedrock Knowledge Base auto-wired to a gateway +- **Harness**: Declarative agent — runtime + tools + skills + memory + observability without writing agent code +- **PolicyEngine** + **Policy**: Cedar policy engine with form-based guardrails (Bedrock content filters, prompt-attack, sensitive-info) or raw Cedar policies +- **PaymentManager** + **PaymentConnector**: x402-protocol payment orchestration with provider credentials (CoinbaseCDP, StripePrivy) +- **ConfigBundle**: Versioned runtime configuration as a separately-deployable resource +- **Dataset**: Curated session dataset for batch evaluation and recommendation runs +- **RuntimeEndpoint**: Named endpoint (e.g. `PROMPT_V1`) targeting a specific runtime version ### Common Enum Values @@ -70,8 +78,11 @@ file maps to a JSON config file and includes validation constraints as comments - **NetworkMode**: `'PUBLIC'` | `'VPC'` - **RuntimeVersion**: `'PYTHON_3_10'` | `'PYTHON_3_11'` | `'PYTHON_3_12'` | `'PYTHON_3_13'` | `'PYTHON_3_14'` | `'NODE_18'` | `'NODE_20'` | `'NODE_22'` - **MemoryStrategyType**: `'SEMANTIC'` | `'SUMMARIZATION'` | `'USER_PREFERENCE'` | `'EPISODIC'` -- **GatewayTargetType**: `'lambda'` | `'mcpServer'` | `'openApiSchema'` | `'smithyModel'` | `'apiGateway'` | `'lambdaFunctionArn'` +- **GatewayTargetType**: `'lambda'` | `'mcpServer'` | `'openApiSchema'` | `'smithyModel'` | `'apiGateway'` | `'lambdaFunctionArn'` | `'connector'` (web-search, bedrock-knowledge-bases, bedrock-agentic-retrieve) - **ModelProvider**: `'Bedrock'` | `'Gemini'` | `'OpenAI'` | `'Anthropic'` +- **PaymentProvider**: `'CoinbaseCDP'` | `'StripePrivy'` +- **PolicyEnforcementMode**: `'ACTIVE'` | `'PASSIVE'` +- **GuardrailContentFilter**: `'VIOLENCE'` | `'HATE'` | `'SEXUAL'` | `'MISCONDUCT'` | `'INSULTS'` ### Build Types @@ -137,19 +148,70 @@ cat app//EXPORT_NOTES.md # read this before touching anyt ## CLI Commands +Run `agentcore --help` or `agentcore --help` for full flags. Commonly used: + +**Project lifecycle** + | Command | Description | | --- | --- | | `agentcore create` | Create a new project | -| `agentcore add ` | Add agent, memory, credential, gateway, evaluator, policy | -| `agentcore remove ` | Remove a resource | -| `agentcore export harness` | Export a harness to a Strands runtime agent | | `agentcore dev` | Run agent locally with hot-reload | | `agentcore deploy` | Deploy to AWS | -| `agentcore status` | Show deployment status | | `agentcore invoke` | Invoke agent (local or deployed) | -| `agentcore logs` | View agent logs | -| `agentcore traces` | View agent traces | -| `agentcore eval` | Run evaluations against an agent | -| `agentcore package` | Package agent artifacts | +| `agentcore status` | Show deployment status | | `agentcore validate` | Validate configuration | -| `agentcore pause` / `resume` | Pause or resume a deployed agent | +| `agentcore package` | Package agent artifacts | +| `agentcore import` | Import resources from a Bedrock AgentCore Starter Toolkit project | + +**Resources** + +| Command | Description | +| --- | --- | +| `agentcore add ` | Add agent, memory, credential, gateway, gateway-target, evaluator, online-eval, online-insights, knowledge-base, harness, policy-engine, policy, payment-manager, payment-connector, config-bundle, dataset, runtime-endpoint | +| `agentcore remove ` | Remove any resource | +| `agentcore export harness` | Export a harness to a Strands runtime agent under `app//` | + +**Jobs (run, view, archive, lifecycle)** + +| Command | Description | +| --- | --- | +| `agentcore run eval` | Run on-demand evaluation against agent traces | +| `agentcore run batch-evaluation` | Run evaluators across all sessions at scale | +| `agentcore run recommendation` | Optimize prompts or tool descriptions from real traces | +| `agentcore run insights` _[preview]_ | Run failure-pattern analysis across sessions | +| `agentcore run ab-test` | Start an A/B test (config-bundle or target-based) | +| `agentcore run ingest` | Ingest dataset rows for evaluation | +| `agentcore view ` | List or view jobs (recommendation, batch-evaluation, ab-test, insights) | +| `agentcore archive ` | Delete a job on the service + clear local history | +| `agentcore stop ` | Stop a running batch-evaluation or ab-test | +| `agentcore promote ab-test` | Apply the winning variant to `agentcore.json` | +| `agentcore pause ` / `agentcore resume ` | Pause/resume a deployed online-eval, online-insights, or ab-test | + +**Config bundles & datasets** + +| Command | Description | +| --- | --- | +| `agentcore config-bundle versions` (alias `cb versions`) | List version history for a bundle | +| `agentcore config-bundle diff` | Diff two versions of a bundle | +| `agentcore config-bundle create-branch` | Create a new branch on an existing bundle | +| `agentcore dataset download` | Download a dataset version locally | +| `agentcore dataset publish-version` | Publish a new dataset version | +| `agentcore dataset remove-version` | Remove a dataset version | + +**Observability & history** + +| Command | Description | +| --- | --- | +| `agentcore logs` | Stream/search agent runtime logs | +| `agentcore logs evals` | Stream/search online-eval logs | +| `agentcore traces list` / `agentcore traces get` | List recent traces or download one to JSON | +| `agentcore evals history` | View past on-demand eval results | + +**Utilities** + +| Command | Description | +| --- | --- | +| `agentcore fetch access` | Fetch access info for deployed gateway or agent | +| `agentcore feedback` | Send feedback (with optional screenshot) to the AgentCore team | +| `agentcore update` | Check for and install CLI updates | +| `agentcore telemetry` | View or change telemetry preferences | From 2abc1a056ea7dad9dab2a343dda11b9c93dec5b0 Mon Sep 17 00:00:00 2001 From: notgitika Date: Thu, 18 Jun 2026 13:58:24 -0400 Subject: [PATCH 2/5] docs: mark Harness section as preview + npm install -g @aws/agentcore@preview MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per Avi's review on PR #1577 — harness commands are only registered in preview builds (registry.ts:26 gates HarnessPrimitive on isPreviewEnabled()), so a GA-tarball user reading the main README would hit 'unknown command' if they tried any of the harness commands. Add a [preview] tag to the section header and a one-line install hint. --- README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 84eec0531..5a07cf93d 100644 --- a/README.md +++ b/README.md @@ -167,7 +167,13 @@ clusters of bad outcomes. | `resume online-insights` | Resume a paused online insights config | | `archive insights` | Delete an insights job on the service + clear local history | -### Harness +### Harness — `[preview]` + +> Harness commands are only available in the preview release of the CLI. Install it with: +> +> ```bash +> npm install -g @aws/agentcore@preview +> ``` A harness bundles a runtime, model, tools, skills, memory, and observability into one declarative config. Use it when you want infra without writing agent code. From e104e721cbe48ed20dea3bfacd4ca267cb48ebc6 Mon Sep 17 00:00:00 2001 From: notgitika Date: Thu, 18 Jun 2026 14:30:08 -0400 Subject: [PATCH 3/5] =?UTF-8?q?docs:=20fix=20Harness=20commands=20?= =?UTF-8?q?=E2=80=94=20add=20tool/add=20skill=20are=20top-level?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Avi spotted that 'add harness --tool' and 'add harness --skill' are wrong: tool and skill are top-level subcommands under add, not flags on add harness. Verified against src/cli/cli.ts:137-139 (registerAddTool/registerAddSkill) and src/cli/commands/add/{tool,skill}-command.ts. Real shape: 'agentcore add tool --harness --type ' and 'agentcore add skill --harness --path|--s3|--git'. Also clarified 'add harness' is just runtime + model + memory — tools and skills get added separately afterward. Audited every other command row in the README against the source while in here. Everything else (config-bundle aliases, run/view/pause/resume/ stop/promote/archive subcommands, evals history, online-eval/insights pairs, payment commands, policy-engine/policy, knowledge-base, dataset subcommands, web-search via gateway-target connector) matches what the CLI actually exposes. --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 5a07cf93d..1ec84d801 100644 --- a/README.md +++ b/README.md @@ -178,12 +178,12 @@ clusters of bad outcomes. A harness bundles a runtime, model, tools, skills, memory, and observability into one declarative config. Use it when you want infra without writing agent code. -| Command | Description | -| --------------------- | ------------------------------------------------------------------------- | -| `add harness` | Add a harness resource (runtime + tools + skills + memory) | -| `add harness --tool` | Add a tool to an existing harness | -| `add harness --skill` | Add a skill to an existing harness | -| `export harness` | Export a harness config to a deployable Strands Python agent under `app/` | +| Command | Description | +| ---------------- | --------------------------------------------------------------------------- | +| `add harness` | Add a harness resource (runtime + model + memory) | +| `add tool` | Add a tool to a harness (`--harness --type --name `) | +| `add skill` | Add a skill to a harness (`--harness ` + `--path` / `--s3` / `--git`) | +| `export harness` | Export a harness config to a deployable Strands Python agent under `app/` | > After `export harness`, **read `app//EXPORT_NOTES.md`** before running `deploy` — it lists any manual > follow-up the exporter could not automate. From 0dba37cefd8e938f10e8b9bdf9e2af73efb340c3 Mon Sep 17 00:00:00 2001 From: notgitika Date: Thu, 18 Jun 2026 15:16:23 -0400 Subject: [PATCH 4/5] docs: fix run ingest description + mark Payments [preview] MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two more mismatches found while addressing Avi's review of PR #1577: - 'agentcore run ingest' in the vended AGENTS.md said 'Ingest dataset rows for evaluation' but the actual command is for knowledge base ingestion. Source description (run/command.tsx:657) is 'Start a fresh ingestion job for every data source on a deployed knowledge base.' — --name validates against project.knowledgeBases. Updated the AGENTS.md row + regenerated the snapshot. - Payments commands carry [preview] tags in source (PaymentManagerPrimitive.ts:325, PaymentConnectorPrimitive.ts:317 and the matching remove counterparts) but the README's Payments section had no preview tag. Added [preview] to the section header + 'npm install -g @aws/agentcore@preview' install hint, mirroring what we did for Harness. Audited every other command's .description() text in source vs my README/AGENTS.md rows; nothing else mismatched. The remaining feature sections (Knowledge Bases, Policies & Guardrails, Datasets, Config Bundles, A/B Tests, Evaluations) are not preview-gated in source. --- README.md | 8 +++++++- .../__tests__/__snapshots__/assets.snapshot.test.ts.snap | 2 +- src/assets/agents/AGENTS.md | 2 +- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 1ec84d801..774660739 100644 --- a/README.md +++ b/README.md @@ -198,7 +198,13 @@ Policy engines apply Cedar-based pre/post-call policies to agent invocations — | `add policy-engine` | Add a Cedar policy engine to the project | | `add policy` | Add a policy to a policy engine (form-based guardrails or raw Cedar) | -### Payments +### Payments — `[preview]` + +> Payment commands are only available in the preview release of the CLI. Install it with: +> +> ```bash +> npm install -g @aws/agentcore@preview +> ``` Pay-per-call agent transactions via the [x402 protocol](https://www.x402.org/). When a tool call returns `402 Payment Required`, the payments system signs and submits payment then retries automatically. diff --git a/src/assets/__tests__/__snapshots__/assets.snapshot.test.ts.snap b/src/assets/__tests__/__snapshots__/assets.snapshot.test.ts.snap index 22242c4ec..154ece89b 100644 --- a/src/assets/__tests__/__snapshots__/assets.snapshot.test.ts.snap +++ b/src/assets/__tests__/__snapshots__/assets.snapshot.test.ts.snap @@ -7272,7 +7272,7 @@ Run \`agentcore --help\` or \`agentcore --help\` for full flags. Commo | \`agentcore run recommendation\` | Optimize prompts or tool descriptions from real traces | | \`agentcore run insights\` _[preview]_ | Run failure-pattern analysis across sessions | | \`agentcore run ab-test\` | Start an A/B test (config-bundle or target-based) | -| \`agentcore run ingest\` | Ingest dataset rows for evaluation | +| \`agentcore run ingest\` | Start a fresh ingestion job for every data source on a deployed knowledge base | | \`agentcore view \` | List or view jobs (recommendation, batch-evaluation, ab-test, insights) | | \`agentcore archive \` | Delete a job on the service + clear local history | | \`agentcore stop \` | Stop a running batch-evaluation or ab-test | diff --git a/src/assets/agents/AGENTS.md b/src/assets/agents/AGENTS.md index d60ba4446..7fd817773 100644 --- a/src/assets/agents/AGENTS.md +++ b/src/assets/agents/AGENTS.md @@ -180,7 +180,7 @@ Run `agentcore --help` or `agentcore --help` for full flags. Commonly | `agentcore run recommendation` | Optimize prompts or tool descriptions from real traces | | `agentcore run insights` _[preview]_ | Run failure-pattern analysis across sessions | | `agentcore run ab-test` | Start an A/B test (config-bundle or target-based) | -| `agentcore run ingest` | Ingest dataset rows for evaluation | +| `agentcore run ingest` | Start a fresh ingestion job for every data source on a deployed knowledge base | | `agentcore view ` | List or view jobs (recommendation, batch-evaluation, ab-test, insights) | | `agentcore archive ` | Delete a job on the service + clear local history | | `agentcore stop ` | Stop a running batch-evaluation or ab-test | From 0d3363c279b0b0ca1de5d2160ba29eb8fe71b446 Mon Sep 17 00:00:00 2001 From: notgitika Date: Thu, 18 Jun 2026 15:34:11 -0400 Subject: [PATCH 5/5] docs: revert Payments [preview] tag (it ships in GA too) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per maintainer guidance, Payments is available in both GA and preview releases — only Harness is preview-only. The [preview] prefix in the PaymentManager/PaymentConnector source descriptions is just a label hint for the help text, not a build-gating indicator. Drop the tag and the install hint from the README's Payments section. --- README.md | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/README.md b/README.md index 774660739..1ec84d801 100644 --- a/README.md +++ b/README.md @@ -198,13 +198,7 @@ Policy engines apply Cedar-based pre/post-call policies to agent invocations — | `add policy-engine` | Add a Cedar policy engine to the project | | `add policy` | Add a policy to a policy engine (form-based guardrails or raw Cedar) | -### Payments — `[preview]` - -> Payment commands are only available in the preview release of the CLI. Install it with: -> -> ```bash -> npm install -g @aws/agentcore@preview -> ``` +### Payments Pay-per-call agent transactions via the [x402 protocol](https://www.x402.org/). When a tool call returns `402 Payment Required`, the payments system signs and submits payment then retries automatically.