chore: re-gate web search#1625
Conversation
|
Claude Security Review: no high-confidence findings. (run) |
Package TarballHow to installgh release download pr-1625-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.20.2.tgz |
agentcore-cli-automation
left a comment
There was a problem hiding this comment.
Thanks for re-gating this. I have two concerns worth addressing before merge — the new AddScreen test doesn't actually exercise the gating logic it claims to, and the top-level add web-search / remove web-search subcommands ignore --json on the gating error path. Details inline.
|
|
||
| expect(lastFrame()).toContain('Web Search'); | ||
| expect(lastFrame()).toContain('Coming soon'); | ||
| }); |
There was a problem hiding this comment.
This test does not actually exercise the gating logic for web-search.
ADD_RESOURCE_ITEMS in AddScreen.tsx is computed at module import time, so its disabled/description values are frozen based on whatever ENABLE_GATED_FEATURES was at the moment the test file imported AddScreen. The beforeEach setting process.env.ENABLE_GATED_FEATURES = '1' (and this test's delete) have no effect on the rendered output — the constants are already baked in.
In the default vitest env, ENABLE_GATED_FEATURES is unset, so this test passes coincidentally. It would still pass even if the source change in AddScreen.tsx were reverted, since the only thing being asserted is that "Coming soon" appears anywhere in the frame, and knowledge-base already produces that text on the existing module-load gating path. Symmetrically, the existing beforeEach(... = '1') is also a no-op for tests 1 and 2.
A few options:
- Use
vi.resetModules()+ dynamicimport('../AddScreen.js')inside each test after setting/unsettingENABLE_GATED_FEATURES, so the module re-evaluates with the desired env. This is the pattern used insrc/lib/utils/__tests__/platform.test.ts. - Move the gating computation inside the
AddScreencomponent (e.g.,useMemoon render, likeAddGatewayTargetScreendoes) so the env is read per render and tests can flip the env between renders. - To meaningfully assert the gating for web-search specifically, also assert that the
knowledge-baserow is not showing "Coming soon" whenENABLE_GATED_FEATURES=1, and that both rows show "Coming soon" when it's unset — so the test would fail if gating regressed.
Without one of these, the test gives a false sense of coverage.
| if (!isGatedFeaturesEnabled()) { | ||
| console.error('Error: Web search target type is not yet available.'); | ||
| process.exit(1); | ||
| } |
There was a problem hiding this comment.
When --json is passed and the feature is gated, this writes a plain-text error to stderr and exits 1, breaking the JSON output contract for the subcommand. A caller piping agentcore add web-search --json will see no JSON on stdout and a human-readable line on stderr — different from every other failure mode of this command (e.g. validation errors at L961 / L877-878 still emit {success: false, error: ...} JSON).
Same issue at L996-998 for remove web-search.
Options:
- Mirror the existing error pattern: if
cliOptions.json,console.log(JSON.stringify({ success: false, error: 'Web search target type is not yet available.' }))beforeprocess.exit(1); otherwiseconsole.error(...). - Move the gate check inside
runCliCommandbythrow new ValidationError(...)(like the--type webSearchpath at L599 does forgateway-target), which handles JSON formatting uniformly. You'd need to move the gate check belowrunCliCommand('add.web-search', ...)and similarly forremove.web-search.
Option 2 also has the side benefit of routing the gating rejection through the existing telemetry/error-classification path for these commands.
| .option('--json', 'Output as JSON [non-interactive]') | ||
| .action(async (cliOptions: { name?: string; gateway?: string; excludeDomains?: string; json?: boolean }) => { | ||
| if (!isGatedFeaturesEnabled()) { | ||
| console.error('Error: Web search target type is not yet available.'); |
There was a problem hiding this comment.
Minor consistency: this error is prefixed with Error: while the corresponding gate at L996 (remove) and the inner ValidationError at L599 (gateway-target) are not. Pick one style. (Non-blocking — feel free to defer if you address the JSON-output issue below.)
Coverage Report
|
Description
Re-gate web search after changes to devEx
Related Issue
Closes #
Documentation PR
Type of Change
Testing
How have you tested the change?
npm run test:unitandnpm run test:integnpm run typechecknpm run lintsrc/assets/, I rannpm run test:update-snapshotsand committed the updated snapshotsChecklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.