test(e2e): fix shard 4/5 stale paths and missing --wait#1560
Conversation
Three failures on the full-suite run for main, all stale assumptions the production code already moved past: - archive-lifecycle.test.ts: 'local .cli/batch-eval-results contains the run record' (line 154) and 'local .cli/recommendations contains the run record' (line 199) check filesystem paths under '.cli/<dir>/<id>.json'. The jobs engine refactor (aws#221) moved record storage under '.cli/jobs/<dir>/<id>.json' (see src/cli/operations/jobs/shared/storage.ts:28). Add the missing 'jobs' segment in all four occurrences (capture + post-archive deletion checks; the post-archive checks were silently passing-by-omission because they assert non-existence at a path that never existed). - config-bundle-eval-rec.test.ts: 'runs system prompt recommendation with inline content via CLI' (line 478) is the only recommendation test that asserts on json.result. Without --wait the CLI returns immediately after StartRecommendation while the job is still PENDING/IN_PROGRESS; record.result is only populated by refresh() once status reaches COMPLETED. Add --wait to that test so it blocks to a terminal state before reading result.
github-actions
Bot
added
the
agentcore-harness-reviewing
agentcore-devx-automation
Bot
added
the
claude-security-reviewing
|
Claude Security Review: no high-confidence findings. (run) |
Package TarballHow to installgh release download pr-1560-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.19.0.tgz |
agentcore-devx-automation
Bot
removed
the
claude-security-reviewing
|
should close #1553 will verify post merge |
agentcore-cli-automation
left a comment
agentcore-cli-automation
left a comment
There was a problem hiding this comment.
LGTM. Verified the fixes against the codebase:
Path migration (archive-lifecycle.test.ts): src/cli/operations/jobs/shared/storage.ts writes to <configRoot>/.cli/jobs/{STORAGE_DIRS[type]}/{id}.json, and STORAGE_DIRS in shared/constants.ts maps 'recommendation' → 'recommendations' and 'batch-evaluation' → 'batch-eval-results'. The four updated paths in the test match. Good catch on the two post-archive deletion checks at lines 240/311 — those were trivially passing before since they assert the file doesn't exist at the (wrong) path.
--wait for the inline-content recommendation test (config-bundle-eval-rec.test.ts:494): Confirmed record.result is only populated by refresh() (handler.ts:293) and --wait (command.tsx:620–627) routes through waitForTerminal which polls until isTerminal(record). Without --wait the synchronous post-StartRecommendation path returns before refresh ever runs, so result is necessarily undefined. The other three recommendation tests in the file don't read json.result, so leaving them without --wait is fine.
Test-only change; no telemetry or mocking concerns. No new issues.
github-actions
Bot
removed
the
agentcore-harness-reviewing
Hweinstock
left a comment
Hweinstock
left a comment
There was a problem hiding this comment.
nice! (will merge in post-release)
5 participants
there are 3 failures in e2e shard 4. they are based on stale assumptions.
2 of them have the same root cause: path migration to a common jobs/ dir. those tests are looking at the wrong place for the run results file.
the other one is expecting json.result to be defined but since that was made async recently, the user has to pass --wait, otherwise it works async.
this PR aims to fix all of those 3 tests.
note: no CLI code is being touched here, only text fixes