Align workshop with memory-first recommendations#56
Open
markjbrown wants to merge 3 commits into
Open
Conversation
Improve Explore city handling, add themed place-search fallback, and make recommendation specialist nodes use deterministic memory-first retrieval with taxonomy-based preference alignment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update the completed recommendation path and workshop progression so deterministic specialist nodes check memories first without requiring extra specialist LLM/tool loops. Refresh evaluation expectations and observability guidance to match the final architecture. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR updates the workshop’s “completed” and “exercise” paths to a memory-first architecture: the orchestrator still uses LLM tools for preference extraction and routing, while hotel/activity/dining recommendation turns become deterministic nodes that load memories and query places directly (avoiding an extra specialist LLM loop). It also aligns workshop docs/datasets/tests and adds a theme-search fallback in the filtering API.
Changes:
- Replaced LLM-backed Hotel/Activity/Dining ReAct agents with deterministic memory-first recommendation nodes and added deterministic routing for obvious domain requests.
- Updated MCP preference-extraction tool to skip LLM calls for plain search requests (when no explicit preference statement is present).
- Aligned workshop modules, evaluation datasets/evaluators, and frontend tests with the updated flows; added theme-search fallback to filtered search.
Reviewed changes
Copilot reviewed 31 out of 34 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| 02_completed/python/src/app/travel_agents.py | Implements deterministic memory-first specialist nodes and deterministic routing; updates graph edges to return to human. |
| 02_completed/python/src/app/travel_agents_api.py | Adds fallback to filtered search when theme search returns no results. |
| 02_completed/python/src/app/prompts/orchestrator.prompty | Updates routing guidance so “what preferences are saved?” uses recall_memories directly. |
| 02_completed/python/src/app/prompts/hotel_agent.prompty | Deprecates the old specialist prompt (kept for workshop history). |
| 02_completed/python/src/app/prompts/dining_agent.prompty | Deprecates the old specialist prompt (kept for workshop history). |
| 02_completed/python/src/app/prompts/activity_agent.prompty | Deprecates the old specialist prompt (kept for workshop history). |
| 02_completed/mcp_server/mcp_http_server.py | Adds heuristic to skip preference-extraction LLM for plain searches; updates discover_places comments. |
| 02_completed/frontend/src/app/components/shared/place-card/place-card.component.spec.ts | Updates tests to match new price-tier display/icon behavior and removed action buttons. |
| 02_completed/frontend/src/app/components/shared/message/message.component.spec.ts | Updates tests to match new avatar rendering and styling selectors. |
| 02_completed/frontend/src/app/components/profile/profile.component.spec.ts | Updates memory model expectations in tests (text/facets/salience/etc.). |
| 02_completed/frontend/src/app/components/login/login.component.spec.ts | Updates tests for new login method, navigation target, and error messages. |
| 02_completed/frontend/src/app/components/home/home.component.spec.ts | Updates tests to incorporate city loading and query param navigation. |
| 02_completed/frontend/src/app/components/explore/explore.component.ts | Fixes city selection/start trip logic and avoids redundant reloads. |
| 02_completed/frontend/src/app/components/explore/explore.component.spec.ts | Updates tests for new city/thread flows and filter API usage. |
| 02_completed/frontend/src/app/app.component.spec.ts | Updates tests for navigation screens and safer router.url stubbing. |
| 02_completed/frontend/package-lock.json | Lockfile updates (adds nested chokidar/readdirp entries). |
| 01_exercises/workshop/Module-06.md | Updates evaluation narrative to reflect deterministic specialist paths and “no tool calls expected” cases. |
| 01_exercises/workshop/Module-05.md | Updates observability guidance for deterministic specialist nodes. |
| 01_exercises/workshop/Module-04.md | Updates module instructions to implement the new memory-first architecture and extraction skipping. |
| 01_exercises/workshop/Module-02.md | Updates routing helpers and specialist-edge wiring to match new routing approach. |
| 01_exercises/python/src/app/travel_agents_api.py | Adds theme-search fallback to filtered search, mirroring completed path. |
| 01_exercises/frontend/tsconfig.spec.json | Adds spec tsconfig for Jasmine typing/compilation. |
| 01_exercises/frontend/src/app/components/shared/place-card/place-card.component.spec.ts | Mirrors completed frontend test updates. |
| 01_exercises/frontend/src/app/components/shared/message/message.component.spec.ts | Mirrors completed frontend test updates. |
| 01_exercises/frontend/src/app/components/profile/profile.component.spec.ts | Mirrors completed frontend memory model test updates. |
| 01_exercises/frontend/src/app/components/login/login.component.spec.ts | Mirrors completed frontend login test updates. |
| 01_exercises/frontend/src/app/components/home/home.component.spec.ts | Mirrors completed frontend home test updates. |
| 01_exercises/frontend/src/app/components/explore/explore.component.ts | Mirrors completed frontend explore fixes. |
| 01_exercises/frontend/src/app/components/explore/explore.component.spec.ts | Mirrors completed frontend explore test updates. |
| 01_exercises/frontend/src/app/app.component.spec.ts | Mirrors completed frontend app component test updates. |
| 01_exercises/frontend/package-lock.json | Lockfile updates (removes license fields for some deps). |
| 01_exercises/evaluation/evaluators/heuristic_evaluators.py | Updates tool accuracy scoring when no tools are expected. |
| 01_exercises/evaluation/datasets/tool_usage_dataset.json | Updates required tools to reflect deterministic specialist paths. |
| 01_exercises/evaluation/datasets/e2e_dataset.json | Updates expected tools/agents and names to reflect new flows. |
Files not reviewed (2)
- 01_exercises/frontend/package-lock.json: Language not supported
- 02_completed/frontend/package-lock.json: Language not supported
Comments suppressed due to low confidence (1)
02_completed/python/src/app/travel_agents.py:478
- Same issue as the earlier geo-scope branch: this inserts the existing
HumanMessageinto the update, which will duplicate the user turn inMessagesState(messages are appended, not replaced). Return only the newAIMessage(and any tool messages if you add them later).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+676
to
+679
| deterministic_destination = _deterministic_specialist_route(_latest_user_text(state)) | ||
| if deterministic_destination: | ||
| logger.info(f"🎯 Orchestrator deterministically routing to {deterministic_destination}") | ||
| return Command(update={}, goto=deterministic_destination) |
Comment on lines
+412
to
+417
| response_text = "Which city should I use for these recommendations?" | ||
| latest_user_message = _latest_user_message(state) | ||
| messages = [AIMessage(content=response_text, name=agent_label)] | ||
| if latest_user_message: | ||
| messages.insert(0, latest_user_message) | ||
| return {"messages": messages} |
Comment on lines
+461
to
+468
| if not places: | ||
| places = query_places_filtered( | ||
| geo_scope_id=geo_scope, | ||
| place_type=place_type, | ||
| dietary=filters.get("dietary"), | ||
| accessibility=filters.get("accessibility"), | ||
| price_tier=None, | ||
| )[:10] |
Comment on lines
918
to
+920
| if activeAgent in [None, "unknown"]: | ||
| logger.info(f"� activeAgent is '{activeAgent}', defaulting to Orchestrator") | ||
| activeAgent = "Orchestrator" | ||
| activeAgent = "orchestrator" |
| ## What's Next? | ||
|
|
||
| Proceed to Module 06: **[Evaluating Your Multi-Agent Application](./Module-05.md)** No newline at end of file | ||
| Proceed to Module 06: **[Evaluating Your Multi-Agent Application](./Module-05.md)** |
|
|
||
| _EXPLICIT_PREFERENCE_PATTERNS = [ | ||
| r"\b(i|we)\s+(am|are|have|need|prefer|like|love|hate|avoid|require|want)\b", | ||
| r"\b(i'm|im|we're|were)\b", |
Preserve preference capture before deterministic routing, avoid duplicate user messages in specialist responses, make fallback place search async and guarded, and fix stale workshop documentation details. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 31 out of 34 changed files in this pull request and generated 1 comment.
Files not reviewed (2)
- 01_exercises/frontend/package-lock.json: Language not supported
- 02_completed/frontend/package-lock.json: Language not supported
Comment on lines
+371
to
+376
| if memory_error: | ||
| intro = f"I tried to check your saved preferences first, but that lookup failed ({memory_error}). I still found these {noun} in {city}:" | ||
| elif memories: | ||
| intro = f"I checked your saved preferences first and found {len(memories)} relevant preference(s). Based on those, here are good {noun} in {city}:" | ||
| else: | ||
| intro = f"I checked your saved preferences first and did not find anything relevant, so I used general quality signals for {city}:" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Validation