docs(skills): use live Peec MCP in aeo_new_guide_recommendations - #470
docs(skills): use live Peec MCP in aeo_new_guide_recommendations#470rachaelrenk wants to merge 6 commits into
Conversation
The scheduled cloud agent can authenticate with Peec via the PEEC_PAT secret and the peec-ai MCP server, as aeo_crosslink_audit already does. Replace the committed-snapshot workflow and its 14-day staleness gate with live Peec MCP calls, and switch the documented cadence to monthly. - Add PEEC_PAT and the peec-ai mcp_servers config to environment requirements - Replace the snapshot source-data section and freshness gate with live MCP - Replace the stale-snapshot report with a "Peec unavailable" degrade path - Update run log, run output, and Slack formats to drop snapshot states - Mark refresh-peec-aeo-snapshot as legacy for this pipeline - Update skill-authoring-guidelines to prefer token-authenticated live calls over snapshots, since the old guidance taught the reverse Co-Authored-By: Warp Agent <agent@warp.dev>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
I'm starting a first review of this pull request. You can view the conversation on Warp. I completed the review and no human review was requested for this pull request. Comment Powered by Oz |
There was a problem hiding this comment.
Overview
This PR updates the AEO new-guide recommendation skill and related guidance to use live Peec MCP data with a documented unavailable path instead of a committed snapshot freshness gate.
Concerns
- The new Peec-unavailable degraded mode conflicts with the unchanged brief quality gate requiring every recommendation to be backed by a Peec signal, so docs-only low-confidence briefs cannot actually pass the skill's own final checks.
Verdict
Found: 0 critical, 1 important, 0 suggestions
Request changes
Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).
Powered by Oz
| Before finalizing each brief: | ||
|
|
||
| - **Real signal** — Each recommendation is backed by a Peec snapshot signal, not generic SEO advice. | ||
| - **Real signal** — Each recommendation is backed by a Peec signal, not generic SEO advice. |
There was a problem hiding this comment.
Validation run 019fce3f-6517-754e-8e97-7af8531e5d62 executed the updated skill end to end and surfaced concrete gaps. - Document the Peec MCP call contract: cloud runs may not surface peec-ai as a native tool, so call the JSON-RPC endpoint directly; resolve project_id via list_projects; get_actions drill-downs require url_classification; list_search_queries returns query_text - Tell the agent to treat OWNED how-to/article rows as the docs signal, since many EDITORIAL rows are outreach rather than content gaps - Add channel_not_found handling for Slack: attempt a lookup by channel name, then fall back to run output and report the failure explicitly - Replace oz-dev with oz for run-link resolution. oz-dev is a local dev build and is absent from cloud sandboxes, so build and is absent from cloud sandboxes, so build and is absent from cloud sandboxes, so build and is absent from cloud sandboxes, so build and is hored-By: Warp Agent <agent@warp.dev>
oz-dev is a local development build shipped in the Warp dev app; cloud sandboxes only have oz. Every scheduled skill calling oz-dev to resolve its session link hit command-not-found and silently omitted the run link from its Slack notification. - weekly-404-monitor (run link + the schedule create deploy command) - improve-drafting-skills (conversation fetch + run link) - improve-aeo-crosslink-skill - improve-404-monitor-skill Also add the channel_not_found recovery path to aeo_crosslink_audit, which shares the GROWTH_DOCS_SLACK_CHANNEL_ID secret and has been failing to deliver its #growth-docs notifications for the same reason. Co-Authored-By: Warp Agent <agent@warp.dev>
SLACK_BOT_TOKEN authenticates as the hubble bot, which is not a member of #growth-docs and cannot be invited. buzz is the account that posts to that channel, so both AEO skills now use BUZZ_SLACK_TOKEN. Name the expected bot in each skill's environment requirements, and add guidance to the authoring reference that Slack tokens authenticate as different bots with different channel memberships, so a token that authenticates is not necessarily a token that can deliver. Co-Authored-By: Warp Agent <agent@warp.dev>
Summary
aeo_new_guide_recommendationswas written when cloud agents could not authenticate with Peec, so it read a snapshot committed to thebuzzrepo and hard-exited when that snapshot was 14 days or older. That constraint no longer holds:aeo_crosslink_auditalready calls Peec MCP directly using thePEEC_PATsecret, and its 2026-08-03 run logged "Peec available" and opened #453 while the committed snapshot was 40 days old.This updates the new-guide skill to the same live-MCP pattern and switches its documented cadence from every two weeks to monthly.
Why this matters
The snapshot in
buzzwas last generated 2026-06-24. Scheduling the new-guide agent as written would have produced a stale-snapshot exit on every run, since nothing refreshes the snapshot on a cadence and the 14-day gate is shorter than a monthly interval.Changes
.agents/skills/aeo_new_guide_recommendations/SKILL.mdPEEC_PATand thepeec-aimcp_serversconfig block to environment requirements, mirroringaeo_crosslink_audit.Peec [available | unavailable].refresh-peec-aeo-snapshotas legacy for this pipeline..agents/references/skill-authoring-guidelines.mdmcp_serversconfig, define an explicit unavailable path, and treat snapshots as a last resort that needs a paired refresh schedule..agents/logs/aeo_new_guide_recommendation_runs.mdFollow-up (not in this PR)
The schedule for this skill still needs to be created with the
peec-aiMCP server wired toPEEC_PAT, and pointed directly at the skill spec rather than the Docs Agent profile (which would attach the unrelatedrelease_updatesskill).Open questions for human review
aeo_crosslink_auditand this skill share the Peec query scope, or is per-skill divergence fine?Update: findings from the validation run
Run 019fce3f executed the updated skill end to end. Peec MCP authenticated successfully with
PEEC_PAT(HTTP 200,list_projectsreturned the Warp project) and produced 4 briefs, confirming the live-MCP approach. It also surfaced three real defects, now fixed here.Peec MCP call contract was under-specified. The agent had no native
peec-aitool despite the MCP being in the agent config, so it called the JSON-RPC endpoint directly. The skill now documents that path plus the contract details it had to discover:list_projectsmust run first forproject_id,get_actionsdrill-downs requireurl_classification, andlist_search_queriesreturnsquery_text. It also notes that mostEDITORIALaction rows are outreach pitches rather than docs gaps, soOWNEDhow-to/article rows are the real signal.oz-devdoes not exist in cloud sandboxes. It ships only in the local Warp dev app. Six occurrences across seven skill files instructed agents to call it to resolve their session link, so every scheduled agent hit command-not-found and silently dropped the run link from its Slack notification. Swept toozacrossweekly-404-monitor,improve-drafting-skills,improve-aeo-crosslink-skill, andimprove-404-monitor-skill, plus both AEO skills and the authoring guidelines.Slack posting fails with
channel_not_found.SLACK_BOT_TOKENauthenticates, but the storedGROWTH_DOCS_SLACK_CHANNEL_IDdoes not resolve to a channel either bot can see. Both AEO skills now attempt a channel-name lookup, then fall back to run output and report the failure explicitly instead of logging a clean run. This affectsaeo_crosslink_audittoo — its weekly#growth-docsnotifications have not been landing.Still needs a human
The
GROWTH_DOCS_SLACK_CHANNEL_IDsecret value is wrong and must be corrected outside this PR. Slack delivery remains unvalidated until then.