From 8679e2d5686d0a175536ca7be52bca1e73f1b10d Mon Sep 17 00:00:00 2001 From: Rachael Rose Renk <91027132+rachaelrenk@users.noreply.github.com> Date: Tue, 4 Aug 2026 13:10:33 -0600 Subject: [PATCH 1/4] docs(skills): use live Peec MCP in aeo_new_guide_recommendations The scheduled cloud agent can authenticate with Peec via the PEEC_PAT secret and the peec-ai MCP server, as aeo_crosslink_audit already does. Replace the committed-snapshot workflow and its 14-day staleness gate with live Peec MCP calls, and switch the documented cadence to monthly. - Add PEEC_PAT and the peec-ai mcp_servers config to environment requirements - Replace the snapshot source-data section and freshness gate with live MCP - Replace the stale-snapshot report with a "Peec unavailable" degrade path - Update run log, run output, and Slack formats to drop snapshot states - Mark refresh-peec-aeo-snapshot as legacy for this pipeline - Update skill-authoring-guidelines to prefer token-authenticated live calls over snapshots, since the old guidance taught the reverse Co-Authored-By: Warp Agent --- .../logs/aeo_new_guide_recommendation_runs.md | 2 +- .../references/skill-authoring-guidelines.md | 46 ++++---- .../aeo_new_guide_recommendations/SKILL.md | 103 +++++++----------- 3 files changed, 67 insertions(+), 84 deletions(-) diff --git a/.agents/logs/aeo_new_guide_recommendation_runs.md b/.agents/logs/aeo_new_guide_recommendation_runs.md index 6d74e665..8bb35d34 100644 --- a/.agents/logs/aeo_new_guide_recommendation_runs.md +++ b/.agents/logs/aeo_new_guide_recommendation_runs.md @@ -2,7 +2,7 @@ New entries are prepended by each scheduled agent run. Most recent entry first. -This log tracks every bi-weekly run of the `aeo_new_guide_recommendations` skill — both runs that produced briefs and runs that wrote a no-brief or stale-snapshot report — so the team can detect recurring topic gaps and answer questions like "has this topic been flagged before?" without replaying individual Oz runs. +This log tracks every monthly run of the `aeo_new_guide_recommendations` skill — both runs that produced briefs and runs that wrote a no-brief report — so the team can detect recurring topic gaps and answer questions like "has this topic been flagged before?" without replaying individual Oz runs. **Format**: see the `## Run log format` section in `.agents/skills/aeo_new_guide_recommendations/SKILL.md`. diff --git a/.agents/references/skill-authoring-guidelines.md b/.agents/references/skill-authoring-guidelines.md index eab3fbcf..707ae4f9 100644 --- a/.agents/references/skill-authoring-guidelines.md +++ b/.agents/references/skill-authoring-guidelines.md @@ -45,33 +45,37 @@ Agents often proceed past a failed file write without noticing. For any log upda - After appending: `tail -5 ` and confirm the new entry appears. - After push: `git log --oneline -1 origin/` and confirm the commit SHA matches the expected commit. -### Source data and freshness - -**Cloud agents cannot call Peec MCP directly.** Peec requires OAuth authentication, which is not available in cloud agent environments. Any skill that needs Peec data must read from a pre-exported snapshot committed to the `buzz` repo. - -If your skill uses external data (Peec, GSC, or any authenticated API) that is unavailable in cloud agents: - -1. Build the data export into a separate **local-only skill** (e.g., `refresh-peec-aeo-snapshot`). -2. The cloud skill reads the committed snapshot, not the live API. -3. Add an explicit **freshness gate**: define a maximum age (e.g., 14 days), check `generated_at`, and exit with a stale-snapshot report if the threshold is exceeded. Never proceed with stale data. -4. Document the freshness constraint clearly at the top of the `## Source data` section: explain why a snapshot is used instead of a live call, so future editors don't remove the constraint thinking it is overly cautious. +### Source data from authenticated APIs + +**Prefer live API calls with a token over pre-exported snapshots.** Cloud agents can authenticate with services that issue long-lived tokens. Peec, for example, is reached through its MCP server using a Personal Access Token stored as the `PEEC_PAT` Oz secret — cloud agents do not need OAuth, and they do not need a committed snapshot. + +When your skill needs data from an authenticated API: + +1. Store the credential as an Oz secret and reference it by name in the skill's `## Environment requirements` section. Never inline it. +2. For MCP-based sources, document the `mcp_servers` config the scheduled agent needs, so whoever creates the schedule knows the skill will not work without it: + ```json + { + "peec-ai": { + "url": "https://api.peec.ai/mcp", + "headers": { + "Authorization": "Bearer ${PEEC_PAT}" + } + } + } + ``` +3. Define an explicit **unavailable path**: what the skill does when the token is missing or expired, the MCP server is not configured, or the call fails. Log the specific failure (never the token value), degrade to the remaining signals, and raise the confidence bar for any output produced without the primary signal. +4. Record availability in the run log (for example, `Source signals: Peec [available | unavailable]`) so the outer loop can distinguish a low-signal period from a broken credential. -Freshness gate pattern: -- Read `generated_at` from the snapshot metadata file. -- If the file is missing, `generated_at` is absent, or the age exceeds the threshold: - - Write a stale-snapshot report with the exact age (or error reason). - - Write a run log entry with a `No-run reason` of `snapshot stale — N days old`. - - Post a Slack alert. - - Exit. Do not proceed or open a PR. +**Use a committed snapshot only as a last resort** — when a source genuinely cannot be authenticated from a cloud agent. Snapshots introduce a freshness gate, a manual local refresh step, and a failure mode where the agent exits without doing work because nobody refreshed the data. If you do use one, define a maximum age, check it before use, and pair the skill with a scheduled refresh so the gate cannot silently starve the pipeline. ### Scope consistency When you add a new topic area to a skill's scope, audit every section — especially `## Source data` — to confirm the source data actually covers the new topic. A common mistake: a skill lists four topic areas but the source data description names only three. The agent then produces lower-quality briefs for the fourth topic with no signal, or invents signals. Checklist when expanding scope: -- Does the snapshot include data for the new topic? If not, update the snapshot refresh skill, or document the lower confidence explicitly. +- Does the source data include the new topic? If not, extend the tracked queries or prompts at the source, or document the lower confidence explicitly. - Are all quality gates still valid for the new topic? (e.g., minimum brief count thresholds) -- Does the stale-snapshot report reflect the full scope? +- Do the no-action and unavailable-signal reports reflect the full scope? ### Scope contradictions in "Do not" lists @@ -107,7 +111,7 @@ Always use `SLACK_BOT_TOKEN` and other secrets from environment variables — ne ### Slack notifications -Post a Slack notification on every run, including no-action runs and stale-snapshot exits. A missing notification on a no-action run is indistinguishable from a run that silently failed. Use a simple text message (not Block Kit) that can be scanned in under 30 seconds. +Post a Slack notification on every run, including no-action runs and runs that exited early because a source signal was unavailable. A missing notification on a no-action run is indistinguishable from a run that silently failed. Use a simple text message (not Block Kit) that can be scanned in under 30 seconds. --- @@ -117,7 +121,7 @@ Outer loop skills run less frequently (typically monthly) and read the inner loo ### Data minimum before the outer loop can run -The outer loop needs enough run log entries to identify real patterns, not noise. Require a minimum entry count before acting (the `improve-aeo-crosslink-skill` uses 8 entries ≈ 2 months; `improve-aeo-new-guide-rec-skill` should start after ~4 entries ≈ 6–8 weeks). If the minimum is not met, write a "too early to analyze" notice to run output and skip the PR. +The outer loop needs enough run log entries to identify real patterns, not noise. Require a minimum entry count before acting (the `improve-aeo-crosslink-skill` uses 8 entries ≈ 2 months at a weekly cadence; `improve-aeo-new-guide-rec-skill` should start after ~4 entries ≈ 4 months at a monthly cadence). If the minimum is not met, write a "too early to analyze" notice to run output and skip the PR. This minimum must be stated explicitly in the skill's `## Schedule` section so the deployer knows when to start the agent. diff --git a/.agents/skills/aeo_new_guide_recommendations/SKILL.md b/.agents/skills/aeo_new_guide_recommendations/SKILL.md index 81beeaaf..7c2c87ac 100644 --- a/.agents/skills/aeo_new_guide_recommendations/SKILL.md +++ b/.agents/skills/aeo_new_guide_recommendations/SKILL.md @@ -1,11 +1,11 @@ --- name: aeo_new_guide_recommendations -description: Run a recurring AEO new-guide recommendations brief for Warp docs using Peec data and existing docs. Use for scheduled agents that run every two weeks and identify missing or underserved topics in agents, cloud agents, orchestration, and Oz — and recommend whether to create a new page, update an existing page, or avoid a topic. Does not draft content. +description: Run a recurring AEO new-guide recommendations brief for Warp docs using Peec data and existing docs. Use for scheduled agents that run monthly and identify missing or underserved topics in agents, cloud agents, orchestration, and Oz — and recommend whether to create a new page, update an existing page, or avoid a topic. Does not draft content. --- # AEO new-guide recommendations -Produce 3–5 source-grounded AEO briefs that recommend whether to create a new guide, update an existing page, or avoid a topic — without drafting content. This skill is designed for an Oz scheduled agent that runs every two weeks (not twice a week — "bi-weekly" is intentionally avoided here because it is ambiguous). The goal is to give the Docs team a repeatable upstream input to content planning without generating more noise than the team can act on. +Produce 3–5 source-grounded AEO briefs that recommend whether to create a new guide, update an existing page, or avoid a topic — without drafting content. This skill is designed for an Oz scheduled agent that runs monthly. The goal is to give the Docs team a repeatable upstream input to content planning without generating more noise than the team can act on. ## Scope @@ -28,33 +28,42 @@ Do not: The following environment secrets should be set in the Oz cloud agent environment: +- `PEEC_PAT` — Peec Personal Access Token for MCP authentication. Create one at **app.peec.ai → Company → API Keys → Personal Access Tokens**. If unavailable or expired, the run proceeds with docs-only signals and logs "Peec: unavailable" in the run output. - `SLACK_BOT_TOKEN` — Slack bot token for posting to `#growth-docs`. If unavailable, write the notification body to the run output instead and skip Slack posting. - `GROWTH_DOCS_SLACK_CHANNEL_ID` — Slack channel ID for `#growth-docs`. Find it in Slack by right-clicking the channel → Copy link (the ID begins with `C`). If unavailable, skip Slack posting. +The scheduled cloud agent must also include the Peec MCP server in its agent config (pass via `--mcp` flag or the agent config file `mcp_servers` key): + +```json +{ + "peec-ai": { + "url": "https://api.peec.ai/mcp", + "headers": { + "Authorization": "Bearer ${PEEC_PAT}" + } + } +} +``` + Do NOT print, log, commit, or include secret values in reports or Slack messages. ## Source data Use the smallest reliable set of source data needed to justify recommendations. -**Why a snapshot, not live Peec MCP calls:** This skill runs as an Oz cloud agent, which cannot authenticate with Peec MCP directly. All Peec data must come from a pre-exported snapshot committed to the `buzz` repo. The snapshot is generated locally (where Peec OAuth works) using the `refresh-peec-aeo-snapshot` skill, then committed so cloud runs can read it. If the snapshot is stale, this run exits rather than proceeding without data. - -- **Peec snapshot** - Check freshness before using any data: - 1. Read `generated_at` from `/workspace/buzz/aeo-snapshots/docs/agents-orchestration/latest.json`. - 2. If the file is missing, `generated_at` is absent, or the snapshot is **14 days old or older**: write the stale-snapshot report (see "Stale snapshot report" below), write a run log entry (step 7) with the appropriate `No-run reason` (see "Stale snapshot report" for exact wording), post the stale Slack alert (step 8), and exit. Do not continue or open a PR. - 3. If the snapshot is fewer than 14 days old, read both `latest.json` and `latest.md` as source signals. These contain pre-exported Peec data (prompts, recommendations, source URLs, query vocabulary, and visibility scores) for agents, cloud agents, and orchestration. Because Oz is the agent platform underlying cloud agents and orchestration, substantial Oz-relevant signal is present in this snapshot — look for Oz-related prompts and queries within the agents/orchestration data. However, dedicated Oz-surface signals (Oz web app, Oz CLI, Oz scheduling) may be limited; when Oz-specific coverage is thin, flag the brief as lower-confidence and note what additional signal would strengthen the recommendation. Use the snapshot as the primary Peec source — do not attempt to call Peec MCP directly (cloud agents cannot authenticate). +- **Peec** - Use the Peec MCP (configured in the agent with a Personal Access Token via the `PEEC_PAT` secret) to collect prompts, search queries, actions/recommendations, and source URLs for agents, cloud agents, orchestration, and Oz (last 30 days). Filter prompts and queries for relevance to the topic areas. Because Oz is the agent platform underlying cloud agents and orchestration, substantial Oz-relevant signal appears within the agents and orchestration data — look for Oz-related prompts and queries there. Dedicated Oz-surface signals (Oz web app, Oz CLI, Oz scheduling) may still be thin; when Oz-specific coverage is sparse, flag the brief as lower-confidence and note what additional signal would strengthen the recommendation. If the Peec MCP returns an error or is unavailable (missing `PEEC_PAT`, expired token, or connection failure), log "Peec: unavailable" in the run output and follow the "Peec unavailable" section below. - **Docs repo** - Search existing pages under `src/content/docs/` for relevant coverage of each candidate topic. Read `AGENTS.md` and `.agents/references/terminology.md` for product naming guidance. - **Prior run log** - Read `.agents/logs/aeo_new_guide_recommendation_runs.md` to identify topics that were recommended in previous runs. If a candidate topic from this run matches a topic from a prior run, note it explicitly in the brief (see "Repeat topic flag" below). -Do not invent Peec signals. If the snapshot has no usable data for a candidate topic, say so in the brief and flag it as low-confidence. +Do not invent Peec signals. If Peec has no usable data for a candidate topic, say so in the brief and flag it as low-confidence. ## Workflow -1. **Check snapshot freshness.** Read `generated_at` from `latest.json`. If the snapshot is missing or 14 days old or older, write the stale-snapshot report, write a run log entry (step 7), post the stale Slack alert (step 8), and exit — do not proceed further. +1. **Collect Peec signals.** Use the Peec MCP to collect prompts, search queries, actions/recommendations, and source URLs for agents, cloud agents, orchestration, and Oz (last 30 days). If the Peec MCP is unavailable, log "Peec: unavailable" and follow the "Peec unavailable" section. 2. **Read the prior run log.** Open `.agents/logs/aeo_new_guide_recommendation_runs.md` and extract the `Topics` field from each previous entry. Build a list of previously recommended topic slugs so you can detect repeats in step 4. -3. **Gather source signals from the Peec snapshot.** From `latest.json` and `latest.md`, extract: +3. **Extract and cluster the signals.** From the Peec results, pull out: - Top prompts and their relevance to agents, cloud agents, orchestration, and Oz. - Search-query clusters — group repeated phrasing; do not list every raw query. - Action recommendations — use the recommendation text as the source of truth. Focus on owned and editorial opportunities relevant to docs content. @@ -138,7 +147,7 @@ Do not invent Peec signals. If the snapshot has no usable data for a candidate t Before finalizing each brief: -- **Real signal** — Each recommendation is backed by a Peec snapshot signal, not generic SEO advice. +- **Real signal** — Each recommendation is backed by a Peec signal, not generic SEO advice. - **Actionable scope** — A reviewer can act on the recommendation in a 15–30 minute planning session without re-doing the research. - **No drafting** — The brief recommends what to create or update; it does not draft any docs content. - **Distinct from crosslinks** — Do not recommend adding a link as a substitute for a new or updated page. @@ -153,7 +162,7 @@ Write the full run output to the Oz run report. Structure it as follows: # AEO new-guide recommendations — YYYY-MM-DD **Topic areas:** Agents, cloud agents, orchestration, and Oz. -**Snapshot age:** N days (generated YYYY-MM-DD) +**Source signals:** Peec [available | unavailable] **Briefs produced:** N --- @@ -187,7 +196,7 @@ If no briefs can be produced (fewer than 3 topics with usable signal), write a n ## No-run report -If the run cannot produce at least 3 actionable briefs — because the snapshot has insufficient signal for the topic areas, existing docs already cover all candidates well, or fewer than 3 topics pass the brief quality rules — write a no-run report in the Oz run output instead of the full brief set. +If the run cannot produce at least 3 actionable briefs — because Peec has insufficient signal for the topic areas, existing docs already cover all candidates well, or fewer than 3 topics pass the brief quality rules — write a no-run report in the Oz run output instead of the full brief set. ```text ## AEO new-guide recommendations — no briefs produced — YYYY-MM-DD @@ -195,7 +204,7 @@ If the run cannot produce at least 3 actionable briefs — because the snapshot **Topic area:** Agents, cloud agents, orchestration, and Oz. **Source signals reviewed:** -- [Peec snapshot summary.] +- [Peec signal summary, or "Peec: unavailable".] - [Existing-docs signal.] **Docs pages inspected:** @@ -211,51 +220,32 @@ If the run cannot produce at least 3 actionable briefs — because the snapshot - [One specific improvement for the next run.] ``` -## Stale snapshot report - -If the Peec snapshot is missing or 14 days old or older, stop immediately. Write this report to the Oz run output: +## Peec unavailable -```text -## AEO new-guide recommendations — snapshot stale - -**Date:** YYYY-MM-DD -**Snapshot age:** [N days (generated YYYY-MM-DD) | file not found | generated_at field missing] -**Threshold:** 14 days +Peec is the primary signal for this skill. When the Peec MCP is unavailable — missing or expired `PEEC_PAT`, the MCP server is not configured in the agent, or the connection fails — do not invent signals and do not silently continue as though the data existed. -The Peec snapshot is too old to support high-confidence recommendations. No briefs were produced. +Instead: -**Action required:** -Run the `refresh-peec-aeo-snapshot` skill in a local Warp agent session where Peec MCP is authenticated. -Skill: buzz/.agents/skills/refresh-peec-aeo-snapshot/SKILL.md - -The agent will run normally on the next scheduled execution once a fresh snapshot is committed to the buzz repo. -``` +1. Log `Peec: unavailable` in the run output, along with the specific failure (for example, "MCP server not configured" or "401 from api.peec.ai"). Never include the token value. +2. Continue with docs-only analysis. A docs-only run can still identify obvious coverage gaps, but every brief it produces must be flagged as low-confidence with a note that Peec signal was unavailable. +3. Apply a higher bar: only produce a brief when the gap is clearly evident from existing docs structure alone. If fewer than 3 topics clear that bar, write the no-run report with `peec unavailable` as the `No-run reason`. +4. Post the Slack notification as usual, using the "No briefs" format when no briefs were produced. -Fill in the `Snapshot age` field as follows — do not invent values: -- File exists and `generated_at` is present: `N days (generated YYYY-MM-DD)` — compute `N` from today's date minus `generated_at`. -- File does not exist: `file not found`. -- File exists but `generated_at` is absent or unparseable: `generated_at field missing`. - -Use the same wording in the `No-run reason` field of the run log entry: -- File stale: `snapshot stale — N days old` -- File missing: `snapshot missing — file not found` -- Field missing: `snapshot missing — generated_at field absent` - -Then post the stale Slack alert (step 8). Exit. Do not write a no-run report. Do not open a PR. +If `PEEC_PAT` is present but Peec still fails, note in the run output that the token may need rotating at **app.peec.ai → Company → API Keys → Personal Access Tokens** so a human can fix it before the next run. ## Run log format Prepend each new entry at the top of `.agents/logs/aeo_new_guide_recommendation_runs.md`, immediately after the `---` separator line. Use this format: ```markdown -## YYYY-MM-DD — [Briefs produced | No briefs | Snapshot stale] +## YYYY-MM-DD — [Briefs produced | No briefs] - **Run**: [Oz run URL if available, otherwise the run ID] -- **Source signals**: Peec snapshot [available | stale | missing] -- **Briefs produced**: [N | 0 | N/A] +- **Source signals**: Peec [available | unavailable] +- **Briefs produced**: [N | 0] - **Topics**: [comma-separated topic slugs, e.g. "oz-scheduling, cloud-agent-setup, ambient-agents" | N/A] - **Repeat topics**: [comma-separated slugs that appeared in a prior run, or "none"] -- **No-run reason**: [low signal | well-covered | snapshot stale — N days old | N/A] +- **No-run reason**: [low signal | well-covered | peec unavailable | N/A] ``` Keep each entry to 6 fields and under 8 lines. Do not add narrative prose. The `Topics` field is how future runs detect repeats — be consistent with slug naming (lowercase, hyphenated). @@ -278,18 +268,7 @@ Oz run: [run URL] ``` ℹ️ AEO new-guide recommendations · YYYY-MM-DD — No briefs Topics reviewed: agents, cloud agents, orchestration, Oz -No briefs: [brief reason — e.g., "fewer than 3 topics with usable signal"] -Oz run: [run URL] -``` - -**Snapshot stale:** - -``` -⚠️ AEO new-guide recommendations · YYYY-MM-DD — Snapshot stale -Snapshot: [N days old (generated YYYY-MM-DD) | file not found | generated_at missing], threshold: 14 days -No recommendations produced. Refresh the snapshot before the next run. -How: run refresh-peec-aeo-snapshot in a local Warp session -Skill: buzz/.agents/skills/refresh-peec-aeo-snapshot/SKILL.md +No briefs: [brief reason — e.g., "fewer than 3 topics with usable signal" or "Peec unavailable"] Oz run: [run URL] ``` @@ -311,17 +290,17 @@ The Docs team reviewer should be able to read the full brief set in 15–30 minu ## Relationship to other AEO skills - **`aeo_brief`** — The brief format this skill uses for each recommendation. Read `.agents/skills/aeo_brief/SKILL.md` for format guidance and vocabulary translation rules. -- **`aeo_crosslink_audit`** — Handles internal cross-linking between existing pages. If the main gap is a missing link, refer to that skill instead of creating a brief here. -- **`refresh-peec-aeo-snapshot`** — Run locally to refresh the Peec snapshot when stale. See `buzz/.agents/skills/refresh-peec-aeo-snapshot/SKILL.md`. +- **`aeo_crosslink_audit`** — Handles internal cross-linking between existing pages. If the main gap is a missing link, refer to that skill instead of creating a brief here. It reads Peec through the same MCP and `PEEC_PAT` setup. +- **`refresh-peec-aeo-snapshot`** — Legacy. This skill previously read a Peec snapshot committed to the `buzz` repo because cloud agents could not authenticate with Peec. That is no longer the case, and the snapshot is not part of this pipeline. See `buzz/.agents/skills/refresh-peec-aeo-snapshot/SKILL.md` only if you need a local snapshot for another purpose. ## Future expansion Do not implement future expansion ideas in this pilot skill. If the audit finds opportunities outside the four topic areas, mention them only as follow-up notes in the run output. Possible future phases include: -- **Outer self-improvement loop** — After the run log has accumulated several entries (roughly 3–4 runs, equivalent to 6–8 weeks of data), an `improve-aeo-new-guide-rec-skill` skill should read the log and the run outputs to identify systematic weaknesses: topics that keep repeating without being addressed, briefs that reviewers consistently ignore, vocabulary the agent gets wrong, or signal gaps that suggest the Peec snapshot scope needs expanding. This mirrors the `improve-aeo-crosslink-skill` pattern used for the crosslink audit. The improvement skill should run manually (not on a schedule) and propose diffs to this SKILL.md for human review before being applied. +- **Outer self-improvement loop** — After the run log has accumulated several entries (roughly 3–4 runs, equivalent to 3–4 months at a monthly cadence), an `improve-aeo-new-guide-rec-skill` skill should read the log and the run outputs to identify systematic weaknesses: topics that keep repeating without being addressed, briefs that reviewers consistently ignore, vocabulary the agent gets wrong, or signal gaps that suggest the Peec query scope needs expanding. This mirrors the `improve-aeo-crosslink-skill` pattern used for the crosslink audit. The improvement skill should run manually (not on a schedule) and propose diffs to this SKILL.md for human review before being applied. - **Expanding topic areas** beyond agents, cloud agents, orchestration, and Oz once the pilot cadence is stable. - **Comparison against open Notion content-planning items** to detect when a recommended topic is already tracked or in progress. - **Lightweight trend reporting** across scheduled runs (e.g., topics that recur three or more times without a logged action become high-priority backlog items). - **Integration with the `missing_docs` skill's output** as an additional signal source. -- **Oz-specific Peec snapshot expansion** — Update the `refresh-peec-aeo-snapshot` skill to collect dedicated Oz-surface signals (Oz web app, Oz CLI, Oz scheduling) so the snapshot covers the full pilot scope with equal confidence. +- **Oz-specific Peec coverage** — Add tracked Peec prompts for dedicated Oz surfaces (Oz web app, Oz CLI, Oz scheduling) so the pilot scope is covered with equal confidence. From 2ad86e95ea5ad472e41b0cfe414c6c0bef7381c4 Mon Sep 17 00:00:00 2001 From: Rachael Rose Renk <91027132+rachaelrenk@users.noreply.github.com> Date: Tue, 4 Aug 2026 14:14:35 -0600 Subject: [PATCH 2/4] docs(skills): harden AEO skills from validation run findings Validation run 019fce3f-6517-754e-8e97-7af8531e5d62 executed the updated skill end to end and surfaced concrete gaps. - Document the Peec MCP call contract: cloud runs may not surface peec-ai as a native tool, so call the JSON-RPC endpoint directly; resolve project_id via list_projects; get_actions drill-downs require url_classification; list_search_queries returns query_text - Tell the agent to treat OWNED how-to/article rows as the docs signal, since many EDITORIAL rows are outreach rather than content gaps - Add channel_not_found handling for Slack: attempt a lookup by channel name, then fall back to run output and report the failure explicitly - Replace oz-dev with oz for run-link resolution. oz-dev is a local dev build and is absent from cloud sandboxes, so build and is absent from cloud sandboxes, so build and is absent from cloud sandboxes, so build and is absent from cloud sandboxes, so build and is hored-By: Warp Agent --- .../references/skill-authoring-guidelines.md | 6 ++++- .agents/skills/aeo_crosslink_audit/SKILL.md | 4 ++- .../aeo_new_guide_recommendations/SKILL.md | 27 +++++++++++++++++-- 3 files changed, 33 insertions(+), 4 deletions(-) diff --git a/.agents/references/skill-authoring-guidelines.md b/.agents/references/skill-authoring-guidelines.md index 707ae4f9..86aecfb8 100644 --- a/.agents/references/skill-authoring-guidelines.md +++ b/.agents/references/skill-authoring-guidelines.md @@ -100,15 +100,19 @@ Never hard-code the Oz host in Slack messages or run output. The agent may run o Always resolve the Oz run link at runtime: ```bash -oz-dev run get "" --output-format json | jq -r '.session_link' +oz run get "" --output-format json | jq -r '.session_link' ``` +Use `oz`, not `oz-dev`. `oz-dev` is a local development build that ships with the Warp dev app; cloud sandboxes only have `oz`, so any skill instructing an agent to call `oz-dev` silently loses its run link. + If the command fails or returns an empty value, omit the `Oz run` line rather than posting a broken link. ### Secrets and environment variables Always use `SLACK_BOT_TOKEN` and other secrets from environment variables — never inline them or print them to run output, logs, or Slack messages. If a required secret is unavailable, write the payload to the run output instead of posting to Slack. Do not crash the run on missing notification credentials. Include this in the skill as an explicit fallback, not just as an assumed environment guarantee. +**A secret being present does not mean it works.** A token can authenticate while the paired channel ID is stale, or the bot may not be a member of the target channel — Slack returns `channel_not_found` in both cases. Skills that post to Slack should define what to do on a failed post (attempt a lookup by channel name, then fall back to run output) and must report the failure explicitly rather than logging the run as fully successful. + ### Slack notifications Post a Slack notification on every run, including no-action runs and runs that exited early because a source signal was unavailable. A missing notification on a no-action run is indistinguishable from a run that silently failed. Use a simple text message (not Block Kit) that can be scanned in under 30 seconds. diff --git a/.agents/skills/aeo_crosslink_audit/SKILL.md b/.agents/skills/aeo_crosslink_audit/SKILL.md index c22a59c7..a071b00f 100644 --- a/.agents/skills/aeo_crosslink_audit/SKILL.md +++ b/.agents/skills/aeo_crosslink_audit/SKILL.md @@ -51,6 +51,8 @@ Do NOT print, log, commit, or include secret values in reports or Slack messages Use the smallest reliable set of source data needed to justify link changes: - **Peec** - Use the Peec MCP (configured in the agent with a Personal Access Token via the `PEEC_PAT` secret) to collect prompts, search queries, actions/recommendations, and source URLs for agents, cloud agents, and orchestration (last 30 days). Filter prompts and queries for relevance to the topic area. If the Peec MCP returns an error or is unavailable (missing `PEEC_PAT`, expired token, or connection failure), log "Peec: unavailable" in the run output and proceed with GSC and docs-only signals only. + - A cloud run may not expose `peec-ai` as a native tool even when it is in the agent config. If no tool appears, call `https://api.peec.ai/mcp` directly over JSON-RPC with `Authorization: Bearer $PEEC_PAT` (initialize, capture the `Mcp-Session-Id` header, then `tools/call`) rather than declaring Peec unavailable. + - Resolve the project with `list_projects` first; all other tools require `project_id`. `get_actions` needs `url_classification` for `scope=owned` and `scope=editorial` drill-downs, and `list_search_queries` returns `query_text` rather than `query`. - **Google Search Console** - When available, use the environment's `GSC_SERVICE_ACCOUNT_CREDENTIALS_JSON` secret to inspect recent queries and pages related to agents, cloud agents, and orchestration. Never print, log, commit, or include the secret value in reports. If a GSC client requires a credentials file path, write the secret to a restricted temporary file, use it for the run, and remove it before finishing. - **Docs repo** - Search existing pages under `src/content/docs/` for relevant source pages, link targets, and related terminology. @@ -277,7 +279,7 @@ Oz run: [run URL] Rules: - Post on every run, including no-change runs. - Never include raw secret values, personal access tokens, or credential file paths in the Slack message. -- Build the `Oz run` link at runtime — never hard-code the Oz host (for example `app.warp.dev` or `oz.warp.dev`). This agent may run on staging or production, and a hard-coded host resolves to the wrong environment (or a generic Runs page). Resolve the environment-correct link from your current run with `oz-dev run get "" --output-format json | jq -r '.session_link'`, substituting the run ID this agent is executing as. +- Build the `Oz run` link at runtime — never hard-code the Oz host (for example `app.warp.dev` or `oz.warp.dev`). This agent may run on staging or production, and a hard-coded host resolves to the wrong environment (or a generic Runs page). Resolve the environment-correct link from your current run with `oz run get "" --output-format json | jq -r '.session_link'`, substituting the run ID this agent is executing as. Cloud sandboxes ship the `oz` CLI; `oz-dev` is a local development build and is not present, so do not call it. - If the Oz run URL is unavailable, omit that line rather than posting a broken link. ## Future expansion boundaries diff --git a/.agents/skills/aeo_new_guide_recommendations/SKILL.md b/.agents/skills/aeo_new_guide_recommendations/SKILL.md index 7c2c87ac..2acf863a 100644 --- a/.agents/skills/aeo_new_guide_recommendations/SKILL.md +++ b/.agents/skills/aeo_new_guide_recommendations/SKILL.md @@ -51,12 +51,31 @@ Do NOT print, log, commit, or include secret values in reports or Slack messages Use the smallest reliable set of source data needed to justify recommendations. -- **Peec** - Use the Peec MCP (configured in the agent with a Personal Access Token via the `PEEC_PAT` secret) to collect prompts, search queries, actions/recommendations, and source URLs for agents, cloud agents, orchestration, and Oz (last 30 days). Filter prompts and queries for relevance to the topic areas. Because Oz is the agent platform underlying cloud agents and orchestration, substantial Oz-relevant signal appears within the agents and orchestration data — look for Oz-related prompts and queries there. Dedicated Oz-surface signals (Oz web app, Oz CLI, Oz scheduling) may still be thin; when Oz-specific coverage is sparse, flag the brief as lower-confidence and note what additional signal would strengthen the recommendation. If the Peec MCP returns an error or is unavailable (missing `PEEC_PAT`, expired token, or connection failure), log "Peec: unavailable" in the run output and follow the "Peec unavailable" section below. +- **Peec** - Call the Peec MCP (authenticated with the `PEEC_PAT` secret) to collect prompts, search queries, actions/recommendations, and source URLs for agents, cloud agents, orchestration, and Oz (last 30 days). See "Calling Peec" below for the call contract — the MCP may not appear as a native tool in a cloud run, in which case call the HTTP endpoint directly. Filter prompts and queries for relevance to the topic areas. Because Oz is the agent platform underlying cloud agents and orchestration, substantial Oz-relevant signal appears within the agents and orchestration data — look for Oz-related prompts and queries there. Dedicated Oz-surface signals (Oz web app, Oz CLI, Oz scheduling) may still be thin; when Oz-specific coverage is sparse, flag the brief as lower-confidence and note what additional signal would strengthen the recommendation. If the Peec MCP returns an error or is unavailable (missing `PEEC_PAT`, expired token, or connection failure), log "Peec: unavailable" in the run output and follow the "Peec unavailable" section below. - **Docs repo** - Search existing pages under `src/content/docs/` for relevant coverage of each candidate topic. Read `AGENTS.md` and `.agents/references/terminology.md` for product naming guidance. - **Prior run log** - Read `.agents/logs/aeo_new_guide_recommendation_runs.md` to identify topics that were recommended in previous runs. If a candidate topic from this run matches a topic from a prior run, note it explicitly in the brief (see "Repeat topic flag" below). Do not invent Peec signals. If Peec has no usable data for a candidate topic, say so in the brief and flag it as low-confidence. +### Calling Peec + +Even when `peec-ai` is configured in the agent config, a cloud run may not expose it as a native tool. In that case, call the MCP endpoint directly over HTTP with the `PEEC_PAT` secret — do not conclude that Peec is unavailable just because no `peec-ai` tool appears in your tool list. + +The endpoint speaks JSON-RPC over HTTP at `https://api.peec.ai/mcp`: + +1. `POST` an `initialize` request with `Authorization: Bearer $PEEC_PAT`, `Content-Type: application/json`, and `Accept: application/json, text/event-stream`. Capture the `Mcp-Session-Id` response header. +2. Send the `notifications/initialized` notification with that session header. +3. Call tools with `method: "tools/call"` and `params: {"name": "", "arguments": {...}}`, passing the session header on every request. + +Resolve the project first with `list_projects` — every other tool requires a `project_id`. Select the Warp project from the result rather than hard-coding an ID. + +Tool contract details that are easy to get wrong: + +- **`get_actions` requires `url_classification` for drill-downs.** Call `scope=overview` first; those rows are navigation metadata and carry no recommendation text. Drilling into `scope=owned` or `scope=editorial` fails validation unless you pass the `url_classification` from the overview row (for example `HOW_TO_GUIDE` or `ARTICLE`). `scope=reference` and `scope=ugc` require `domain` instead. +- **`list_search_queries` returns `query_text`**, not `query`. Parsing for a generic `query` field yields empty clusters. +- **Editorial actions are often outreach, not docs work.** Many `EDITORIAL` rows read like "pitch this publication" or "contact this author." Use `OWNED` rows with a `HOW_TO_GUIDE` or `ARTICLE` classification as the primary docs signal, and only treat an editorial row as a docs signal when its text describes a genuine content gap. +- Responses are columnar JSON (`{columns, rows, rowCount}`), so map values by column index rather than assuming objects. + ## Workflow 1. **Collect Peec signals.** Use the Peec MCP to collect prompts, search queries, actions/recommendations, and source URLs for agents, cloud agents, orchestration, and Oz (last 30 days). If the Peec MCP is unavailable, log "Peec: unavailable" and follow the "Peec unavailable" section. @@ -143,6 +162,10 @@ Do not invent Peec signals. If Peec has no usable data for a candidate topic, sa Replace `` with the message from the appropriate format in the "Slack notification format" section. Do not print `SLACK_BOT_TOKEN` or `GROWTH_DOCS_SLACK_CHANNEL_ID` values in the run output or in any file. + **If `chat.postMessage` returns `channel_not_found`**, the secrets being set is not sufficient — either the channel ID is stale or the bot is not a member of the channel. Do not treat this as a successful post. Instead: + 1. Try resolving the channel by name: call `conversations.list` (types `public_channel,private_channel`) and look for `growth-docs`. If found, retry the post with that ID and report that the stored `GROWTH_DOCS_SLACK_CHANNEL_ID` is wrong so a human can correct the secret. + 2. If the lookup also fails or returns `missing_scope`, the bot is not in the channel or lacks scope. Write the notification body to the run output, and state explicitly in the run output that the Slack post failed with `channel_not_found` — never imply it was delivered. + ## Brief quality rules Before finalizing each brief: @@ -275,7 +298,7 @@ Oz run: [run URL] Rules: - Post on every run, including no-brief runs. - Never include raw secret values, personal access tokens, or credential file paths in the Slack message. -- Build the `Oz run` link at runtime — never hard-code the Oz host (for example `app.warp.dev` or `oz.warp.dev`). This agent may run on staging or production, and a hard-coded host resolves to the wrong environment (or a generic Runs page). Resolve the environment-correct link from your current run with `oz-dev run get "" --output-format json | jq -r '.session_link'`, substituting the run ID this agent is executing as. +- Build the `Oz run` link at runtime — never hard-code the Oz host (for example `app.warp.dev` or `oz.warp.dev`). This agent may run on staging or production, and a hard-coded host resolves to the wrong environment (or a generic Runs page). Resolve the environment-correct link from your current run with `oz run get "" --output-format json | jq -r '.session_link'`, substituting the run ID this agent is executing as. Cloud sandboxes ship the `oz` CLI; `oz-dev` is a local development build and is not present, so do not call it. - If the Oz run URL is unavailable, omit that line rather than posting a broken link. ## Human review expectations From d3387f566badd6819cf24278f2f2f47866fedb19 Mon Sep 17 00:00:00 2001 From: Rachael Rose Renk <91027132+rachaelrenk@users.noreply.github.com> Date: Tue, 4 Aug 2026 14:23:22 -0600 Subject: [PATCH 3/4] docs(skills): replace oz-dev with oz across remaining skills oz-dev is a local development build shipped in the Warp dev app; cloud sandboxes only have oz. Every scheduled skill calling oz-dev to resolve its session link hit command-not-found and silently omitted the run link from its Slack notification. - weekly-404-monitor (run link + the schedule create deploy command) - improve-drafting-skills (conversation fetch + run link) - improve-aeo-crosslink-skill - improve-404-monitor-skill Also add the channel_not_found recovery path to aeo_crosslink_audit, which shares the GROWTH_DOCS_SLACK_CHANNEL_ID secret and has been failing to deliver its #growth-docs notifications for the same reason. Co-Authored-By: Warp Agent --- .agents/skills/aeo_crosslink_audit/SKILL.md | 4 ++++ .agents/skills/improve-404-monitor-skill/SKILL.md | 2 +- .agents/skills/improve-aeo-crosslink-skill/SKILL.md | 2 +- .agents/skills/improve-drafting-skills/SKILL.md | 4 ++-- .agents/skills/weekly-404-monitor/SKILL.md | 4 ++-- 5 files changed, 10 insertions(+), 6 deletions(-) diff --git a/.agents/skills/aeo_crosslink_audit/SKILL.md b/.agents/skills/aeo_crosslink_audit/SKILL.md index a071b00f..b12a65c0 100644 --- a/.agents/skills/aeo_crosslink_audit/SKILL.md +++ b/.agents/skills/aeo_crosslink_audit/SKILL.md @@ -128,6 +128,10 @@ If Google Search Console data is unavailable, say what could not be verified and Replace `` with the message from the appropriate format in the "Slack notification format" section. Do not print `SLACK_BOT_TOKEN` or `GROWTH_DOCS_SLACK_CHANNEL_ID` values in the run output or in any file. + **If `chat.postMessage` returns `channel_not_found`**, the secrets being set is not sufficient — either the channel ID is stale or the bot is not a member of the channel. Do not treat this as a successful post. Instead: + 1. Try resolving the channel by name: call `conversations.list` (types `public_channel,private_channel`) and look for `growth-docs`. If found, retry the post with that ID and report that the stored `GROWTH_DOCS_SLACK_CHANNEL_ID` is wrong so a human can correct the secret. + 2. If the lookup also fails or returns `missing_scope`, the bot is not in the channel or lacks scope. Write the notification body to the run output, and state explicitly in the run output that the Slack post failed with `channel_not_found` — never imply it was delivered. + ## Link quality rules When adding links, follow the link style guidance in `AGENTS.md` and validate with `style_lint`. diff --git a/.agents/skills/improve-404-monitor-skill/SKILL.md b/.agents/skills/improve-404-monitor-skill/SKILL.md index ccf2d305..339babc4 100644 --- a/.agents/skills/improve-404-monitor-skill/SKILL.md +++ b/.agents/skills/improve-404-monitor-skill/SKILL.md @@ -155,7 +155,7 @@ Oz run: [run URL] In both messages, build the `Oz run` link at runtime — never hard-code the Oz host. Resolve from your current run: ```bash -oz-dev run get "" --output-format json | jq -r '.session_link' +oz run get "" --output-format json | jq -r '.session_link' ``` If the command fails or returns an empty value, omit the `Oz run` line. diff --git a/.agents/skills/improve-aeo-crosslink-skill/SKILL.md b/.agents/skills/improve-aeo-crosslink-skill/SKILL.md index 546f82d0..1347d6af 100644 --- a/.agents/skills/improve-aeo-crosslink-skill/SKILL.md +++ b/.agents/skills/improve-aeo-crosslink-skill/SKILL.md @@ -166,7 +166,7 @@ Oz run: [run URL] ``` In both messages, build the `Oz run` link at runtime — never hard-code the Oz host (for example `app.warp.dev` or `oz.warp.dev`). This agent may run on staging or production, and a hard-coded host resolves to the wrong environment (or a generic Runs page). Resolve the environment-correct link from your current run, substituting the run ID this agent is executing as: ```bash -oz-dev run get "" --output-format json | jq -r '.session_link' +oz run get "" --output-format json | jq -r '.session_link' ``` If the command fails or returns an empty value, omit the `Oz run` line rather than posting a hard-coded or broken URL. diff --git a/.agents/skills/improve-drafting-skills/SKILL.md b/.agents/skills/improve-drafting-skills/SKILL.md index 8c56df7e..58b43b38 100644 --- a/.agents/skills/improve-drafting-skills/SKILL.md +++ b/.agents/skills/improve-drafting-skills/SKILL.md @@ -54,7 +54,7 @@ This produces one perpetual, low-noise PR that accumulates every run's log entri 1. Use `oz run list` to find all Oz runs in the past 30 days whose skill name matches a drafting skill (`draft_docs`, `draft_feature_doc`, `draft_conceptual`, etc.) or `review-docs-pr`. 2. For each run, retrieve the full conversation and extract agent text messages: ```bash - oz-dev run get --conversation RUN_ID --output-format json | \ + oz run get --conversation RUN_ID --output-format json | \ jq -r '[.. | objects | select(.role? == "assistant") | .content[]? | select(.type? == "text") | .text] | .[]' ``` The top-level response is `{steps: [...]}`, not `{messages: [...]}`, and steps can be nested — use recursive descent (`..`) to reach all assistant messages at any depth. Do not rely on `oz run get` without `--conversation` — that returns only the brief `status_message` field, not conversation content or shell stdout. @@ -189,7 +189,7 @@ Oz run: [run URL] ``` Build the `Oz run` link at runtime — never hard-code the Oz host (for example `app.warp.dev` or `oz.warp.dev`). This agent may run on staging or production, and a hard-coded host resolves to the wrong environment (or a generic Runs page). Resolve the environment-correct link from your current run, substituting the run ID this agent is executing as: ```bash -oz-dev run get "" --output-format json | jq -r '.session_link' +oz run get "" --output-format json | jq -r '.session_link' ``` If the command fails or returns an empty value, omit the `Oz run` line rather than posting a hard-coded or broken URL. diff --git a/.agents/skills/weekly-404-monitor/SKILL.md b/.agents/skills/weekly-404-monitor/SKILL.md index 13a7f2cb..2a2b98b1 100644 --- a/.agents/skills/weekly-404-monitor/SKILL.md +++ b/.agents/skills/weekly-404-monitor/SKILL.md @@ -98,7 +98,7 @@ _+{long_tail_count} other uncovered URLs under {report_min_hits} hits each (most Build `{oz_run_url}` at runtime — never hard-code the Oz host (for example `app.warp.dev` or `oz.warp.dev`). This agent may run on staging or production, and a hard-coded host resolves to the wrong environment (or a generic Runs page). Resolve the environment-correct link from your current run, substituting the run ID this agent is executing as: ```bash -oz-dev run get "" --output-format json | jq -r '.session_link' +oz run get "" --output-format json | jq -r '.session_link' ``` If the command fails or returns an empty value, omit the `→ Full breakdown` line rather than posting a hard-coded or broken URL. @@ -243,7 +243,7 @@ To deploy (one-time setup): - `BUZZ_SLACK_TOKEN` — Slack bot token (already provisioned; used by other doc agents in this environment) 3. Register the schedule via the Oz CLI: ```sh - oz-dev schedule create \ + oz schedule create \ --name "weekly-404-monitor" \ --cron "0 17 * * 1" \ --environment K5KStCm5aYvhfBJb8cHol6 \ From 61066a2a2e4182e3d227f3f3d455eeabdbe41d55 Mon Sep 17 00:00:00 2001 From: Rachael Rose Renk <91027132+rachaelrenk@users.noreply.github.com> Date: Tue, 4 Aug 2026 14:29:54 -0600 Subject: [PATCH 4/4] docs(skills): post to #growth-docs as buzz, not hubble SLACK_BOT_TOKEN authenticates as the hubble bot, which is not a member of #growth-docs and cannot be invited. buzz is the account that posts to that channel, so both AEO skills now use BUZZ_SLACK_TOKEN. Name the expected bot in each skill's environment requirements, and add guidance to the authoring reference that Slack tokens authenticate as different bots with different channel memberships, so a token that authenticates is not necessarily a token that can deliver. Co-Authored-By: Warp Agent --- .agents/references/skill-authoring-guidelines.md | 4 +++- .agents/skills/aeo_crosslink_audit/SKILL.md | 10 +++++----- .agents/skills/aeo_new_guide_recommendations/SKILL.md | 10 +++++----- 3 files changed, 13 insertions(+), 11 deletions(-) diff --git a/.agents/references/skill-authoring-guidelines.md b/.agents/references/skill-authoring-guidelines.md index 86aecfb8..a2aa3500 100644 --- a/.agents/references/skill-authoring-guidelines.md +++ b/.agents/references/skill-authoring-guidelines.md @@ -109,7 +109,9 @@ If the command fails or returns an empty value, omit the `Oz run` line rather th ### Secrets and environment variables -Always use `SLACK_BOT_TOKEN` and other secrets from environment variables — never inline them or print them to run output, logs, or Slack messages. If a required secret is unavailable, write the payload to the run output instead of posting to Slack. Do not crash the run on missing notification credentials. Include this in the skill as an explicit fallback, not just as an assumed environment guarantee. +Always read Slack tokens and other secrets from environment variables — never inline them or print them to run output, logs, or Slack messages. If a required secret is unavailable, write the payload to the run output instead of posting to Slack. Do not crash the run on missing notification credentials. Include this in the skill as an explicit fallback, not just as an assumed environment guarantee. + +**Pick the token that matches the destination channel.** Several Slack bot tokens exist in the Oz secret store, and they authenticate as different bots with different channel memberships. A token that authenticates successfully still cannot post to a channel its bot has not joined. Name the expected bot in the skill's environment requirements (for example, `BUZZ_SLACK_TOKEN` posts as `buzz`, which is the account in `#growth-docs`) so a future editor does not swap in a token that authenticates but cannot deliver. **A secret being present does not mean it works.** A token can authenticate while the paired channel ID is stale, or the bot may not be a member of the target channel — Slack returns `channel_not_found` in both cases. Skills that post to Slack should define what to do on a failed post (attempt a lookup by channel name, then fall back to run output) and must report the failure explicitly rather than logging the run as fully successful. diff --git a/.agents/skills/aeo_crosslink_audit/SKILL.md b/.agents/skills/aeo_crosslink_audit/SKILL.md index b12a65c0..35a7fe1a 100644 --- a/.agents/skills/aeo_crosslink_audit/SKILL.md +++ b/.agents/skills/aeo_crosslink_audit/SKILL.md @@ -29,7 +29,7 @@ Do not: The following environment secrets should be set in the Oz cloud agent environment: - `PEEC_PAT` — Peec Personal Access Token for MCP authentication. Create one at **app.peec.ai → Company → API Keys → Personal Access Tokens**. If unavailable or expired, the audit proceeds with GSC and docs-only signals and logs "Peec: unavailable" in the run output. -- `SLACK_BOT_TOKEN` — Slack bot token for posting to `#growth-docs`. If unavailable, write the notification body to the run output instead and skip Slack posting. +- `BUZZ_SLACK_TOKEN` — Slack bot token for posting to `#growth-docs`, authenticating as the `buzz` bot. This is the account that posts to that channel; do not substitute another Slack token without confirming the bot is a member of the channel. If unavailable, write the notification body to the run output instead and skip Slack posting. - `GROWTH_DOCS_SLACK_CHANNEL_ID` — Slack channel ID for `#growth-docs`. Find it in Slack by right-clicking the channel → Copy link (the ID begins with `C`). If unavailable, skip Slack posting. The scheduled cloud agent must also include the Peec MCP server in its agent config (pass via `--mcp` flag or the agent config file `mcp_servers` key): @@ -91,16 +91,16 @@ If Google Search Console data is unavailable, say what could not be verified and This produces one perpetual, low-noise PR that accumulates every run's entry regardless of outcome. Reviewers merge it periodically (at minimum before each monthly `improve-aeo-crosslink-skill` run) so the log reaches `main`. If any git step fails, write the log entry to the run output instead and continue to step 8. -8. **Post Slack notification.** After writing the log entry, post the formatted message to `#growth-docs` using the Python snippet below. Python is preferred over curl because it reads `SLACK_BOT_TOKEN` from the environment (keeping the token out of process argv) and JSON-encodes the payload correctly regardless of newlines or special characters. If either secret is unavailable, write the notification body to the run output instead. +8. **Post Slack notification.** After writing the log entry, post the formatted message to `#growth-docs` using the Python snippet below. Python is preferred over curl because it reads `BUZZ_SLACK_TOKEN` from the environment (keeping the token out of process argv) and JSON-encodes the payload correctly regardless of newlines or special characters. If either secret is unavailable, write the notification body to the run output instead. ```bash python3 - <<'SLACK_EOF' import os, json, urllib.request, sys - token = os.environ.get("SLACK_BOT_TOKEN", "") + token = os.environ.get("BUZZ_SLACK_TOKEN", "") channel = os.environ.get("GROWTH_DOCS_SLACK_CHANNEL_ID", "") if not token or not channel: - print("SLACK_BOT_TOKEN or GROWTH_DOCS_SLACK_CHANNEL_ID not set — skipping Slack notification", file=sys.stderr) + print("BUZZ_SLACK_TOKEN or GROWTH_DOCS_SLACK_CHANNEL_ID not set — skipping Slack notification", file=sys.stderr) sys.exit(0) # Replace the triple-quoted string with the message from the Slack notification format section. @@ -126,7 +126,7 @@ If Google Search Console data is unavailable, say what could not be verified and SLACK_EOF ``` - Replace `` with the message from the appropriate format in the "Slack notification format" section. Do not print `SLACK_BOT_TOKEN` or `GROWTH_DOCS_SLACK_CHANNEL_ID` values in the run output or in any file. + Replace `` with the message from the appropriate format in the "Slack notification format" section. Do not print `BUZZ_SLACK_TOKEN` or `GROWTH_DOCS_SLACK_CHANNEL_ID` values in the run output or in any file. **If `chat.postMessage` returns `channel_not_found`**, the secrets being set is not sufficient — either the channel ID is stale or the bot is not a member of the channel. Do not treat this as a successful post. Instead: 1. Try resolving the channel by name: call `conversations.list` (types `public_channel,private_channel`) and look for `growth-docs`. If found, retry the post with that ID and report that the stored `GROWTH_DOCS_SLACK_CHANNEL_ID` is wrong so a human can correct the secret. diff --git a/.agents/skills/aeo_new_guide_recommendations/SKILL.md b/.agents/skills/aeo_new_guide_recommendations/SKILL.md index 2acf863a..64e82a02 100644 --- a/.agents/skills/aeo_new_guide_recommendations/SKILL.md +++ b/.agents/skills/aeo_new_guide_recommendations/SKILL.md @@ -29,7 +29,7 @@ Do not: The following environment secrets should be set in the Oz cloud agent environment: - `PEEC_PAT` — Peec Personal Access Token for MCP authentication. Create one at **app.peec.ai → Company → API Keys → Personal Access Tokens**. If unavailable or expired, the run proceeds with docs-only signals and logs "Peec: unavailable" in the run output. -- `SLACK_BOT_TOKEN` — Slack bot token for posting to `#growth-docs`. If unavailable, write the notification body to the run output instead and skip Slack posting. +- `BUZZ_SLACK_TOKEN` — Slack bot token for posting to `#growth-docs`, authenticating as the `buzz` bot. This is the account that posts to that channel; do not substitute another Slack token without confirming the bot is a member of the channel. If unavailable, write the notification body to the run output instead and skip Slack posting. - `GROWTH_DOCS_SLACK_CHANNEL_ID` — Slack channel ID for `#growth-docs`. Find it in Slack by right-clicking the channel → Copy link (the ID begins with `C`). If unavailable, skip Slack posting. The scheduled cloud agent must also include the Peec MCP server in its agent config (pass via `--mcp` flag or the agent config file `mcp_servers` key): @@ -125,16 +125,16 @@ Tool contract details that are easy to get wrong: This produces one perpetual, low-noise PR that accumulates every run's entry regardless of outcome. Reviewers merge it periodically so the log data reaches `main` and can inform the skill-improvement loop. If any git step fails, write the log entry to the run output instead and continue to step 8 — do not silently skip the log. -8. **Post Slack notification.** After writing the log entry, post the formatted message to `#growth-docs` using the Python snippet below. Python is preferred over curl because it reads `SLACK_BOT_TOKEN` from the environment (keeping the token out of process argv) and JSON-encodes the payload correctly regardless of newlines or special characters. If either secret is unavailable, write the notification body to the run output instead. +8. **Post Slack notification.** After writing the log entry, post the formatted message to `#growth-docs` using the Python snippet below. Python is preferred over curl because it reads `BUZZ_SLACK_TOKEN` from the environment (keeping the token out of process argv) and JSON-encodes the payload correctly regardless of newlines or special characters. If either secret is unavailable, write the notification body to the run output instead. ```bash python3 - <<'SLACK_EOF' import os, json, urllib.request, sys - token = os.environ.get("SLACK_BOT_TOKEN", "") + token = os.environ.get("BUZZ_SLACK_TOKEN", "") channel = os.environ.get("GROWTH_DOCS_SLACK_CHANNEL_ID", "") if not token or not channel: - print("SLACK_BOT_TOKEN or GROWTH_DOCS_SLACK_CHANNEL_ID not set — skipping Slack notification", file=sys.stderr) + print("BUZZ_SLACK_TOKEN or GROWTH_DOCS_SLACK_CHANNEL_ID not set — skipping Slack notification", file=sys.stderr) sys.exit(0) # Replace the triple-quoted string with the message from the Slack notification format section. @@ -160,7 +160,7 @@ Tool contract details that are easy to get wrong: SLACK_EOF ``` - Replace `` with the message from the appropriate format in the "Slack notification format" section. Do not print `SLACK_BOT_TOKEN` or `GROWTH_DOCS_SLACK_CHANNEL_ID` values in the run output or in any file. + Replace `` with the message from the appropriate format in the "Slack notification format" section. Do not print `BUZZ_SLACK_TOKEN` or `GROWTH_DOCS_SLACK_CHANNEL_ID` values in the run output or in any file. **If `chat.postMessage` returns `channel_not_found`**, the secrets being set is not sufficient — either the channel ID is stale or the bot is not a member of the channel. Do not treat this as a successful post. Instead: 1. Try resolving the channel by name: call `conversations.list` (types `public_channel,private_channel`) and look for `growth-docs`. If found, retry the post with that ID and report that the stored `GROWTH_DOCS_SLACK_CHANNEL_ID` is wrong so a human can correct the secret.