diff --git a/.squad/agents/bunk/history.md b/.squad/agents/bunk/history.md index 11e769e..3531590 100644 --- a/.squad/agents/bunk/history.md +++ b/.squad/agents/bunk/history.md @@ -11,3 +11,22 @@ ## Learnings - First session: team initialized 2026-03-05 +- **Reddit JSON API:** Requires User-Agent header, respects X-RateLimit headers, returns 1 request per 2 seconds (~30 requests/min) +- **Stack Overflow API:** 300 req/day free (10k with key); `/search/advanced.json` is the query endpoint; timestamps are Unix epoch +- **Bluesky atproto:** Fully open, no auth required, generous rate limits, standard REST API at https://public.api.bsky.app/xrpc +- **Mastodon:** Instance-specific APIs, typically `/api/v1/timelines/tag/{hashtag}`, HTML-escaped content requires unescaping +- **X/Twitter:** API is now paid (formerly free); Elevated tier ~$5k/month; low ROI for Aspire discovery (high noise) +- **Discord:** Requires bot setup + human server admin approval; historical message API is paginated (100 messages per call) +- **GitHub Discussions:** Supported by REST API, separate from Issues, not yet queried in community.ts +- **Canonical IDs:** Team uses sha256(id+url+author+date), hex 16 chars; ensures deterministic dedupe +- **Community source priority:** Reddit > Stack Overflow > GitHub Discussions (quick wins); defer X/LinkedIn/Discord until budget/access confirmed + +#### Cross-Agent Findings (from squad sync 2026-03-12) + +**Signal gap analysis (Omar):** Reddit is critical for real-time pain signals and adoption friction detection. Stack Overflow captures eternal pain patterns that inform content strategy. Together, these unlock medium-term editorial intelligence. + +**Content discovery (Kima):** RSS feeds are lowest-friction source; blog platforms via API add engagement metrics. Multi-source deduplication is critical. Recommend implementing Dev.to API alongside Reddit and Stack Overflow. + +**Architecture recommendation (Freamon):** Design community sources as SourceAdapter modules. Each source (GitHub, Reddit, Stack Overflow, etc.) becomes self-contained with its own rate limiting, error handling, and validation. + +**Team consensus:** Phase 1 (Reddit + Stack Overflow + Dev.to) gives Beth 50% signal completeness. Phase 2 (YouTube) adds creator mapping. Phase 3 deferred until early wins stabilized. diff --git a/.squad/agents/freamon/history.md b/.squad/agents/freamon/history.md index ac2c760..087a0e1 100644 --- a/.squad/agents/freamon/history.md +++ b/.squad/agents/freamon/history.md @@ -11,3 +11,56 @@ ## Learnings - First session: team initialized 2026-03-05 +- **Architecture patterns**: + - Current discovery structure (content.ts / community.ts) is monolithic and won't scale beyond 3-4 sources + - Recommendation: Adapter pattern with SourceAdapter interface for plug-and-play source extensibility + - Registry pattern (SourceRegistry) for auto-discovery, validation, and parallel execution + - Each adapter owns: env var requirements, rate limiting, error handling, validation logic +- **Source prioritization for Beth**: + - Phase 1 (quick wins): Reddit JSON API, Dev.to API, Stack Overflow API — no auth, immediate value + - Phase 2 (API-key): YouTube Data API, podcast RSS — high value, requires config + - Phase 3 (hard): Social media scrapers (X/LinkedIn/Bluesky), conference crawlers — high effort +- **Key file paths**: + - Discovery modules: src/discovery/content.ts, src/discovery/community.ts + - Types: src/types.ts (all interfaces: ContentItem, DiscoveryResult, RunState, etc.) + - Taxonomy: src/taxonomy.ts (dedupe + classification logic) + - Pipeline: src/index.ts (orchestrator: load → discover → dedupe → classify → analyze → output) + - Output: src/output.ts (generates all 9 reports) + - State: src/state.ts (persists RunState to AspireContentEngine/.state/) +- **User preferences**: + - Brady prioritizes editorial intelligence breadth (cover as many sources as possible) + - Beth needs signal quality over volume (actionability: amplify/respond more important than raw counts) + - Team prefers TypeScript, lean on Node.js built-ins, avoid heavy dependencies unless necessary + +#### Cross-Agent Findings (from squad sync 2026-03-12) + +**Signal analysis (Omar):** Architecture refactor will unlock 50% signal completeness (Phase 1), 65% with Phase 2 (YouTube), 82% with all sources. Current bottleneck is monolithic discovery files. + +**Community discovery (Bunk):** Prioritizes Reddit > Stack Overflow > GitHub Discussions for immediate Phase 1 value. X/Twitter deferred due to API cost (\/month). LinkedIn deferred in favor of link enrichment. + +**Content discovery (Kima):** RSS expansion + YouTube + podcasts gives 85%+ blog coverage. Multi-source deduplication strategy (sha256 canonical IDs) scales to 20+ sources. + +**Squad consensus:** Refactor first (2.5 sessions), then Phase 1 (2-3 sessions). This gives Brady technical debt cleanup + Beth immediate editorial ROI. Phase 2 & 3 follow after Phase 1 validation. + +#### PRD Authoring Session (2026-03-12) + +**Issues filed:** 7 GitHub PRDs on bradygaster/ACCES (issues #1-#7), all labeled `squad` +- **Issue #1:** Architecture Refactor (McNulty + Stringer, 2.5 sessions) — foundation for all adapters +- **Issues #2-#5:** Phase 1 adapters (Reddit, Dev.to, Stack Overflow, GitHub Discussions) — 3.5 sessions total +- **Issues #6-#7:** Phase 2 adapters (YouTube, Podcasts) — 2.5 sessions total + +**PRD structure used:** +- Overview (what + why for Beth) +- Background (gap analysis context) +- Requirements (implementation checklist) +- Squad SDK Integration (adapter patterns) +- Acceptance Criteria (testable outcomes) +- Dependencies (issue blocking relationships) +- Assigned To (squad member ownership) +- Estimated Effort (session count) + +**Dependency chain:** Issue #1 blocks all others. Phase 1 issues (#2-#5) can parallelize after #1. Phase 2 issues (#6-#7) follow Phase 1 validation. + +**Key learning:** GitHub CLI required account switch (`gh auth switch --user bradygaster`) when working with personal repos while logged into Enterprise Managed User account. All issues created successfully after switch. + +**Deliverable:** Summary written to `.squad/decisions/inbox/freamon-prd-issues.md` for Ralph's triage. diff --git a/.squad/agents/kima/history.md b/.squad/agents/kima/history.md index 9d67bf6..354d525 100644 --- a/.squad/agents/kima/history.md +++ b/.squad/agents/kima/history.md @@ -10,4 +10,40 @@ ## Learnings -- First session: team initialized 2026-03-05 +### Session 2026-03-05: Content Source Gap Analysis + +#### Discovered Patterns +- **RSS feeds are the lowest-friction source:** Dev.to, Microsoft blogs all use RSS. This pattern should be expanded to podcasts and any new feeds discovered. +- **Multi-source deduplication is critical:** Same content (esp. talks) reappear across YouTube, blogs, social media, conferences. Need a robust canonical ID strategy. +- **API availability varies widely:** Official Microsoft channels have simple URLs; YouTube/Reddit are well-documented; Substack/Medium are either deprecated or behind scraping concerns. + +#### Key Implementation Notes +- All sources feed into `ContentItem` type from `src/types.ts` — no custom models needed +- `generateCanonicalId()` function (SHA256 of title+url+author+date) is stable and reusable +- `isAspireRelated()` + `isExcluded()` helpers should remain the core relevance filter +- Topic extraction via keyword matching works well for broad categories; fine-tuning will happen later + +#### Architecture Decisions +- **Blog discovery strategy:** Start with RSS for low-friction sources (podcasts), then layer Hashnode API (free, clean). Defer Medium scraping until yield metrics justify the effort. +- **Social media:** X API and Bluesky are viable; LinkedIn is blocked (TOS). Recommend X + Bluesky for Phase 2. +- **Conferences:** No single API. Must implement per-conference (NDC scraper first, then .NET Conf, Build). Manual watchlist is acceptable until volume justifies automation. +- **Podcast feeds:** Expand `RSS_FEEDS` constant to include 5–10 known .NET podcasts. Zero API cost, high long-tail value. + +#### For Next Scout Sessions +- YouTube implementation should leverage `@googleapis/youtube` package; start with simple title-based search +- Reddit can use either RSS (lower friction) or PRAW (higher signal); recommend RSS-first approach for simplicity +- When implementing blog platforms, coordinate with Bubbles (taxonomy scout) on topic inference; many platforms use custom tag systems +- All new sources should write raw evidence to `/raw/` folder (JSON snippets of API responses or feed excerpts) for auditability + +#### Files Modified / Created +- `.squad/decisions/inbox/kima-content-source-gaps.md` — Primary deliverable (gap analysis) + +#### Cross-Agent Findings (from squad sync 2026-03-12) + +**Signal gap analysis (Omar):** Content sources alone capture ~85% of blog content but only 40% of overall tech adoption signals. YouTube, podcast, and conference discovery will significantly improve coverage. + +**Community discovery (Bunk):** GitHub is fully implemented for community signals but covers only ~20% of real community conversation. Reddit, Stack Overflow, and GitHub Discussions are critical quick wins. + +**Architecture recommendation (Freamon):** Implement SourceAdapter pattern before adding 6+ new sources. Current monolithic structure (content.ts, community.ts) needs refactoring for scale. + +**Team consensus:** Prioritize Phase 1 (Reddit, Dev.to, Stack Overflow) for immediate editorial value; Phase 2 (YouTube, podcasts) adds creator/influencer coverage; Phase 3 (social, conferences) is secondary. diff --git a/.squad/agents/mcnulty/history.md b/.squad/agents/mcnulty/history.md index aeffb57..22f6186 100644 --- a/.squad/agents/mcnulty/history.md +++ b/.squad/agents/mcnulty/history.md @@ -21,3 +21,8 @@ - Pipeline flow: loadState → discoverContent + discoverCommunity → deduplicate → classify → analyze → generateOutput → saveState - Canonical ID = sha256(title|url|author|date).slice(0,16) - The `Channel` type uses `(string & {})` pattern to allow known unions + arbitrary strings +- `generateCanonicalId` is duplicated in `content.ts` (line 214) and `taxonomy.ts` (line 88) — must consolidate during SourceAdapter refactor +- `truncate` helper is duplicated in `content.ts` and `community.ts` — consolidate into shared helpers module +- Skeleton discovery functions (blog search, YouTube, Reddit, social media) all return `[]` — safe to remove during adapter extraction +- Enterprise Managed User (EMU) GitHub account can't use GraphQL for issue comments — use `gh issue comment --body-file` with REST API instead +- Produced implementation plan for Issue #1 (SourceAdapter pattern) — posted to https://github.com/bradygaster/ACCES/issues/1#issuecomment-4045230035 and `.squad/decisions/inbox/mcnulty-issue1-plan.md` diff --git a/.squad/agents/omar/history.md b/.squad/agents/omar/history.md index 6a30acc..4ac20bd 100644 --- a/.squad/agents/omar/history.md +++ b/.squad/agents/omar/history.md @@ -10,4 +10,53 @@ ## Learnings -- First session: team initialized 2026-03-05 +### Architecture & Implementation + +- **Two-scout model**: `discoverContent()` (blogs/RSS) and `discoverCommunity()` (GitHub) are separate pipelines, both returning `DiscoveryResult[]` to be merged and deduplicated +- **GitHub is the only live community source**: Issues, PRs, and repo metadata provide GitHub signal; Reddit/social/YT/podcasts are stubs waiting implementation +- **Canonical ID strategy**: Content is deduplicated via `generateCanonicalId(title, url, author, date)` — SHA256 hash, deterministic across runs +- **Signal classification happens late**: `tags.signal` is populated post-discovery (e.g., `inferIssueSignal()` for GitHub issues; RSS items default to `['other']`) +- **Taxonomy is inclusive**: All channels/types/topics are extensible strings; no hard enums except for the core types (`ContentType`, `Channel`, `Signal`, etc.) + +### Signal Gaps (Key Finding) + +- **Current completeness**: ~32% (RSS + GitHub only) — Beth sees blogs and repos, misses community friction +- **Critical blind spots** (in order): + 1. **Reddit** (real-time pain signals, adoption friction, sentiment) — HIGH priority + 2. **Stack Overflow** (eternal pain patterns, docs gaps) — MEDIUM-HIGH priority + 3. **YouTube** (creator ecosystem, tutorial gaps, reach) — HIGH priority + 4. **Social media X/LinkedIn** (amplification, mood, decision-makers) — MEDIUM-HIGH priority + 5. **Podcasts** (industry narrative, thought leadership) — MEDIUM priority +- **Projected completeness with all sources**: 82% — would give Beth ~4/5 of the conversation + +### User Needs (Beth's Perspective) + +- **Editorial priorities**: Detection of pain points, creator mapping, gap filling, amplification queues +- **Real-time vs. batch**: Blogs can be daily; GitHub/Reddit/X are real-time; SO/podcasts are evergreen/weekly +- **Actionability hierarchy**: Amplify > Respond > Follow-up > Investigate > Ignore (tagged on every item) +- **Sentiment is missing**: No current way to measure community mood; social signals + Reddit would unlock this + +### File Paths & Key Code + +- **Discovery modules**: `src/discovery/content.ts` (RSS) and `src/discovery/community.ts` (GitHub) +- **Analysis**: `src/analysis.ts` generates trends, gaps, amplifications from classified items +- **Taxonomy**: `src/taxonomy.ts` (not viewed yet) handles classification +- **Run state**: `.state/` directory persists `known_ids` and `known_dupes` for dedup across runs +- **ACCES spec**: `ACCES.md` is the ground truth for output format and role definitions + +### First Session Outputs + +- **Signal gaps document**: `.squad/decisions/inbox/omar-signal-gaps.md` — detailed analysis of 5 missing signal types, completeness framework, and implementation priority roadmap +- **Recommendation**: Start with Reddit + SO (high impact, medium effort); defer podcasts to medium term + +- First session: team initialized 2026-03-05; signal gap analysis completed 2026-03-05 + +#### Cross-Agent Findings (from squad sync 2026-03-12) + +**Community discovery (Bunk):** Reddit and Stack Overflow address the largest signal blind spots. Reddit captures real-time pain/confusion (15% → 50% completeness). Stack Overflow captures eternal patterns and docs gaps. + +**Content discovery (Kima):** YouTube is the second-highest-priority discovery source for creator ecosystem mapping. Currently 0% coverage; Phase 2 implementation will address creator influence gap. + +**Architecture recommendation (Freamon):** Phased rollout requires adapter pattern foundation. Suggests SourceAdapter interface with validation, rate limiting, and graceful error handling built into each source module. + +**Team consensus:** Phase 1 → 50% completeness (Reddit, Dev.to, Stack Overflow). Phase 2 → 65% completeness (add YouTube). Phase 3 → 82% completeness (add social, podcasts, conferences). Recommend refactor before Phase 1 implementation. diff --git a/.squad/agents/scribe/history.md b/.squad/agents/scribe/history.md index 3949df2..a0bfe0f 100644 --- a/.squad/agents/scribe/history.md +++ b/.squad/agents/scribe/history.md @@ -1,16 +1,60 @@ -# Project Context +# Scribe — History -- **Project:** Acces -- **Created:** 2026-03-05 +## Project Context -## Core Context +- **Project:** ACCES — Aspire Community Content Engine Squad +- **Role:** Documentation specialist maintaining history, decisions, and technical records +- **Owner:** bradygaster (Brady), with Beth Massi as the primary customer/user +- **Purpose:** Consolidate squad outputs, maintain decision registry, cross-agent knowledge sharing -Agent Scribe initialized and ready for work. +## Learnings -## Recent Updates +### Session 2026-03-12: Squad Output Consolidation -📌 Team initialized on 2026-03-05 +#### Responsibilities Completed +1. ✅ Orchestration logs for 4 agents (Kima, Bunk, Omar, Freamon) +2. ✅ Session log entry +3. ✅ Merged decision inbox files into decisions.md (4 major decisions documented) +4. ✅ Updated all agent history files with cross-agent findings +5. ✅ Git commit of .squad/ directory -## Learnings +#### Key Findings from Agent Reports + +**Signal Completeness Analysis (Omar):** +- Current: 32% (RSS + GitHub only) +- Phase 1: 50% (add Reddit, Dev.to, Stack Overflow) +- Phase 2: 65% (add YouTube) +- Phase 3: 82% (add social, podcasts, conferences) + +**Community Discovery Gaps (Bunk):** +- Reddit: High priority, 40-80 posts/week, medium effort +- Stack Overflow: Medium priority, 20-60 questions/week, low-medium effort +- X/Twitter: Deferred due to API cost ($5k/month) +- Discord: Deferred, requires server admin approval + +**Content Discovery Gaps (Kima):** +- RSS expansion: Low effort, immediate value +- YouTube: High value, requires API key +- Podcasts: Medium value, no auth required +- Conferences: Medium-high effort per conference + +**Architecture Recommendation (Freamon):** +- Implement SourceAdapter pattern before adding 6+ new sources +- Refactor timeline: 2.5 sessions (prevents technical debt) +- Phase 1: 2-3 sessions (quick wins) +- Phase 2: 2-3 sessions (YouTube, podcasts) +- Phase 3: 4-6 sessions (social, conferences, HN) + +#### Files Created/Modified +- `.squad/orchestration-log/2026-03-12T09-15-15Z-*.md` — 4 agent execution logs +- `.squad/log/2026-03-12T09-15-15Z-scribe.md` — Session log +- `.squad/decisions.md` — Merged 4 major decisions from inbox +- `.squad/agents/*/history.md` — Updated all 4 agent files with cross-team findings +- `.squad/decisions/inbox/` — Deleted after merge (managed by git) + +#### Cross-Agent Insights +- **Consensus:** Refactor → Phase 1 → Phase 2 → Phase 3 sequencing +- **Priority alignment:** Reddit > Stack Overflow > YouTube > Podcasts +- **Architecture:** SourceAdapter pattern unblocks scale to 20+ sources +- **Timeline:** Phase 1 implementation gives 50% signal completeness in 2-3 sessions -Initial setup complete. diff --git a/.squad/agents/stringer/history.md b/.squad/agents/stringer/history.md index 9d1a583..4927f78 100644 --- a/.squad/agents/stringer/history.md +++ b/.squad/agents/stringer/history.md @@ -11,3 +11,6 @@ ## Learnings - First session: team initialized 2026-03-05 +- **2025-07-18 — Issue #1 SDK Architecture Review:** Reviewed PRD for SourceAdapter pattern against Squad SDK (`@bradygaster/squad-sdk@0.8.24`) internals. Key finding: Squad SDK's `SkillSource`/`SkillSourceRegistry` patterns are the right structural inspiration, but the SDK itself should NOT be a runtime dependency (transitive deps too heavy, semantic mismatch between markdown skill loading and async I/O discovery). Recommended "inspired-by, not coupled-to" approach. Posted architectural review on Issue #1 with proposed interfaces, error handling strategy, and registration pattern. Decision doc written to `.squad/decisions/inbox/stringer-issue1-sdk-design.md`. +- **SDK patterns mapped to ACCES:** `SkillSource` → `SourceAdapter`, `SkillSourceRegistry` → `SourceRegistry`, `ErrorFactory.wrap()` → `AdapterError`, `EventBus` error isolation → `Promise.allSettled()`. Registration should be explicit (not filesystem scanning). Only discovery layer needs adapter pattern — other pipeline stages stay as pure functions. +- **Future bridge design:** A Squad SDK bridge (adapters → skills) is ~10 LOC via `defineSkill()`. Don't build until agents need to invoke discovery programmatically. diff --git a/.squad/decisions.md b/.squad/decisions.md index 4a22498..2763316 100644 --- a/.squad/decisions.md +++ b/.squad/decisions.md @@ -2,7 +2,93 @@ ## Active Decisions -No decisions recorded yet. +### Decision: Community Discovery Source Gap Analysis + +**Author:** Bunk (Source Scout — Community) +**Date:** 2026-03-06 +**Status:** Audit Complete — Ready for Implementation Planning + +**Summary:** +The ACCES engine currently implements GitHub discovery only, leaving 7 major community sources untouched. Reddit, Stack Overflow, Discord, GitHub Discussions, and social platforms represent significant blind spots in adoption signals. Current coverage: ~20% of community signal. + +**Key Recommendations:** +1. Implement GitHub Discussions (quick win, 30 min) +2. Implement Reddit JSON API (2 hours, high yield) +3. Implement Stack Overflow API (2 hours, high yield) +4. Defer X/Twitter (API costs ~$5k/month, low ROI) +5. Defer LinkedIn (scraping violates ToS, recommend link enrichment instead) +6. Defer Discord (requires server admin approval) + +**Implementation Roadmap:** +- **Phase 1 (Immediate):** GitHub Discussions, Reddit, Stack Overflow +- **Phase 2 (Medium-effort):** Bluesky, Mastodon +- **Phase 3 (Deferred):** X, LinkedIn, Discord + +--- + +### Decision: Signal Gap Analysis: What Beth Can't See (Yet) + +**Author:** Omar (Signal Analyst) +**Date:** 2026-03-05 +**Status:** Complete + +**Summary:** +ACCES currently captures only ~32% of the actual Aspire community conversation through RSS feeds and GitHub. Five critical signal blind spots leave Beth unable to detect adoption friction, confusion patterns, creator influence, viral moments, and industry narrative. + +**Current Signal Completeness Score: 32%** +- Blog Content: 85% | GitHub Activity: 90% | Tech Adoption: 40% +- Real Pain/Confusion: 15% | Community Mood: 5% | Decision-Maker Signals: 10% +- Creator Influence: 0% | Viral Moments: 0% | Industry Narrative: 20% + +**Critical Blind Spots (by priority):** +1. **Reddit** — Real-time community questions, pain signals, adoption stories (HIGH) +2. **YouTube** — Creator ecosystem, tutorial gaps, influencer reach (HIGH) +3. **Stack Overflow** — Eternal pain patterns, documentation gaps (MEDIUM-HIGH) +4. **Social Media (X, LinkedIn, Bluesky)** — Amplification, community mood (MEDIUM-HIGH) +5. **Podcasts** — Industry narrative, thought leadership (MEDIUM) + +**Projected Completeness with All Sources: 82%** + +--- + +### Decision: ACCES Source Expansion Roadmap + +**Author:** Freamon (Lead / Editor-in-Chief) +**Date:** 2026-03-05 +**Status:** Proposed for team review + +**Summary:** +Current ACCES architecture (monolithic discovery files) cannot scale beyond 3-4 sources. Proposes implementing SourceAdapter pattern with SourceRegistry for plug-and-play extensibility, followed by phased rollout of discovery sources. + +**Architecture Recommendation:** +- Implement SourceAdapter interface with centralized SourceRegistry +- Benefits: Clean separation, graceful degradation, parallel execution, easy testing, simple extension + +**Phased Rollout:** +- **Refactor** (2.5 sessions): Adapter pattern, no new dependencies +- **Phase 1** (2-3 sessions): Reddit, Dev.to, Stack Overflow — High value, zero auth +- **Phase 2** (2-3 sessions): YouTube, Podcasts — High value, requires API key config +- **Phase 3** (4-6 sessions): Bluesky, X RSS, HN, Conferences — Medium value, high effort + +**Recommended Sequence:** Refactor → Phase 1 → Phase 2 → Phase 3 (as-needed) + +--- + +### Decision: TypeScript Project Foundation + +**Author:** McNulty (TypeScript Implementer) +**Date:** 2025-07-18 +**Status:** Complete + +**Summary:** +Scaffolded ACCES TypeScript project with strict mode, ES2022 target, NodeNext module resolution. Minimal dependencies: tsx, typescript, @types/node, rss-parser, @octokit/rest. + +**Key Technical Decisions:** +- `(string & {})` for Channel type — allows known union + arbitrary strings without losing autocomplete +- Canonical ID = sha256 hash, 16 hex chars — deterministic, collision-resistant +- State serialized as JSON with Set→array and Map→object conversion (no external deps) +- All 9 report files generated in parallel via Promise.all +- Graceful error handling — partial discovery results kept even if one source fails ## Governance diff --git a/.squad/decisions/inbox/copilot-directive-2026-03-12T09-17.md b/.squad/decisions/inbox/copilot-directive-2026-03-12T09-17.md new file mode 100644 index 0000000..fa88388 --- /dev/null +++ b/.squad/decisions/inbox/copilot-directive-2026-03-12T09-17.md @@ -0,0 +1,9 @@ +### 2026-03-12T09:17: User directives +**By:** Brady (via Copilot) +**What:** +1. Use the Squad SDK for all new functionality — the adapter/source system must be pluggable using the Squad SDK. +2. File issues on the repo as PRDs and implementation plans before doing work. +3. Work on branches and PR back to main. +4. Team reviews PRs before merge — quality must meet Beth's standards. +5. Don't stop until Beth has a useful system covering all sources she needs. +**Why:** User request — captured for team memory diff --git a/.squad/decisions/inbox/freamon-prd-issues.md b/.squad/decisions/inbox/freamon-prd-issues.md new file mode 100644 index 0000000..f3022e4 --- /dev/null +++ b/.squad/decisions/inbox/freamon-prd-issues.md @@ -0,0 +1,76 @@ +# ACCES Source Expansion PRD Issues — Filed 2026-03-12 + +## Summary +Filed 7 GitHub issues on bradygaster/ACCES as PRDs for source expansion work. All issues labeled `squad` for Ralph's triage queue. + +## Issues Created (Dependency Order) + +### Foundation (Prerequisite) +- **Issue #1:** [Architecture Refactor: SourceAdapter Pattern + SourceRegistry](https://github.com/bradygaster/ACCES/issues/1) + - Assigned: McNulty + Stringer + - Effort: 2.5 sessions + - Blocks: All Phase 1 & Phase 2 work + +### Phase 1 (High Value, Zero Auth) +- **Issue #2:** [Reddit Discovery Adapter](https://github.com/bradygaster/ACCES/issues/2) + - Assigned: Bunk + - Effort: 1 session + - Expected yield: 40-80 posts/week + - Depends on: #1 + +- **Issue #3:** [Dev.to API Discovery Adapter](https://github.com/bradygaster/ACCES/issues/3) + - Assigned: Kima + - Effort: 1 session + - Expected yield: 20-40 articles/week + - Depends on: #1 + +- **Issue #4:** [Stack Overflow Discovery Adapter](https://github.com/bradygaster/ACCES/issues/4) + - Assigned: Bunk + - Effort: 1 session + - Expected yield: 20-60 questions/week + - Depends on: #1 + +- **Issue #5:** [GitHub Discussions Discovery Enhancement](https://github.com/bradygaster/ACCES/issues/5) + - Assigned: Bunk + - Effort: 0.5 sessions + - Expected yield: 10-30 discussions/week + - Depends on: #1 + +### Phase 2 (High Value, API Key Required) +- **Issue #6:** [YouTube Discovery Adapter](https://github.com/bradygaster/ACCES/issues/6) + - Assigned: Kima + - Effort: 1.5 sessions + - Expected yield: 30-60 videos/month + - Requires: YOUTUBE_API_KEY environment variable + - Depends on: #1 + +- **Issue #7:** [Podcast Discovery Adapter](https://github.com/bradygaster/ACCES/issues/7) + - Assigned: Kima + - Effort: 1 session + - Expected yield: 5-15 episodes/month + - Depends on: #1 + +## Total Effort Estimate +- **Foundation:** 2.5 sessions +- **Phase 1:** 3.5 sessions (4 adapters) +- **Phase 2:** 2.5 sessions (2 adapters) +- **Total:** 8.5 sessions + +## Signal Completeness Roadmap +- **Current (RSS + GitHub):** 32% signal completeness +- **After Refactor:** Foundation ready, no signal improvement +- **After Phase 1:** ~50% signal completeness (+Reddit, Dev.to, Stack Overflow, Discussions) +- **After Phase 2:** ~65% signal completeness (+YouTube, Podcasts) + +## Next Steps +1. Ralph triages issues and assigns to squad members +2. McNulty + Stringer begin Issue #1 (architecture refactor) immediately +3. Phase 1 adapters parallelized after #1 completes (Bunk + Kima work simultaneously) +4. Phase 2 follows after Phase 1 validation with Beth +5. Each issue becomes a branch → PR → review → merge workflow + +--- + +**Author:** Freamon +**Date:** 2026-03-12 +**Status:** PRDs filed, awaiting triage diff --git a/.squad/decisions/inbox/mcnulty-issue1-plan.md b/.squad/decisions/inbox/mcnulty-issue1-plan.md new file mode 100644 index 0000000..70952da --- /dev/null +++ b/.squad/decisions/inbox/mcnulty-issue1-plan.md @@ -0,0 +1,39 @@ +# Decision: Implementation Plan for Issue #1 — SourceAdapter Pattern + SourceRegistry + +**Author:** McNulty (TypeScript Implementer) +**Date:** 2025-07-18 +**Status:** Proposed — posted to [Issue #1](https://github.com/bradygaster/ACCES/issues/1#issuecomment-4045230035) + +## Summary + +Reviewed the PRD for Issue #1 and produced a detailed implementation plan covering: + +1. **SourceAdapter interface** — `getName()`, `validate()`, `discover(state)` contract +2. **SourceRegistry class** — auto-discovery, parallel execution via `Promise.all`, graceful degradation with per-adapter try/catch +3. **AdapterResult type** — structured output per adapter (name, results, error, duration) +4. **Two concrete adapters:** `RSSSourceAdapter` (from content.ts) and `GitHubSourceAdapter` (from community.ts) +5. **Shared helpers module** — consolidates `generateCanonicalId`, `truncate`, keyword constants +6. **Backward-compat facades** — content.ts and community.ts become thin wrappers during transition +7. **5-step migration** — additive-only first, wire registry, facade, cleanup, verify parity + +## PRD Gaps Identified + +- Duplicate `generateCanonicalId` in content.ts and taxonomy.ts — needs consolidation +- Duplicate `truncate` helper — needs single source +- Skeleton functions (blog search, YouTube, Reddit, social media) — plan removes them cleanly +- No logging strategy specified — plan keeps adapters pure, registry handles output + +## Key Decisions + +- Adapters own their constants (RSS_FEEDS, GITHUB_QUERIES stay private) +- Registry validates all adapters in parallel before executing +- Facades preserve import compatibility during transition +- No new dependencies (per PRD requirement) + +## Estimated Effort + +~2.5-3 hours total across 5 migration steps. + +## Full Plan + +Posted as comment on Issue #1: https://github.com/bradygaster/ACCES/issues/1#issuecomment-4045230035 diff --git a/.squad/decisions/inbox/mcnulty-ts-scaffold.md b/.squad/decisions/inbox/mcnulty-ts-scaffold.md deleted file mode 100644 index 8a5a237..0000000 --- a/.squad/decisions/inbox/mcnulty-ts-scaffold.md +++ /dev/null @@ -1,21 +0,0 @@ -# Decision: TypeScript Project Foundation - -**Author:** McNulty (TypeScript Implementer) -**Date:** 2025-07-18 - -## What - -Scaffolded the full ACCES TypeScript project with strict mode, ES2022 target, NodeNext module resolution. Minimal dependencies: `tsx`, `typescript`, `@types/node`, `rss-parser`, `@octokit/rest`. - -## Why - -The team needs a buildable, runnable foundation. Every module has typed interfaces, the pipeline is wired end-to-end, and `npm start` runs the full engine immediately. Skeleton functions are clearly marked for scouts to fill. - -## Key decisions - -- **`(string & {})` for Channel type** — allows known union values + arbitrary strings without losing autocomplete -- **Canonical ID = sha256 hash, 16 hex chars** — deterministic, collision-resistant, human-inspectable -- **State serialized as JSON** with Set→array and Map→object conversion (no external deps) -- **rss-parser** for RSS (Kima's scout work), **@octokit/rest** for GitHub (Bunk's scout work) -- **All 9 report files** generated in parallel via Promise.all for speed -- **Graceful error handling** — partial discovery results are kept even if one source fails diff --git a/.squad/decisions/inbox/stringer-issue1-sdk-design.md b/.squad/decisions/inbox/stringer-issue1-sdk-design.md new file mode 100644 index 0000000..dbb5404 --- /dev/null +++ b/.squad/decisions/inbox/stringer-issue1-sdk-design.md @@ -0,0 +1,91 @@ +# Decision: Squad SDK Integration Pattern for SourceAdapter Refactor + +**Author:** Stringer (Squad SDK Orchestrator) +**Date:** 2025-07-18 +**Status:** Proposed — Architectural review of Issue #1 +**Issue:** [#1 Architecture Refactor: SourceAdapter Pattern + SourceRegistry](https://github.com/bradygaster/ACCES/issues/1) + +## Context + +Brady requested that the SourceAdapter pattern "use the Squad SDK for the functionality" and be "a pluggable system using our SDK." I reviewed the actual Squad SDK (`@bradygaster/squad-sdk@0.8.24`) internals against the PRD requirements for Issue #1. + +## Decision: Inspired-By, Not Coupled-To + +**Mirror Squad SDK patterns in standalone TypeScript interfaces. Do not add `@bradygaster/squad-sdk` as a runtime dependency.** + +### Rationale + +1. **PRD constraint:** "No new external dependencies." Squad SDK transitively brings `@github/copilot-sdk` + `vscode-jsonrpc` — too heavy for a CLI content engine. +2. **Semantic mismatch:** Squad's `SkillSource` loads markdown knowledge files. ACCES adapters execute async I/O against external APIs. Same shape, different semantics. +3. **Future-proof:** By mirroring the contract, a Squad SDK bridge is trivial (~10 LOC) if we later want agents to invoke discovery. + +### Squad SDK Patterns to Mirror + +| SDK Pattern | ACCES Pattern | Key Methods | +|---|---|---| +| `SkillSource` | `SourceAdapter` | `name`, `validate()`, `discover()` | +| `SkillSourceRegistry` | `SourceRegistry` | `register()`, `validateAll()`, `discoverAll()` | +| `ErrorFactory.wrap()` + `SquadError` | `AdapterError` | Severity, recoverability, original error wrapping | +| `EventBus` error isolation | `Promise.allSettled()` in registry | Per-adapter isolation, pipeline continues on failure | + +### Interface Design + +```typescript +export interface SourceAdapter { + readonly name: string; // kebab-case, Squad convention + readonly displayName: string; + readonly channel: Channel; + validate(): Promise; + discover(state: RunState): Promise; +} + +export interface AdapterValidation { + valid: boolean; + reason?: string; + warnings?: string[]; +} + +export class SourceRegistry { + register(adapter: SourceAdapter): void; + list(): string[]; + validateAll(): Promise>; + discoverAll(state: RunState): Promise; +} + +export interface RegistryResult { + results: DiscoveryResult[]; + errors: Map; + skipped: string[]; + timing: Map; +} +``` + +### Registration Strategy + +Explicit `register()` calls, not filesystem auto-discovery. Squad's `loadSkillsFromDirectory()` works for markdown; TypeScript adapters need explicit imports for type safety and tree-shaking. + +### Scope Boundary + +Only the discovery layer gets the adapter pattern. Dedupe, classify, analyze, and output remain pure functions. Don't over-engineer stages that don't need pluggability yet. + +### What Changes in index.ts + +Before: Two hardcoded `discoverContent()` / `discoverCommunity()` calls with manual error handling. + +After: `createDefaultRegistry()` → `registry.discoverAll(state)` — single call, parallel execution, automatic error isolation. + +## Implications + +- **McNulty:** Implement `SourceAdapter` interface, `SourceRegistry` class, extract `RSSSourceAdapter` + `GitHubSourceAdapter` from existing code +- **Kima/Bunk:** Phase 1 adapters (Reddit, Dev.to, Stack Overflow) implement `SourceAdapter` — no changes to orchestrator +- **Future:** Squad SDK bridge is ~10 LOC when needed — don't build until an agent needs to invoke discovery programmatically + +## Alternatives Considered + +1. **Tight coupling (import Squad SDK):** Rejected — violates "no new deps" constraint, brings heavyweight transitive deps +2. **Adapters AS Squad skills:** Rejected for now — skills are knowledge injection, not I/O execution; bridge later +3. **Filesystem auto-discovery:** Rejected — not type-safe, breaks tree-shaking, debugging harder + +--- + +*Comment posted on Issue #1: https://github.com/bradygaster/ACCES/issues/1#issuecomment-4045233243* diff --git a/src/discovery/adapters/github.ts b/src/discovery/adapters/github.ts new file mode 100644 index 0000000..f1e1a9e --- /dev/null +++ b/src/discovery/adapters/github.ts @@ -0,0 +1,211 @@ +/** + * GitHubSourceAdapter — GitHub search discovery implementation. + * Extracted from community.ts with no functional changes. + */ + +import { Octokit } from '@octokit/rest'; +import type { Channel, ContentItem, ContentType, DiscoveryResult, RunState, Signal } from '../../types.js'; +import type { AdapterValidation, SourceAdapter } from './types.js'; +import { generateCanonicalId, truncate } from './helpers.js'; + +const GITHUB_QUERIES: readonly string[] = [ + 'aspire apphost', + 'aspire dashboard', + 'dotnet aspire', + 'aspire service discovery', + 'aspire integration', +] as const; + +export class GitHubSourceAdapter implements SourceAdapter { + readonly name = 'github'; + readonly displayName = 'GitHub'; + readonly channel: Channel = 'github'; + + async validate(): Promise { + const token = process.env['GITHUB_TOKEN']; + if (!token) { + return { + valid: true, + warnings: ['GITHUB_TOKEN not set - using unauthenticated API (lower rate limits)'], + }; + } + return { valid: true }; + } + + async discover(state: RunState): Promise { + const token = process.env['GITHUB_TOKEN']; + const octokit = new Octokit(token ? { auth: token } : {}); + const results: DiscoveryResult[] = []; + const sinceDate = state.last_run; + + for (const query of GITHUB_QUERIES) { + try { + console.log(` 🔍 GitHub search: "${query}"`); + + const repoResults = await searchGitHubRepos(octokit, query, sinceDate); + if (repoResults.items.length > 0) { + results.push(repoResults); + } + + const issueResults = await searchGitHubIssues(octokit, query, sinceDate); + if (issueResults.items.length > 0) { + results.push(issueResults); + } + } catch (err) { + const message = err instanceof Error ? err.message : String(err); + console.warn(` ⚠️ GitHub search "${query}" failed: ${message}`); + } + } + + return results; + } +} + +async function searchGitHubRepos( + octokit: Octokit, + query: string, + since: string, +): Promise { + const sinceDate = since.split('T')[0] ?? since; + const searchQuery = `${query} pushed:>=${sinceDate}`; + + const response = await octokit.search.repos({ + q: searchQuery, + sort: 'updated', + order: 'desc', + per_page: 30, + }); + + const items: ContentItem[] = response.data.items.map((repo) => { + const type: ContentType = inferGitHubContentType(repo.description ?? ''); + return { + canonical_id: generateCanonicalId( + repo.full_name, + repo.html_url, + repo.owner?.login ?? null, + repo.pushed_at, + ), + title: repo.full_name, + url: repo.html_url, + type, + channel: 'github', + published_at: repo.created_at, + author: repo.owner?.login ?? null, + summary: truncate(repo.description ?? 'No description', 300), + tags: { + topic: extractGitHubTopics(repo.topics ?? [], repo.description ?? ''), + audience: ['intermediate'], + signal: ['adoption'], + confidence: 'medium', + actionability: 'investigate', + }, + provenance: { + discovered_from: 'github:repos', + discovered_query: query, + source_first_seen: new Date().toISOString(), + raw_evidence_path: null, + }, + dedupe: { + is_duplicate: false, + duplicate_of: null, + duplicate_reason: null, + }, + }; + }); + + return { items, source: 'github:repos', query }; +} + +async function searchGitHubIssues( + octokit: Octokit, + query: string, + since: string, +): Promise { + const sinceDate = since.split('T')[0] ?? since; + const searchQuery = `${query} created:>=${sinceDate}`; + + const response = await octokit.search.issuesAndPullRequests({ + q: searchQuery, + sort: 'created', + order: 'desc', + per_page: 30, + }); + + const items: ContentItem[] = response.data.items.map((issue) => { + const type: ContentType = issue.pull_request ? 'repo' : 'issue'; + return { + canonical_id: generateCanonicalId( + issue.title, + issue.html_url, + issue.user?.login ?? null, + issue.created_at, + ), + title: issue.title, + url: issue.html_url, + type, + channel: 'github', + published_at: issue.created_at, + author: issue.user?.login ?? null, + summary: truncate(issue.body ?? 'No description', 300), + tags: { + topic: extractGitHubTopics(issue.labels + .map((l) => (typeof l === 'string' ? l : l.name ?? '')) + .filter(Boolean), issue.title), + audience: ['intermediate'], + signal: inferIssueSignal(issue.title, issue.body ?? ''), + confidence: 'medium', + actionability: 'investigate', + }, + provenance: { + discovered_from: 'github:issues', + discovered_query: query, + source_first_seen: new Date().toISOString(), + raw_evidence_path: null, + }, + dedupe: { + is_duplicate: false, + duplicate_of: null, + duplicate_reason: null, + }, + }; + }); + + return { items, source: 'github:issues', query }; +} + +function inferGitHubContentType(description: string): ContentType { + const lower = description.toLowerCase(); + if (lower.includes('sample') || lower.includes('example') || lower.includes('template')) return 'sample'; + if (lower.includes('release')) return 'release'; + return 'repo'; +} + +function extractGitHubTopics(topics: string[], description: string): string[] { + const text = [...topics, description].join(' ').toLowerCase(); + const topicMap: Record = { + apphost: ['apphost', 'app-host'], + dashboard: ['dashboard'], + integrations: ['integration'], + k8s: ['kubernetes', 'k8s'], + aca: ['azure-container-app', 'aca'], + otel: ['opentelemetry', 'otel'], + dotnet: ['.net', 'dotnet', 'csharp'], + docker: ['docker', 'container'], + }; + + const found: string[] = []; + for (const [topic, keywords] of Object.entries(topicMap)) { + if (keywords.some((k) => text.includes(k))) { + found.push(topic); + } + } + return found.length > 0 ? found : ['aspire']; +} + +function inferIssueSignal(title: string, body: string): Signal[] { + const text = `${title} ${body}`.toLowerCase(); + if (text.includes('bug') || text.includes('error') || text.includes('crash')) return ['complaint']; + if (text.includes('feature') || text.includes('request') || text.includes('please add')) return ['request']; + if (text.includes('how to') || text.includes('help') || text.includes('question')) return ['confusion']; + return ['other']; +} diff --git a/src/discovery/adapters/helpers.ts b/src/discovery/adapters/helpers.ts new file mode 100644 index 0000000..d292a8b --- /dev/null +++ b/src/discovery/adapters/helpers.ts @@ -0,0 +1,76 @@ +/** + * Shared helpers for discovery adapters. + * Consolidates ID generation, keyword matching, and text utilities. + */ + +import { createHash } from 'node:crypto'; + +export const SEARCH_KEYWORDS: readonly string[] = [ + 'Aspire dev', + 'Aspire 9', + 'Aspire 9.1', + 'Aspire 9.2', + 'Aspire 9.3', + 'Aspirified', + 'Aspire AppHost', + 'Aspire dashboard', + 'Aspire manifest', + 'Aspire service discovery', + 'Aspire .NET', + 'Aspire dotnet', + 'Aspire C#', + 'Aspire csharp', + 'Aspire CLI', + 'Aspire javascript', + 'Aspire python', + 'Aspire azure', + 'Aspire aws', + 'Aspire deploy', + 'Aspire docker', + 'Aspire distributed', + 'Aspire app', + 'Aspire code', + 'Aspire kubernetes', + 'Aspire aca', + 'Aspire redis', + 'Aspire otel', +] as const; + +export const EXCLUSION_KEYWORDS: readonly string[] = [ + 'aspirelearning', + 'aspiremag', + 'buildinpublic', + '#openenrollment', + '#aspirepublicschools', + '#aspirelosangeles', +] as const; + +export function generateCanonicalId( + title: string, + url: string, + author: string | null, + date: string | null, +): string { + const input = [title, url, author ?? '', date ?? ''].join('|').toLowerCase(); + return createHash('sha256').update(input).digest('hex').slice(0, 16); +} + +export function isAspireRelated(text: string): boolean { + const lower = text.toLowerCase(); + const markers = [ + 'aspire', 'apphost', 'aspire dashboard', + 'service discovery', 'aspire manifest', + '.net aspire', 'dotnet aspire', + ]; + return markers.some((m) => lower.includes(m)); +} + +export function isExcluded(text: string): boolean { + const lower = text.toLowerCase(); + return EXCLUSION_KEYWORDS.some((k) => lower.includes(k.toLowerCase())); +} + +export function truncate(str: string, maxLen: number): string { + if (str.length <= maxLen) return str; + return str.slice(0, maxLen - 3) + '...'; +} diff --git a/src/discovery/adapters/index.ts b/src/discovery/adapters/index.ts new file mode 100644 index 0000000..c72d3bc --- /dev/null +++ b/src/discovery/adapters/index.ts @@ -0,0 +1,21 @@ +/** + * Adapters module — pluggable discovery source pattern. + * Export all adapters and provide a default registry factory. + */ + +export * from './types.js'; +export * from './helpers.js'; +export { SourceRegistry } from './registry.js'; +export { RSSSourceAdapter } from './rss.js'; +export { GitHubSourceAdapter } from './github.js'; + +import { SourceRegistry } from './registry.js'; +import { RSSSourceAdapter } from './rss.js'; +import { GitHubSourceAdapter } from './github.js'; + +export function createDefaultRegistry(): SourceRegistry { + const registry = new SourceRegistry(); + registry.register(new RSSSourceAdapter()); + registry.register(new GitHubSourceAdapter()); + return registry; +} diff --git a/src/discovery/adapters/registry.ts b/src/discovery/adapters/registry.ts new file mode 100644 index 0000000..1ab57e9 --- /dev/null +++ b/src/discovery/adapters/registry.ts @@ -0,0 +1,96 @@ +/** + * SourceRegistry — central registry for discovery adapters. + * Executes adapters in parallel via Promise.allSettled for graceful degradation. + */ + +import type { DiscoveryResult, RunState } from '../../types.js'; +import type { AdapterValidation, RegistryResult, SourceAdapter } from './types.js'; + +export class SourceRegistry { + private adapters: Map = new Map(); + + register(adapter: SourceAdapter): void { + if (this.adapters.has(adapter.name)) { + throw new Error(`Adapter "${adapter.name}" is already registered`); + } + this.adapters.set(adapter.name, adapter); + } + + list(): string[] { + return Array.from(this.adapters.keys()); + } + + async validateAll(): Promise> { + const results = new Map(); + const promises = Array.from(this.adapters.entries()).map(async ([name, adapter]) => { + try { + const validation = await adapter.validate(); + return { name, validation }; + } catch (err) { + const message = err instanceof Error ? err.message : String(err); + return { name, validation: { valid: false, reason: `Validation threw: ${message}` } }; + } + }); + + const settled = await Promise.allSettled(promises); + for (const result of settled) { + if (result.status === 'fulfilled') { + results.set(result.value.name, result.value.validation); + } + } + + return results; + } + + async discoverAll(state: RunState): Promise { + const allResults: DiscoveryResult[] = []; + const errors = new Map(); + const skipped: string[] = []; + const timing = new Map(); + + const validations = await this.validateAll(); + + // Skip invalid adapters before execution + const validAdapters: [string, SourceAdapter][] = []; + for (const [name, adapter] of this.adapters.entries()) { + const validation = validations.get(name); + if (!validation || !validation.valid) { + skipped.push(name); + console.log(` ⏭️ Skipping ${adapter.displayName}: ${validation?.reason ?? 'validation failed'}`); + continue; + } + if (validation.warnings && validation.warnings.length > 0) { + for (const warning of validation.warnings) { + console.log(` ⚠️ ${adapter.displayName}: ${warning}`); + } + } + validAdapters.push([name, adapter]); + } + + // Execute valid adapters in parallel — Promise.allSettled handles isolation + const startTimes = new Map(); + const promises = validAdapters.map(([name, adapter]) => { + startTimes.set(name, Date.now()); + return adapter.discover(state).then(results => ({ name, results })); + }); + + const settled = await Promise.allSettled(promises); + + for (let i = 0; i < settled.length; i++) { + const result = settled[i]; + const [name, adapter] = validAdapters[i]; + const elapsed = Date.now() - (startTimes.get(name) ?? Date.now()); + timing.set(name, elapsed); + + if (result.status === 'fulfilled') { + allResults.push(...result.value.results); + } else { + const error = result.reason instanceof Error ? result.reason : new Error(String(result.reason)); + errors.set(name, error); + console.error(` ❌ ${adapter.displayName} failed: ${error.message}`); + } + } + + return { results: allResults, errors, skipped, timing }; + } +} diff --git a/src/discovery/adapters/rss.ts b/src/discovery/adapters/rss.ts new file mode 100644 index 0000000..e442279 --- /dev/null +++ b/src/discovery/adapters/rss.ts @@ -0,0 +1,143 @@ +/** + * RSSSourceAdapter — RSS feed discovery implementation. + * Extracted from content.ts with no functional changes. + */ + +import RssParser from 'rss-parser'; +import type { Channel, ContentItem, ContentType, DiscoveryResult, RunState } from '../../types.js'; +import type { AdapterValidation, SourceAdapter } from './types.js'; +import { generateCanonicalId, isAspireRelated, isExcluded, truncate } from './helpers.js'; + +const RSS_FEEDS: readonly { url: string; channel: Channel; name: string }[] = [ + { url: 'https://devblogs.microsoft.com/dotnet/feed/', channel: 'personal_blog', name: '.NET Blog' }, + { url: 'https://devblogs.microsoft.com/aspnet/feed/', channel: 'personal_blog', name: 'ASP.NET Blog' }, + { url: 'https://dev.to/feed/tag/dotnetaspire', channel: 'devto', name: 'Dev.to #dotnetaspire' }, + { url: 'https://dev.to/feed/tag/aspire', channel: 'devto', name: 'Dev.to #aspire' }, +] as const; + +export class RSSSourceAdapter implements SourceAdapter { + readonly name = 'rss-feeds'; + readonly displayName = 'RSS Feeds'; + readonly channel: Channel = 'rss'; + + async validate(): Promise { + if (RSS_FEEDS.length === 0) { + return { valid: false, reason: 'No RSS feeds configured' }; + } + return { valid: true }; + } + + async discover(state: RunState): Promise { + const parser = new RssParser(); + const results: DiscoveryResult[] = []; + const sinceDate = new Date(state.last_run); + + for (const feed of RSS_FEEDS) { + try { + console.log(` 📡 Fetching RSS: ${feed.name}`); + const parsed = await parser.parseURL(feed.url); + + const items: ContentItem[] = []; + for (const entry of parsed.items ?? []) { + const pubDate = entry.pubDate ? new Date(entry.pubDate) : null; + + if (pubDate && pubDate < sinceDate) continue; + + const title = entry.title ?? 'Untitled'; + const url = entry.link ?? ''; + if (!url) continue; + + const text = `${title} ${entry.contentSnippet ?? ''} ${entry.content ?? ''}`.toLowerCase(); + if (!isAspireRelated(text)) continue; + if (isExcluded(text)) continue; + + const canonicalId = generateCanonicalId(title, url, entry.creator ?? null, entry.pubDate ?? null); + + items.push({ + canonical_id: canonicalId, + title, + url, + type: inferContentType(url, text), + channel: feed.channel, + published_at: entry.pubDate ?? null, + author: entry.creator ?? entry['dc:creator'] as string ?? null, + summary: truncate(entry.contentSnippet ?? '', 300), + tags: { + topic: extractTopics(text), + audience: ['intermediate'], + signal: ['other'], + confidence: 'medium', + actionability: 'investigate', + }, + provenance: { + discovered_from: `rss:${feed.name}`, + discovered_query: null, + source_first_seen: new Date().toISOString(), + raw_evidence_path: null, + }, + dedupe: { + is_duplicate: false, + duplicate_of: null, + duplicate_reason: null, + }, + }); + } + + if (items.length > 0) { + results.push({ items, source: `rss:${feed.name}` }); + } + } catch (err) { + const message = err instanceof Error ? err.message : String(err); + console.warn(` ⚠️ RSS feed ${feed.name} failed: ${message}`); + } + } + + return results; + } +} + +function inferContentType(url: string, text: string): ContentType { + const lower = url.toLowerCase(); + if (lower.includes('youtube.com') || lower.includes('youtu.be')) return 'video'; + if (lower.includes('github.com')) { + if (lower.includes('/releases/')) return 'release'; + if (lower.includes('/issues/')) return 'issue'; + if (lower.includes('/discussions/')) return 'discussion'; + return 'repo'; + } + if (lower.includes('reddit.com')) return 'reddit'; + if (text.includes('sample') || text.includes('template')) return 'sample'; + return 'blog'; +} + +function extractTopics(text: string): string[] { + const topicKeywords: Record = { + apphost: ['apphost', 'app host'], + dashboard: ['dashboard'], + integrations: ['integration'], + k8s: ['kubernetes', 'k8s'], + aca: ['azure container app', 'aca'], + otel: ['opentelemetry', 'otel'], + postgres: ['postgres', 'postgresql'], + redis: ['redis'], + dapr: ['dapr'], + auth: ['auth', 'authentication', 'identity'], + caching: ['cache', 'caching'], + dotnet: ['.net', 'dotnet', 'c#', 'csharp'], + typescript: ['typescript'], + python: ['python'], + docker: ['docker', 'container'], + deploy: ['deploy', 'deployment'], + }; + + const found: string[] = []; + const lower = text.toLowerCase(); + + for (const [topic, keywords] of Object.entries(topicKeywords)) { + if (keywords.some((k) => lower.includes(k))) { + found.push(topic); + } + } + + return found.length > 0 ? found : ['aspire']; +} diff --git a/src/discovery/adapters/types.ts b/src/discovery/adapters/types.ts new file mode 100644 index 0000000..08acc34 --- /dev/null +++ b/src/discovery/adapters/types.ts @@ -0,0 +1,27 @@ +/** + * SourceAdapter pattern types — inspired by Squad SDK SkillSource pattern. + * Enables pluggable discovery sources with graceful degradation. + */ + +import type { Channel, DiscoveryResult, RunState } from '../../types.js'; + +export interface AdapterValidation { + valid: boolean; + reason?: string; + warnings?: string[]; +} + +export interface SourceAdapter { + readonly name: string; + readonly displayName: string; + readonly channel: Channel; + validate(): Promise; + discover(state: RunState): Promise; +} + +export interface RegistryResult { + results: DiscoveryResult[]; + errors: Map; + skipped: string[]; + timing: Map; +} diff --git a/src/discovery/community.ts b/src/discovery/community.ts index 07ad2f4..be1c0eb 100644 --- a/src/discovery/community.ts +++ b/src/discovery/community.ts @@ -1,266 +1,27 @@ /** - * Community discovery module — GitHub search, Reddit, social media. - * - * GitHub search is implemented using @octokit/rest. - * Reddit and social media are skeletons for Bunk and the scouts. + * Community discovery module — backward compatibility facade. + * Redirects to adapter-based discovery. GitHub logic moved to adapters/github.ts. */ -import { Octokit } from '@octokit/rest'; -import { generateCanonicalId } from './content.js'; -import type { - ContentItem, - ContentType, - DiscoveryResult, - RunState, -} from '../types.js'; - -// ─── GitHub search queries from ACCES.md §6 ──────────────────────────────── - -const GITHUB_QUERIES: readonly string[] = [ - 'aspire apphost', - 'aspire dashboard', - 'dotnet aspire', - 'aspire service discovery', - 'aspire integration', -] as const; +import { GitHubSourceAdapter } from './adapters/github.js'; +import type { DiscoveryResult, RunState } from '../types.js'; /** - * Run all community discovery sources and return combined results. + * @deprecated Use createDefaultRegistry().discoverAll() instead. */ export async function discoverCommunity(state: RunState): Promise { - const results: DiscoveryResult[] = []; - - // GitHub search — implemented with Octokit - const githubResults = await discoverFromGitHub(state); - results.push(...githubResults); - - // Reddit search — skeleton - const redditResults = await discoverFromReddit(state); - results.push(...redditResults); - - // Social media — skeleton - const socialResults = await discoverFromSocialMedia(state); - results.push(...socialResults); - - return results; -} - -// ─── GitHub discovery (implemented) ───────────────────────────────────────── - -/** - * Search GitHub for Aspire-related repos and issues using @octokit/rest. - * Uses GITHUB_TOKEN from environment if available for higher rate limits. - */ -async function discoverFromGitHub(state: RunState): Promise { - const token = process.env['GITHUB_TOKEN']; - const octokit = new Octokit(token ? { auth: token } : {}); - const results: DiscoveryResult[] = []; - const sinceDate = state.last_run; - - for (const query of GITHUB_QUERIES) { - try { - console.log(` 🔍 GitHub search: "${query}"`); - - // Search repositories - const repoResults = await searchGitHubRepos(octokit, query, sinceDate); - if (repoResults.items.length > 0) { - results.push(repoResults); - } - - // Search issues/discussions - const issueResults = await searchGitHubIssues(octokit, query, sinceDate); - if (issueResults.items.length > 0) { - results.push(issueResults); - } - } catch (err) { - const message = err instanceof Error ? err.message : String(err); - console.warn(` ⚠️ GitHub search "${query}" failed: ${message}`); - } + const adapter = new GitHubSourceAdapter(); + const validation = await adapter.validate(); + if (!validation.valid) { + console.warn(`⚠️ GitHub adapter validation failed: ${validation.reason}`); + return []; } - - return results; -} - -/** Search GitHub repos matching a query, created/pushed since last run */ -async function searchGitHubRepos( - octokit: Octokit, - query: string, - since: string, -): Promise { - const sinceDate = since.split('T')[0] ?? since; - const searchQuery = `${query} pushed:>=${sinceDate}`; - - const response = await octokit.search.repos({ - q: searchQuery, - sort: 'updated', - order: 'desc', - per_page: 30, - }); - - const items: ContentItem[] = response.data.items.map((repo) => { - const type: ContentType = inferGitHubContentType(repo.description ?? ''); - return { - canonical_id: generateCanonicalId( - repo.full_name, - repo.html_url, - repo.owner?.login ?? null, - repo.pushed_at, - ), - title: repo.full_name, - url: repo.html_url, - type, - channel: 'github', - published_at: repo.created_at, - author: repo.owner?.login ?? null, - summary: truncate(repo.description ?? 'No description', 300), - tags: { - topic: extractGitHubTopics(repo.topics ?? [], repo.description ?? ''), - audience: ['intermediate'], - signal: ['adoption'], - confidence: 'medium', - actionability: 'investigate', - }, - provenance: { - discovered_from: 'github:repos', - discovered_query: query, - source_first_seen: new Date().toISOString(), - raw_evidence_path: null, - }, - dedupe: { - is_duplicate: false, - duplicate_of: null, - duplicate_reason: null, - }, - }; - }); - - return { items, source: 'github:repos', query }; -} - -/** Search GitHub issues matching a query */ -async function searchGitHubIssues( - octokit: Octokit, - query: string, - since: string, -): Promise { - const sinceDate = since.split('T')[0] ?? since; - const searchQuery = `${query} created:>=${sinceDate}`; - - const response = await octokit.search.issuesAndPullRequests({ - q: searchQuery, - sort: 'created', - order: 'desc', - per_page: 30, - }); - - const items: ContentItem[] = response.data.items.map((issue) => { - const type: ContentType = issue.pull_request ? 'repo' : 'issue'; - return { - canonical_id: generateCanonicalId( - issue.title, - issue.html_url, - issue.user?.login ?? null, - issue.created_at, - ), - title: issue.title, - url: issue.html_url, - type, - channel: 'github', - published_at: issue.created_at, - author: issue.user?.login ?? null, - summary: truncate(issue.body ?? 'No description', 300), - tags: { - topic: extractGitHubTopics(issue.labels - .map((l) => (typeof l === 'string' ? l : l.name ?? '')) - .filter(Boolean), issue.title), - audience: ['intermediate'], - signal: inferIssueSignal(issue.title, issue.body ?? ''), - confidence: 'medium', - actionability: 'investigate', - }, - provenance: { - discovered_from: 'github:issues', - discovered_query: query, - source_first_seen: new Date().toISOString(), - raw_evidence_path: null, - }, - dedupe: { - is_duplicate: false, - duplicate_of: null, - duplicate_reason: null, - }, - }; - }); - - return { items, source: 'github:issues', query }; -} - -// ─── Reddit search (skeleton) ─────────────────────────────────────────────── - -/** - * Search Reddit for Aspire-related posts. - * TODO: Implement Reddit API search in r/dotnet, r/csharp, etc. - */ -async function discoverFromReddit(_state: RunState): Promise { - console.log(' 💬 Reddit search: not yet implemented (skeleton)'); - return []; -} - -// ─── Social media (skeleton) ──────────────────────────────────────────────── - -/** - * Search social media (X, LinkedIn, Bluesky) for Aspire mentions. - * TODO: Implement social media scrapers/API clients. - */ -async function discoverFromSocialMedia(_state: RunState): Promise { - console.log(' 📱 Social media search: not yet implemented (skeleton)'); - return []; -} - -// ─── Helpers ──────────────────────────────────────────────────────────────── - -/** Infer content type from a GitHub repo description */ -function inferGitHubContentType(description: string): ContentType { - const lower = description.toLowerCase(); - if (lower.includes('sample') || lower.includes('example') || lower.includes('template')) return 'sample'; - if (lower.includes('release')) return 'release'; - return 'repo'; -} - -/** Extract topic tags from GitHub topics and description */ -function extractGitHubTopics(topics: string[], description: string): string[] { - const text = [...topics, description].join(' ').toLowerCase(); - const topicMap: Record = { - apphost: ['apphost', 'app-host'], - dashboard: ['dashboard'], - integrations: ['integration'], - k8s: ['kubernetes', 'k8s'], - aca: ['azure-container-app', 'aca'], - otel: ['opentelemetry', 'otel'], - dotnet: ['.net', 'dotnet', 'csharp'], - docker: ['docker', 'container'], - }; - - const found: string[] = []; - for (const [topic, keywords] of Object.entries(topicMap)) { - if (keywords.some((k) => text.includes(k))) { - found.push(topic); + + if (validation.warnings) { + for (const warning of validation.warnings) { + console.warn(`⚠️ ${warning}`); } } - return found.length > 0 ? found : ['aspire']; -} - -/** Infer signal type from issue title and body */ -function inferIssueSignal(title: string, body: string): import('../types.js').Signal[] { - const text = `${title} ${body}`.toLowerCase(); - if (text.includes('bug') || text.includes('error') || text.includes('crash')) return ['complaint']; - if (text.includes('feature') || text.includes('request') || text.includes('please add')) return ['request']; - if (text.includes('how to') || text.includes('help') || text.includes('question')) return ['confusion']; - return ['other']; -} - -/** Truncate a string to maxLen characters */ -function truncate(str: string, maxLen: number): string { - if (str.length <= maxLen) return str; - return str.slice(0, maxLen - 3) + '...'; + + return adapter.discover(state); } diff --git a/src/discovery/content.ts b/src/discovery/content.ts index dae13a3..12ba696 100644 --- a/src/discovery/content.ts +++ b/src/discovery/content.ts @@ -1,276 +1,22 @@ /** - * Content discovery module — RSS feeds, blog search, YouTube search. - * - * Implements the discovery strategy from ACCES.md §6. - * RSS parsing is fully implemented; other sources are skeletons - * for Kima and the rest of the scouts to fill in. + * Content discovery module — backward compatibility facade. + * Redirects to adapter-based discovery. RSS logic moved to adapters/rss.ts. */ -import RssParser from 'rss-parser'; -import { createHash } from 'node:crypto'; -import type { - ContentItem, - ContentType, - Channel, - DiscoveryResult, - RunState, -} from '../types.js'; +import { RSSSourceAdapter } from './adapters/rss.js'; +import type { DiscoveryResult, RunState } from '../types.js'; -// ─── ACCES.md §6 keyword lists ───────────────────────────────────────────── - -/** Primary search keywords for Aspire content discovery */ -export const SEARCH_KEYWORDS: readonly string[] = [ - 'Aspire dev', - 'Aspire 9', - 'Aspire 9.1', - 'Aspire 9.2', - 'Aspire 9.3', - 'Aspirified', - 'Aspire AppHost', - 'Aspire dashboard', - 'Aspire manifest', - 'Aspire service discovery', - 'Aspire .NET', - 'Aspire dotnet', - 'Aspire C#', - 'Aspire csharp', - 'Aspire CLI', - 'Aspire javascript', - 'Aspire python', - 'Aspire azure', - 'Aspire aws', - 'Aspire deploy', - 'Aspire docker', - 'Aspire distributed', - 'Aspire app', - 'Aspire code', - 'Aspire kubernetes', - 'Aspire aca', - 'Aspire redis', - 'Aspire otel', -] as const; - -/** Keywords that indicate a result is NOT about .NET Aspire */ -export const EXCLUSION_KEYWORDS: readonly string[] = [ - 'aspirelearning', - 'aspiremag', - 'buildinpublic', - '#openenrollment', - '#aspirepublicschools', - '#aspirelosangeles', -] as const; - -/** Seed RSS feeds for Aspire-related blogs */ -const RSS_FEEDS: readonly { url: string; channel: Channel; name: string }[] = [ - { url: 'https://devblogs.microsoft.com/dotnet/feed/', channel: 'personal_blog', name: '.NET Blog' }, - { url: 'https://devblogs.microsoft.com/aspnet/feed/', channel: 'personal_blog', name: 'ASP.NET Blog' }, - { url: 'https://dev.to/feed/tag/dotnetaspire', channel: 'devto', name: 'Dev.to #dotnetaspire' }, - { url: 'https://dev.to/feed/tag/aspire', channel: 'devto', name: 'Dev.to #aspire' }, -] as const; +export { SEARCH_KEYWORDS, EXCLUSION_KEYWORDS, generateCanonicalId } from './adapters/helpers.js'; /** - * Run all content discovery sources and return combined results. + * @deprecated Use createDefaultRegistry().discoverAll() instead. */ export async function discoverContent(state: RunState): Promise { - const results: DiscoveryResult[] = []; - - // RSS feeds — fully implemented - const rssResults = await discoverFromRssFeeds(state); - results.push(...rssResults); - - // Blog search — skeleton for future implementation - const blogResults = await discoverFromBlogSearch(state); - results.push(...blogResults); - - // YouTube search — skeleton for future implementation - const youtubeResults = await discoverFromYouTube(state); - results.push(...youtubeResults); - - return results; -} - -// ─── RSS feed discovery (implemented) ─────────────────────────────────────── - -/** - * Parse configured RSS feeds and extract Aspire-related items. - * Filters by last-run timestamp and exclusion keywords. - */ -async function discoverFromRssFeeds(state: RunState): Promise { - const parser = new RssParser(); - const results: DiscoveryResult[] = []; - const sinceDate = new Date(state.last_run); - - for (const feed of RSS_FEEDS) { - try { - console.log(` 📡 Fetching RSS: ${feed.name}`); - const parsed = await parser.parseURL(feed.url); - - const items: ContentItem[] = []; - for (const entry of parsed.items ?? []) { - const pubDate = entry.pubDate ? new Date(entry.pubDate) : null; - - // Skip items older than last run - if (pubDate && pubDate < sinceDate) continue; - - const title = entry.title ?? 'Untitled'; - const url = entry.link ?? ''; - if (!url) continue; - - // Check relevance — must mention aspire or related terms - const text = `${title} ${entry.contentSnippet ?? ''} ${entry.content ?? ''}`.toLowerCase(); - if (!isAspireRelated(text)) continue; - - // Check exclusion list - if (isExcluded(text)) continue; - - const canonicalId = generateCanonicalId(title, url, entry.creator ?? null, entry.pubDate ?? null); - - items.push({ - canonical_id: canonicalId, - title, - url, - type: inferContentType(url, text), - channel: feed.channel, - published_at: entry.pubDate ?? null, - author: entry.creator ?? entry['dc:creator'] as string ?? null, - summary: truncate(entry.contentSnippet ?? '', 300), - tags: { - topic: extractTopics(text), - audience: ['intermediate'], - signal: ['other'], - confidence: 'medium', - actionability: 'investigate', - }, - provenance: { - discovered_from: `rss:${feed.name}`, - discovered_query: null, - source_first_seen: new Date().toISOString(), - raw_evidence_path: null, - }, - dedupe: { - is_duplicate: false, - duplicate_of: null, - duplicate_reason: null, - }, - }); - } - - if (items.length > 0) { - results.push({ items, source: `rss:${feed.name}` }); - } - } catch (err) { - const message = err instanceof Error ? err.message : String(err); - console.warn(` ⚠️ RSS feed ${feed.name} failed: ${message}`); - } - } - - return results; -} - -// ─── Blog search (skeleton) ───────────────────────────────────────────────── - -/** - * Search blog platforms for Aspire content. - * TODO: Implement Dev.to API search, Medium search, Substack discovery. - */ -async function discoverFromBlogSearch(_state: RunState): Promise { - // Skeleton — Kima and blog scouts will implement - console.log(' 📝 Blog search: not yet implemented (skeleton)'); - return []; -} - -// ─── YouTube search (skeleton) ─────────────────────────────────────────────── - -/** - * Search YouTube for Aspire-related videos. - * TODO: Implement YouTube Data API v3 search with SEARCH_KEYWORDS. - */ -async function discoverFromYouTube(_state: RunState): Promise { - // Skeleton — video scout will implement - console.log(' 🎥 YouTube search: not yet implemented (skeleton)'); - return []; -} - -// ─── Helpers ──────────────────────────────────────────────────────────────── - -/** Check if text is Aspire-related using keyword heuristics */ -function isAspireRelated(text: string): boolean { - const lower = text.toLowerCase(); - const markers = [ - 'aspire', 'apphost', 'aspire dashboard', - 'service discovery', 'aspire manifest', - '.net aspire', 'dotnet aspire', - ]; - return markers.some((m) => lower.includes(m)); -} - -/** Check if text matches exclusion keywords */ -function isExcluded(text: string): boolean { - const lower = text.toLowerCase(); - return EXCLUSION_KEYWORDS.some((k) => lower.includes(k.toLowerCase())); -} - -/** Generate a deterministic canonical ID from content attributes */ -export function generateCanonicalId( - title: string, - url: string, - author: string | null, - date: string | null, -): string { - const input = [title, url, author ?? '', date ?? ''].join('|').toLowerCase(); - return createHash('sha256').update(input).digest('hex').slice(0, 16); -} - -/** Infer content type from URL and text */ -function inferContentType(url: string, text: string): ContentType { - const lower = url.toLowerCase(); - if (lower.includes('youtube.com') || lower.includes('youtu.be')) return 'video'; - if (lower.includes('github.com')) { - if (lower.includes('/releases/')) return 'release'; - if (lower.includes('/issues/')) return 'issue'; - if (lower.includes('/discussions/')) return 'discussion'; - return 'repo'; - } - if (lower.includes('reddit.com')) return 'reddit'; - if (text.includes('sample') || text.includes('template')) return 'sample'; - return 'blog'; -} - -/** Extract topic tags from text using keyword matching */ -function extractTopics(text: string): string[] { - const topicKeywords: Record = { - apphost: ['apphost', 'app host'], - dashboard: ['dashboard'], - integrations: ['integration'], - k8s: ['kubernetes', 'k8s'], - aca: ['azure container app', 'aca'], - otel: ['opentelemetry', 'otel'], - postgres: ['postgres', 'postgresql'], - redis: ['redis'], - dapr: ['dapr'], - auth: ['auth', 'authentication', 'identity'], - caching: ['cache', 'caching'], - dotnet: ['.net', 'dotnet', 'c#', 'csharp'], - typescript: ['typescript'], - python: ['python'], - docker: ['docker', 'container'], - deploy: ['deploy', 'deployment'], - }; - - const found: string[] = []; - const lower = text.toLowerCase(); - - for (const [topic, keywords] of Object.entries(topicKeywords)) { - if (keywords.some((k) => lower.includes(k))) { - found.push(topic); - } + const adapter = new RSSSourceAdapter(); + const validation = await adapter.validate(); + if (!validation.valid) { + console.warn(`⚠️ RSS adapter validation failed: ${validation.reason}`); + return []; } - - return found.length > 0 ? found : ['aspire']; -} - -/** Truncate a string to maxLen characters */ -function truncate(str: string, maxLen: number): string { - if (str.length <= maxLen) return str; - return str.slice(0, maxLen - 3) + '...'; + return adapter.discover(state); } diff --git a/src/index.ts b/src/index.ts index e101628..5fe70f4 100644 --- a/src/index.ts +++ b/src/index.ts @@ -8,12 +8,11 @@ import { mkdir } from 'node:fs/promises'; import { join } from 'node:path'; import { loadState, saveState } from './state.js'; -import { discoverContent } from './discovery/content.js'; -import { discoverCommunity } from './discovery/community.js'; +import { createDefaultRegistry } from './discovery/adapters/index.js'; import { classify, deduplicate } from './taxonomy.js'; import { analyze } from './analysis.js'; import { generateOutput } from './output.js'; -import type { ContentItem, RunOutput } from './types.js'; +import type { ContentItem, DiscoveryResult, RunOutput } from './types.js'; /** * Format a Date as YYYY-MM-DD_HH-mm-ss for output folder names. @@ -57,37 +56,25 @@ async function main(): Promise { // ── Step 2: Discover ──────────────────────────────────────────────── console.log('🔍 Discovering content...'); - const allItems: ContentItem[] = []; - let discoveryErrors = 0; - - try { - const contentResults = await discoverContent(state); - for (const result of contentResults) { - console.log(` ✅ ${result.source}: ${result.items.length} items`); - allItems.push(...result.items); - } - } catch (err) { - discoveryErrors++; - const message = err instanceof Error ? err.message : String(err); - console.error(` ❌ Content discovery failed: ${message}`); - } + const registry = createDefaultRegistry(); + const registryResult = await registry.discoverAll(state); - try { - const communityResults = await discoverCommunity(state); - for (const result of communityResults) { - console.log(` ✅ ${result.source}: ${result.items.length} items`); - allItems.push(...result.items); - } - } catch (err) { - discoveryErrors++; - const message = err instanceof Error ? err.message : String(err); - console.error(` ❌ Community discovery failed: ${message}`); + const allItems: ContentItem[] = []; + for (const result of registryResult.results) { + console.log(` ✅ ${result.source}: ${result.items.length} items`); + allItems.push(...result.items); } console.log(` 📋 Total raw items: ${allItems.length}`); - if (discoveryErrors > 0) { - console.log(` ⚠️ ${discoveryErrors} discovery source(s) had errors`); + + if (registryResult.skipped.length > 0) { + console.log(` ⏭️ Skipped: ${registryResult.skipped.join(', ')}`); + } + + if (registryResult.errors.size > 0) { + console.log(` ⚠️ ${registryResult.errors.size} adapter(s) had errors`); } + console.log(''); // ── Step 3: Deduplicate ───────────────────────────────────────────── diff --git a/src/types.ts b/src/types.ts index 1088f62..0527987 100644 --- a/src/types.ts +++ b/src/types.ts @@ -26,6 +26,7 @@ export type ContentType = /** Discovery channel — known values plus open string for future channels */ export type Channel = + | 'rss' | 'youtube' | 'github' | 'reddit' @@ -37,6 +38,8 @@ export type Channel = | 'X' | 'bluesky' | 'linkedin' + | 'stackoverflow' + | 'podcast' | (string & {}); /** Target audience for the content */