Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ Profiles ([content/profiles/](content/profiles/)) are YAML files that tag which

**Independent sync categories** run in parallel after the archive fetch: `events`, `youtube`, `news`, `discovery`, `tutorials`, `learning`. Each has its own TTL in `config.yaml`. The `news` category (default 2h) uses `runNewsFetch` which tries RSS with retry → YouTube API v3 → baseline file fallback, then caches to `<cacheDir>/news/news-cache.json`.

**Phase 2 — Dynamic Content Expansion:** After the zip fetch, `sync` scans each `context.md` for `<!-- sap-devs:fetch ... -->` markers via `ScanMarkers()`, fetches remote content in parallel (Bubbletea progress UI), then writes `context.expanded.md` alongside `context.md`. `inject` prefers `context.expanded.md` when present. Marker authoring details: [docs/content-authoring.md](docs/content-authoring.md).
**Phase 2 — Dynamic Content Expansion:** After the zip fetch, `sync` scans each `context.md` for `<!-- sap-devs:fetch ... -->` markers via `ScanMarkers()`, fetches remote content in parallel (Bubbletea progress UI), then writes `context.expanded.md` alongside `context.md`. `inject` prefers `context.expanded.md` when present. When a pack loses all its markers between syncs, `runMarkerExpansion` deletes the previously-written `context.expanded.md` so the loader falls back to the fresh `context.md` instead of serving stale fetched content. Marker authoring details: [docs/content-authoring.md](docs/content-authoring.md).

### Discovery Center

Expand Down
17 changes: 17 additions & 0 deletions cmd/sync.go
Original file line number Diff line number Diff line change
Expand Up @@ -448,6 +448,17 @@ func runSyncPlain(plan *syncPlan, out io.Writer) error {
return nil
}

// removeStaleExpansion deletes packDir/context.expanded.md if it exists.
// Used when a pack no longer has sync:fetch markers — without this the loader
// would keep preferring the stale expansion over the fresh context.md.
func removeStaleExpansion(packDir string) error {
path := filepath.Join(packDir, "context.expanded.md")
if err := os.Remove(path); err != nil && !os.IsNotExist(err) {
return err
}
return nil
}

// runMarkerExpansion scans all official-layer packs for sync:fetch markers,
// fetches them in parallel, and writes context.expanded.md alongside each context.md.
// When p is non-nil, it sends SetMarkersMsg and MarkerDoneMsg to the Bubbletea program.
Expand Down Expand Up @@ -488,6 +499,12 @@ func runMarkerExpansion(officialCache string, engine *sapSync.Engine, p *tea.Pro
if hasMarkers {
packContexts[packID] = contextContent
allMarkers = append(allMarkers, markers...)
} else {
// Pack lost its markers since the previous sync — drop the stale
// expanded file so the loader falls back to the fresh context.md.
if err := removeStaleExpansion(filepath.Join(packsDir, packID)); err != nil {
return fmt.Errorf("clean up stale expansion for %s: %w", packID, err)
}
}
}

Expand Down
41 changes: 41 additions & 0 deletions cmd/sync_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
package cmd

import (
"os"
"path/filepath"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)

func TestRemoveStaleExpansion_DeletesFile(t *testing.T) {
packDir := t.TempDir()
expandedPath := filepath.Join(packDir, "context.expanded.md")
require.NoError(t, os.WriteFile(expandedPath, []byte("stale"), 0644))

require.NoError(t, removeStaleExpansion(packDir))

_, err := os.Stat(expandedPath)
assert.True(t, os.IsNotExist(err), "context.expanded.md should be gone, got err=%v", err)
}

func TestRemoveStaleExpansion_NoFileIsNotAnError(t *testing.T) {
packDir := t.TempDir()
// No context.expanded.md exists.
require.NoError(t, removeStaleExpansion(packDir))
}

func TestRemoveStaleExpansion_LeavesContextMdAlone(t *testing.T) {
packDir := t.TempDir()
contextPath := filepath.Join(packDir, "context.md")
expandedPath := filepath.Join(packDir, "context.expanded.md")
require.NoError(t, os.WriteFile(contextPath, []byte("source"), 0644))
require.NoError(t, os.WriteFile(expandedPath, []byte("stale"), 0644))

require.NoError(t, removeStaleExpansion(packDir))

data, err := os.ReadFile(contextPath)
require.NoError(t, err)
assert.Equal(t, "source", string(data))
}
2 changes: 2 additions & 0 deletions docs/content-authoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,8 @@ For a plain-text or non-HTML source, use `format="raw"`:

After `sap-devs sync`, the marker is expanded in `context.expanded.md` and the fetched release notes appear directly below it. The original `context.md` is never modified — only the derived `context.expanded.md` changes.

**Removing a marker.** If you delete every `<!-- sync:fetch ... -->` marker from a pack's `context.md`, the next `sap-devs sync` will delete the previously-written `context.expanded.md` for that pack. This ensures the loader falls back to the fresh `context.md` instead of continuing to serve stale fetched content.

For a real-world example see [`content/packs/cap/context.md`](../content/packs/cap/context.md).

---
Expand Down
Loading