Skip to content

docs(site): agent-ready discovery, MD mirrors, tab labels#104

Open
Tom-Arete-A4 wants to merge 7 commits intomainfrom
docs/agent-ready-docs-updates
Open

docs(site): agent-ready discovery, MD mirrors, tab labels#104
Tom-Arete-A4 wants to merge 7 commits intomainfrom
docs/agent-ready-docs-updates

Conversation

@Tom-Arete-A4
Copy link
Copy Markdown
Contributor

@Tom-Arete-A4 Tom-Arete-A4 commented May 1, 2026

Summary

Improvements for docs.arete.run aimed at AFDocs / agent crawlers (Mintlify Agent Score): clearer llms.txt discovery in-page, static markdown mirrors, lighter HTML head, and unique tab labels where scanners flagged ambiguity.

Changes

  • MarkdownContent override — Screen-reader-only paragraph at the top of article body with links to /llms.txt, /llms-full.txt, and note that .md\ mirrors exist per path. Uses \Astro.site\ for absolute URLs.
  • Head vs footer scripts — \Head.astro\ only emits Starlight meta/links; PostHog + tab/copy analytics moved to \DocsClientScripts.astro, included after the default Footer override so primary content appears earlier in the document order.
  • Post-build .md\ files — \scripts/copy-docs-markdown.mjs\ runs after \�stro build, strips YAML frontmatter from \src/content/docs/**/*.mdx, writes parallel files under \dist/\ (e.g. /using-stacks/quickstart.md). \package.json\ build script chained accordingly.
  • Tab labels — Disambiguated repeated \TabItem\ labels on Quickstart, Filtering Feeds, and Workflow MDX (body copy unchanged).

Out of scope (follow-ups)

  • \Accept: text/markdown\ content negotiation (needs Edge/server middleware).
  • Hosted MCP at docs URL (Mintlify-specific / separate infra).

Testing


  • pm run build\ (includes copy script)

  • pm run check\

- Early sr-only llms.txt directive via MarkdownContent override
- Defer PostHog + tracking to footer (slimmer head)
- Post-build .md copies from content sources for /path.md
- Disambiguate tab labels on quickstart, filtering-feeds, workflow

Co-authored-by: Cursor <cursoragent@cursor.com>
@vercel
Copy link
Copy Markdown

vercel Bot commented May 1, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
arete-docs Ready Ready Preview, Comment May 5, 2026 3:01pm

Request Review

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 1, 2026

Greptile Summary

  • Moves PostHog and analytics click-tracking from Head.astro to a new DocsClientScripts.astro appended after the footer, fixing a listener-accumulation bug and deferring non-critical scripts past primary content. Delegated document-level handlers ensure correct behaviour across Starlight's SPA view transitions.
  • Adds a MarkdownContent.astro override that injects a visually-hidden (sr-only) paragraph advertising llms.txt, llms-full.txt, and per-page .md mirror URLs for agent crawlers; the .md mirrors themselves are not generated (script absent from repo and package.json build command).
  • Disambiguates repeated TabItem labels across workflow.mdx and filtering-feeds.mdx (e.g. \"TypeScript\"\"Generate SDK — TypeScript\") to reduce scanner ambiguity; body content is unchanged.

Confidence Score: 4/5

Safe to merge for the Head/Footer/tab-label changes; the MarkdownContent sr-only paragraph advertises .md mirror URLs that currently return 404, which should be resolved before this goes live.

The analytics refactor and tab-label disambiguation are clean. The MarkdownContent override has an unresolved P1: it advertises per-page .md mirror paths in-page (visible to every agent crawler and user) but the script that generates those files is absent from the repo and the build command was not updated, so every advertised link 404s. This caps the score at 4.

docs/src/components/overrides/MarkdownContent.astro — advertises .md mirror URLs that are not generated; the missing script and package.json build-chain update need to be included in this PR before merging.

Important Files Changed

Filename Overview
docs/src/components/overrides/MarkdownContent.astro New override that injects an sr-only discovery paragraph with links to llms.txt and a per-page .md mirror; the advertised .md mirror URL currently 404s (script absent from repo/build).
docs/src/components/DocsClientScripts.astro Refactored analytics scripts moved from Head to Footer; fixes listener-accumulation bug by using delegated document-level click handlers bundled once per session.
docs/src/components/overrides/Footer.astro New footer override that renders the default footer then appends DocsClientScripts after primary content; straightforward and correct.
docs/src/components/overrides/Head.astro Simplified to emit only Starlight meta/link tags; PostHog and tracking scripts correctly moved to DocsClientScripts.
docs/astro.config.mjs Registers two new Starlight component overrides (Footer, MarkdownContent); no issues.
docs/src/styles/custom.css Adds .sr-only utility class and normalises CSS selector whitespace; no functional changes.

Sequence Diagram

sequenceDiagram
    participant Browser
    participant Head.astro
    participant MarkdownContent.astro
    participant Footer.astro
    participant DocsClientScripts.astro
    participant PostHog

    Browser->>Head.astro: Request page
    Head.astro-->>Browser: Starlight meta/links only

    Browser->>MarkdownContent.astro: Render article body
    MarkdownContent.astro-->>Browser: sr-only discovery paragraph + MDX content

    Browser->>Footer.astro: Render footer
    Footer.astro-->>Browser: Default footer HTML
    Footer.astro->>DocsClientScripts.astro: Include after footer
    DocsClientScripts.astro->>PostHog: Inject analytics snippet
    DocsClientScripts.astro-->>Browser: Delegated click handlers (tab + copy tracking, once per session)
Loading

Reviews (6): Last reviewed commit: "chore(docs): modernize .sr-only with cli..." | Re-trigger Greptile

Comment thread docs/src/components/DocsClientScripts.astro Outdated
Comment thread docs/scripts/copy-docs-markdown.mjs Outdated
Comment thread docs/src/components/overrides/MarkdownContent.astro
Resolve package.json: chain main's fix-llms-full + build-docs-index with copy-docs-markdown. Take main quickstart (syncKey installer tabs, step headings).

Co-authored-by: Cursor <cursoragent@cursor.com>
Use Starlight's page-load lifecycle as the single analytics setup path, and rely on generated Astro markdown routes instead of a post-build mirror script.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread docs/src/components/overrides/MarkdownContent.astro Outdated
Point the hidden agent directive at the generated Astro markdown route for the current page instead of describing a mirror file convention.

Co-authored-by: Cursor <cursoragent@cursor.com>
Satisfy the docs Prettier check after linking exact markdown routes.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Tom-Arete-A4
Copy link
Copy Markdown
Contributor Author

Re: Issue 1 — .md mirror script (Greptile P1)

This was resolved by a different, strictly-better approach already on this branch — flagging here so reviewers don't re-add the redundant post-build script.

What the plan suggested: add docs/scripts/copy-docs-markdown.mjs, walk src/content/docs/**/*.mdx, strip frontmatter, write dist/<route>.md, wire into npm run build.

What this branch actually does (came in via merge with main from #103, then commit c92392e removed the now-redundant script):

  • Astro dynamic routes generate .md mirrors at build time:
    • docs/src/pages/[...slug].md.ts covers every doc page.
    • docs/src/pages/index.md.ts covers the homepage.
  • Frontmatter is preserved as JSON-encoded YAML (title/description/sidebar metadata stays).
  • MDX JSX is stripped via docs/src/lib/strip-mdx.ts, so output is clean CommonMark.
  • Routes set Content-Type: text/markdown; charset=utf-8 directly on the response — no vercel.json header needed.
  • vercel.json already rewrites Accept: text/markdown requests on HTML paths to the .md routes.

Verification (run locally on this branch):

  • npm run build exits 0.
  • dist/ contains 34 .md files at the exact paths advertised by MarkdownContent.astro (/index.md, /cli/commands.md, /sdks/typescript.md, etc.).
  • Each .md file has frontmatter + stripped MDX body.

Net: the sr-only paragraph's advertised .md URLs resolve in the build. No new script needed; adding copy-docs-markdown.mjs would overwrite the better Astro output.

Tom-Arete-A4 and others added 2 commits May 5, 2026 16:00
Move tab and code-copy click handlers out of astro:page-load. Starlight view-transitions fire astro:page-load on every SPA navigation, so the previous setup re-registered duplicate document-level listeners each time, multiplying analytics events. Delegated handlers survive view-transitions natively, so a single attach at script-execution time is enough.

Co-authored-by: Cursor <cursoragent@cursor.com>
Pair the deprecated `clip: rect(0, 0, 0, 0)` rule with `clip-path: inset(50%)` so the visually-hidden agent-discovery paragraph stays hidden under modern CSS Masking semantics while keeping the legacy property for older browsers.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Tom-Arete-A4
Copy link
Copy Markdown
Contributor Author

Review summary — follow-ups for pr-104-plan.md

Updated this PR with two commits to address the Greptile review and the linked plan:

Commit Concern Plan ref Status
abd6262 fix(docs): attach analytics listeners once at module scope Tab + code-copy click listeners stacked on every Starlight view-transition (astro:page-load re-arm), inflating analytics events linearly with navigation depth Issue 2 (Greptile P1) Fixed
ccdc160 chore(docs): modernize .sr-only with clip-path .sr-only used only deprecated clip: rect(0,0,0,0) Nit N2 Fixed

Issue 1 (.md mirrors) is already resolved on this branch — see comment above. The plan's proposed copy-docs-markdown.mjs is intentionally not re-added; Astro dynamic routes (docs/src/pages/[...slug].md.ts, docs/src/pages/index.md.ts) already produce the advertised .md URLs with frontmatter preserved and MDX stripped.

What changed

docs/src/components/DocsClientScripts.astro

  • Removed setupTabTracking / setupCodeCopyTracking / initTracking wrappers and the astro:page-load registration.
  • Click handlers are now attached once at module-scope script execution, delegated on document. They survive view-transitions natively (no per-page re-arm needed).
  • window.location.pathname is read inside the handler, so events still carry the current page after SPA navigations.

docs/src/styles/custom.css

  • .sr-only keeps existing clip: rect(0, 0, 0, 0) for legacy compatibility and adds clip-path: inset(50%) for modern CSS Masking Level 1.
  • Visual behavior unchanged; agent-discovery paragraph stays hidden.

Verification (local)

  • npm run build → 0 errors, 33 pages, 34 .md mirrors generated under dist/.
  • npm run check → 0 errors / 0 warnings (pre-existing Search.astro deprecation hint untouched).
  • npx prettier --check src/components/DocsClientScripts.astro src/styles/custom.css → clean.
  • Source inspection: only one addEventListener(click, ...) per concern at module scope; no astro:page-load listener for tracking.

Verification (preview, post-deploy)

  • curl -I https://<preview>/cli/commands.md → expect 200 + Content-Type: text/markdown; charset=utf-8.
  • DevTools → on a docs page, click any tab once → exactly one docs_tab_clicked PostHog request fires; navigate 5 pages, click again → still one event per click (not N+1).

Skipped intentionally

  • N1 (brittle starlight-tabs [role=tab] selector) — low priority.
  • N3 (MDX comment about syncKey incompatibility with disambiguated tab labels) — no syncKey in affected MDX today.
  • Plan Issue 1 prescription (copy-docs-markdown.mjs) — already obsoleted by Astro routes; would regress.

Once Vercel preview is up I'll re-run the markdown spot-checks and trigger Greptile.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant