Skip to content

basePath is not in the digest hash gate — changing it never rebuilds, and the bare CLI defaults to /docs/ #4

Description

@hev

Problem

basePath can silently disagree with the site it was built for, and the digest
gives no way to notice. The failure mode is agentic answers citing URLs that
404 — confident, well-formed, and wrong, which is the worst shape for a docs
search product.

There are two variants. The second is the serious one: it hits the recommended
npm run build path, not just hand-run CLI.

Variant 1 — bare CLI silently defaults to /docs/

normalizeBuildOptions
(pkg/ask/build.go:284-286)
fills in /docs/ when no --base-path is passed. A bare ask digest build
therefore produces a complete, healthy-looking 124-chunk digest with every URL
prefixed /docs/…, on a site that routes docs at the root.

Nothing about the output suggests a problem — same chunk count, same exit code,
same digest:built line.

This one is at least catchable: digest verify shares the same normalize
(pkg/ask/verify.go:38),
so a bare build + bare verify pair fails loudly on every anchor. The gap is
only for people who skip verify.

Variant 2 — basePath is not in the hash gate, so it never triggers a rebuild

HashableChunkText
(pkg/ask/chunk.go:172-180)
hashes chunk.ID + "\n" + chunk.Text and nothing else. chunk.URL is not in
the hash.
So contentHash is invariant under a basePath change, and the
up-to-date check at
build.go:115
short-circuits the rebuild.

Reproduced against a real digest (Starlight site, basePath: '/', 124 chunks):

$ grep -rho 'url: "/[a-z-]*"' .hev-ask | sort -u | head -3
url: "/"
url: "/faq"
url: "/glossary"

$ ask digest build --collection docs --base-path /docs/     # <-- different base path
[hev-ask] digest:skipped .hev-ask (124 chunks)              # <-- skipped anyway

$ grep -rho 'url: "/[a-z-]*"' .hev-ask | sort -u | head -3
url: "/"                                                     # <-- unchanged
url: "/faq"
url: "/glossary"

The consequence: edit basePath in astro.config.mjs, run npm run build, and
the integration's auto-build
(integration.ts:114-131)
passes the new --base-path — and the builder ignores it, because the content
didn't change. Every URL in the digest keeps the old prefix. The user did
everything right and still ships broken citations.

Same class of bug applies to any option that affects chunk output but not chunk
text — --chunk-heading-depth is worth checking in the same pass.

Nothing downstream can detect the drift

_meta.md records version, generatedAt, contentHash, and suggestions
not basePath. So neither verify, nor the endpoint, nor the integration
can compare what the digest was built for against what the site actually serves.
The information needed to catch this isn't persisted anywhere.

Suggested fix

Three changes, roughly in value order:

  1. Put basePath in the hash input. Either add it to HashableChunkText
    or include chunk.URL alongside chunk.Text. This alone fixes variant 2 and
    makes the gate honest about what the digest depends on.
  2. Record basePath in _meta.md frontmatter. Then verify can fail with
    a precise message (digest built for /docs/, site serves /) instead of 106
    individually-confusing missing-anchor lines, and the integration can warn at
    build time when the committed digest disagrees with astro.config.mjs.
  3. Consider dropping the /docs/ default for an explicit-or-error flag on
    the CLI path. A wrong default that produces plausible output is worse than a
    missing required argument. The integration always passes the value, so this
    only affects hand-run CLI.

(1) and (2) are the ones that matter; (3) is a judgment call.

Coverage gap

normalizeBuildOptions has the default and the hash function omits the URL, but
no test asserts the relationship between them — that two builds differing only
in basePath must produce different digests. That's a one-line property test.


Found while wiring 0.3.4 into a Starlight docs site that routes at /. Present
on main (a3204b4). Related: #2, #3.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions