fix(build): stop build-time projection from caching an empty corpus on Windows#11
Merged
Merged
Conversation
…n Windows On Windows, `dotnet run -- build` could ship an empty search index and header-only llms.txt while reporting success. AuditRunner.StartAsync fired its initial pass fire-and-forget; in build mode that pass self-fetched every route through the SiteProjection before the in-process TestServer had started. The self-fetch threw "server has not been started", RenderOneAsync swallowed it as a per-page skip, SeedAsync completed with an empty corpus, and AsyncLazy cached the poison that the search/llms emitters then consumed. Linux/dev won the startup race; Windows lost it deterministically. Two fixes: 1. Ordering: AuditRunner now runs its initial pass on IHostApplicationLifetime.ApplicationStarted instead of inside StartAsync, so the web server is guaranteed up before any self-fetch. Preserves build-time link-audit coverage. 2. Fail loud: add SelfFetchUnavailableException; HttpDispatcher.CreateClient throws it when the TestServer isn't started or Kestrel has no address. SiteProjection.RenderOneAsync no longer catches it, so an infrastructure failure faults SeedAsync (AsyncLazy evicts and retries) rather than caching an empty corpus as if the crawl had completed. Tests: HttpDispatcherTests pins the wrapped-exception behavior; SiteProjection faults-then-retries to a populated corpus instead of caching empty; AuditRunner tests updated for the ApplicationStarted gate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On Windows,
dotnet run -- buildcould produce a site whose search index andllms.txtare empty even though every HTML page rendered correctly — and the build still reported success. Every route loggedSiteProjection: failed to project /<route>/, skippingwithInvalidOperationException: The server has not been started or no web application was configured. Reproduced on a vanillaAddDocSiteproject; works on Linux CI and in dev/serve mode.Root cause (instrumented + confirmed)
AuditRunner.StartAsync(a hosted service) fires its initial audit pass fire-and-forget. In build mode that pass resolvesLinkAuditorand self-fetches every route throughISiteProjection→RenderedHtmlFetcher→HttpDispatcher.CreateClient(). Startup instrumentation showed this runs before the in-processTestServerhas started (itsApplicationis still null), soCreateHandler()throws.SiteProjection.RenderOneAsync's broadcatchtreated that infrastructure failure identically to a per-page content error: it returnednull,SeedAsynccompleted with an empty corpus, andAsyncLazy(which only evicts on fault) cached the poison. The search/llms emitters then consumed the empty projection. Linux/dev win the startup race (server starts first); Windows loses it deterministically.Fix
AuditRunnerruns its initial pass onIHostApplicationLifetime.ApplicationStartedinstead of insideStartAsync. That fires only after every hosted service (the web server included) has started, so the self-fetch never races server start. Build-time link-audit coverage is preserved.SelfFetchUnavailableException;HttpDispatcher.CreateClientthrows it for both the un-started-TestServerand no-Kestrel-address cases.RenderOneAsyncdeliberately does not catch it (catch (Exception ex) when (ex is not SelfFetchUnavailableException)), so an infra failure faultsSeedAsync,AsyncLazyevicts, and the next access retries instead of baking an empty corpus.The report also suggested an "abort the build if 0-of-N routes project" guardrail; intentionally skipped to keep the change surgical — fixes #1 + #2 already make this failure mode either work or fail loud.
Verification
DocSiteScaffoldExamplebuild: search indexn=2with both docs,llms.txtlists both pages — 3/3 runs (previously a deterministic empty build on Windows).BareHostSearchExample(~100-doc corpus): 85 pages, 484 heading-level search docs.Tests added/updated
HttpDispatcherTests(new) — un-startedTestServerand no-address server both surfaceSelfFetchUnavailableException.SiteProjectionTests— infra failure during seed faults rather than caching empty, then retries to a populated corpus once the server is up.AuditRunnerTests— updated for theApplicationStartedgate.