Add reviewed Gutenberg facets domain relevance corpus#38
Merged
Conversation
Add new domain suite registration for Gutenberg fiction faceted search: - Add GUTENBERG_FICTION_FACETS_DOMAIN_QUERY_TOPICS constant with 5 topics - Spread new topics into DOMAIN_QUERY_TOPICS - Register "gutenberg-fiction-facets" in KNOWN_DOMAIN_SUITES - Update test to expect 4 suites and validate ENOENT for missing fixture Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… types Adds facets/rangeFacets to SnapshotDomainDocument and RelevanceDocument, and filters to DomainJudgedQuery and JudgedQuery, all optional so existing suites (searchable-docs, govuk-learn-to-drive, de-fahrerlaubnisrecht) are unaffected. Types only, no runtime behavior change yet.
Extend validateDomainSuite to check that snapshot document facets are
non-empty string arrays, rangeFacets are finite numbers, and query
filters only reference facet fields declared somewhere in the corpus.
validateCorpus now returns { documentIds, facetFields } instead of a
bare Set<string> to support cross-document facet-field lookups.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Query filters now must be a non-blank string, a non-empty array of non-blank strings, or a range filter with finite min/max, instead of only checking that the filter key names a known facet field. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…apshot evaluator Emit searchable-facet-<field>/searchable-facet-range-<field> meta tags for documents declaring facets/rangeFacets in searchable-runner.ts, and thread query.filters through evaluate.ts's SearchForEvaluation options and domain-runner.ts's toSnapshotEvaluationSuite, using conditional spreads so suites without facets/filters produce identical output.
…nts (draft) Adds packages/relevance/fixtures/domains/gutenberg-fiction-facets.json with 30 real Project Gutenberg public-domain books across 5 genre facets (Gothic Fiction, Science Fiction, Adventure, Detective Fiction, Children's Literature) and a year range facet spanning 1764-1929, per Task 1's source-selection decision note. queries is left empty for Task 7. Also adds packages/relevance/scripts/hash-gutenberg-facets-content.mjs, copied from hash-german-domain-content.mjs, used to compute each document's contentHash.
…ltered queries Add 20 queries spanning the 5 registered topics (genre-browsing, publication-era, author-and-title-lookup, plot-and-theme-search, cross-genre-comparison), 7 of which carry a genre/year/combined filter. Verified via the project's own analyze() tokenizer that every judged document's title/description/body genuinely contains the query's terms, and confirmed by facet/rangeFacet inspection that every filtered query's positively-judged documents actually satisfy their declared filter.
…fixture exists Task 2's registration test anticipated the fixture not existing yet; Tasks 6-7 built it, so the assertion needs to confirm successful load instead. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…atting Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…r consistency Adds a policy test mirroring de-fahrerlaubnisrecht-fixture-policy.test.ts: document/query counts, topic coverage, content-hash consistency, and judgment/rationale parity, plus a Gutenberg-specific check that every filter-bearing query's positively-judged documents genuinely satisfy that query's genre and/or year-range filter. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…aseline Flip gutenberg-fiction-facets.json to review.status: reviewed (ktjn, 2026-07-17) and document the reviewed baseline (MRR 1.0, Precision@5 0.36, Recall@5 1.0, nDCG@5 0.960733, zero-result rate 0.0) in relevance-baselines.md, called out as the first domain suite to exercise judged relevance under real terms/range facet filtering. Update roadmap.md and CHANGELOG.md to reflect the fourth reviewed domain corpus. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
packages/relevance:gutenberg-fiction-facets, a snapshot corpus of 30 Project Gutenberg public-domain books (5 real bookshelf genres, years 1764-1929) — the first corpus in this project to exercise real terms-facet (genre) and range-facet (year) filtering under judged relevance.facets/rangeFacets/filtersschema fields, validator extensions (including filter value-shape validation), and wiring facet-declaring HTML meta tags + query filters through the snapshot-suite runner and evaluator.packages/client/packages/indexer/packages/formatare untouched — the facet feature itself was already fully implemented there.docs/project/relevance-baselines.md. Docs are explicit that facet filtering is covered but facet counts (facetValues()) remain out of scope.Test plan
pnpm --filter @ktjn/searchable-relevance test— 165/165 passingnpx biome checkclean across changed packages/docsgutenberg-fiction-facetsreproduces the published baseline exactly, including all 7 facet-filtered queries independently verified to respect their declared filterssearchable-docs,govuk-learn-to-drive,de-fahrerlaubnisrecht) re-run live and confirmed byte-identical to their recorded baselines — this branch's engine changes are a proven no-op for non-faceted suites🤖 Generated with Claude Code