Add reviewed German domain relevance corpus (driving-license law)

.gitignore

@@ -8,6 +8,7 @@ playwright-report/
 blob-report/
 playwright/.cache/
 .worktrees/
+docs/superpowers/
 /benchmark-results/*
 !/benchmark-results/cms-2k/
 /benchmark-results/cms-2k/*

CHANGELOG.md

@@ -67,6 +67,11 @@ below was built and tested across prior unreleased commits.
 - Five retroactive ADRs (`docs/adr/`) recording the transport, index
   format, ranking model, compatibility policy, and plugin/opt-in-tier
   decisions above.
+- **Relevance domain corpora**: a reviewed German-language judged relevance
+  domain corpus (`de-fahrerlaubnisrecht`, 23 German Wikipedia driving-license-law
+  pages, 19 queries), alongside the existing documentation and GOV.UK
+  learner-driving corpora, evaluated by the deterministic offline relevance
+  evaluator (docs/project/relevance-baselines.md).
 
 ### Notes
 

docs/project/relevance-baselines.md

@@ -4,8 +4,9 @@ Searchable includes a deterministic lexical relevance evaluator for every full
 language profile: English (`en`), German (`de`), Swedish (`sv`), Dutch (`nl`),
 Bokmål (`nb`), and Nynorsk (`nn`). It exercises the public indexer and client
 APIs over local HTTP, using committed native-language FAQ and help excerpts.
-Two reviewed domain suites evaluate the generated Searchable documentation
-index and a normalized snapshot of the GOV.UK learner-driving journey.
+Three reviewed domain suites evaluate the generated Searchable documentation
+index, a normalized snapshot of the GOV.UK learner-driving journey, and a
+German-language corpus of Wikipedia articles on driving-license law.
 
 ## Run the baseline
 
@@ -51,6 +52,12 @@ For its per-query report:
 pnpm relevance -- --suite govuk-learn-to-drive --json
 ```
 
+Run the German driving-license-law suite the same way:
+
+```sh
+pnpm relevance -- --suite de-fahrerlaubnisrecht
+```
+
 `--suite` and `--language` are mutually exclusive. The default command remains
 the six-language regression baseline.
 
@@ -152,21 +159,60 @@ report, and record a fresh reviewer and review date before treating the new
 version as a baseline. Increment the version whenever documents, queries,
 judgments, grades, or rationales change.
 
+## Reviewed German domain corpus
+
+`de-fahrerlaubnisrecht@1.0.0` contains 23 normalized documents drawn from
+German Wikipedia's `Kategorie:Fahrerlaubnisrecht (Deutschland)` category and a
+few buffer-list swaps made during normalization. It has 19 German
+task-oriented queries across five topics: license classes and eligibility,
+fitness-to-drive and the MPU exam, penalties and license withdrawal, the
+probationary period, and special permits and courses.
+
+The suite uses the same graded judgment and page-specific rationale policy as
+the documentation and GOV.UK corpora. Maintainer `ktjn` reviewed every
+normalized document, query, grade, rationale, and measured top-five result on
+2026-07-17.
+
+The reviewed baseline at `k = 5` is:
+
+| Metric | Value |
+|---|---:|
+| MRR | 0.855263 |
+| Precision@5 | 0.284211 |
+| Recall@5 | 0.517544 |
+| nDCG@5 | 0.690330 |
+| Zero-result rate | 0.052632 |
+
+The source snapshot was retrieved on 2026-07-17. It contains German Wikipedia
+article text licensed under [CC BY-SA
+4.0](https://creativecommons.org/licenses/by-sa/4.0/), attributed to Wikipedia
+contributors with edit history available via each source page. Selection
+notes recording the category listing, discarded stubs/overlaps, and buffer
+swaps are recorded in the fixture's `provenance` field.
+
 ## Interpretation and limits
 
-The language suites are small regression fixtures, and the two domain suites
+The language suites are small regression fixtures, and the three domain suites
 are reviewed but intentionally narrow corpora. They verify that content can be
 indexed and retrieved through the shipped public path and provide a
 reproducible signal when ranking or analysis changes. They are not
 production-scale benchmarks.
 
 The language suites use one strongly relevant document per native source
 question or help topic. The domain suites use graded multi-page judgments, but
-cover only one English documentation site and one UK learner-driving journey.
-Their queries are authored intents rather than user-query logs. These scores
-are not latency or memory evidence and do not establish web-scale quality or
-superiority over another engine. Do not compare scores across suites because
-their source material, query sets, and judgments differ.
+cover only one English documentation site, one UK learner-driving journey, and
+one German-language encyclopedia corpus — three domains across two languages
+(English and German). Their queries are authored intents rather than
+user-query logs. These scores are not latency or memory evidence and do not
+establish web-scale quality or superiority over another engine. Do not compare
+scores across suites because their source material, query sets, and judgments
+differ.
+
+The German suite's one remaining zero-result query (out of 19) is an accepted
+lexical-matching limitation rather than a defect: this project's search
+performs strict AND across query terms with no compound-word splitting and no
+stopword filtering, which can miss a match when a query's terms are split
+across a German compound word in the indexed text.
 
 There are deliberately no pass/fail score thresholds yet. Before relevance
 becomes a CI gate, the project still needs broader representative domains,

docs/project/roadmap.md

@@ -7,7 +7,7 @@ This page is the single current list of shipped capability and remaining work; d
 | Area | Current state | Remaining work |
 |---|---|---|
 | Documentation and showcase | Published, searchable, and covered by link, accessibility, and browser checks | Ongoing maintenance alongside product changes |
-| Lexical search | Stable; native six-language regression baseline plus reviewed documentation and GOV.UK learner-driving domain corpora | Broader representative domains and judged sets, real query evidence, quality thresholds, and an internal query-planner abstraction |
+| Lexical search | Stable; native six-language regression baseline plus reviewed documentation, GOV.UK learner-driving, and German (`de-fahrerlaubnisrecht`) domain corpora | Broader representative domains and judged sets, real query evidence, quality thresholds, and an internal query-planner abstraction |
 | Facets, synonyms, fuzzy search, and pins | Stable | No required 1.0 work |
 | Internationalization | English, German, Swedish, Dutch, Bokmål, and Nynorsk profiles; fallback segmenters | Additional profiles only with representative corpora and quality gates |
 | Offline and worker execution | Stable | Resource-aware loading refinements |
@@ -18,7 +18,7 @@ This page is the single current list of shipped capability and remaining work; d
 
 ## Near-term work
 
-- Expand relevance coverage beyond the documentation and learner-driving domains with broader judged sets and real query evidence before defining thresholds or making production-scale claims.
+- Expand relevance coverage beyond the documentation, learner-driving, and German driving-license-law domains — now spanning English and German — with broader judged sets and real query evidence before defining thresholds or making production-scale claims.
 - Add a semantic-search example that states model, download, memory, and latency costs clearly; keep lexical search as the default for content sites.
 - Expand full language profiles only with representative corpora, analyzer fixtures, relevance queries, and cross-implementation conformance tests.
 - Refine loading priority, memory controls, and prefetching from measured browser behavior rather than fixed speculative policies.
@@ -60,9 +60,10 @@ Treat relevance as a measurable quality attribute rather than only a correctness
 - zero-result rate
 
 The deterministic evaluator and small native-source regression suites cover all
-six full language profiles, and two reviewed domains supply graded multi-page
-judgments: the 29-page documentation corpus and the 22-page GOV.UK
-learner-driving journey. See [Relevance
+six full language profiles, and three reviewed domains supply graded multi-page
+judgments: the 29-page documentation corpus, the 22-page GOV.UK
+learner-driving journey, and the 23-page German (`de-fahrerlaubnisrecht`)
+driving-license-law corpus. See [Relevance
 baselines](relevance-baselines.md) for commands, metrics, provenance
 requirements, and interpretation limits. These remain narrow representative
 domains, not broad domain coverage or a CI quality gate. Quality thresholds,

packages/relevance/scripts/hash-german-domain-content.mjs

@@ -0,0 +1,21 @@
+import { createHash } from "node:crypto";
+
+function hashSnapshotContent({ title, description, body }) {
+  return createHash("sha256")
+    .update(`${title}\n${description}\n${body}`, "utf8")
+    .digest("hex");
+}
+
+const [, , titleArg, descriptionArg, bodyFileArg] = process.argv;
+if (!titleArg || !descriptionArg || !bodyFileArg) {
+  console.error(
+    "usage: node hash-german-domain-content.mjs <title> <description> <bodyFile>",
+  );
+  process.exit(1);
+}
+
+const { readFileSync } = await import("node:fs");
+const body = readFileSync(bodyFileArg, "utf8");
+console.log(
+  hashSnapshotContent({ title: titleArg, description: descriptionArg, body }),
+);

packages/relevance/src/domain-schema.ts

@@ -24,9 +24,18 @@ export const GOVUK_DOMAIN_QUERY_TOPICS = [
   "after-passing",
 ] as const;
 
+export const DE_FAHRERLAUBNISRECHT_DOMAIN_QUERY_TOPICS = [
+  "license-classes-eligibility",
+  "fitness-to-drive-mpu",
+  "penalties-license-withdrawal",
+  "probationary-period",
+  "special-permits-courses",
+] as const;
+
 export const DOMAIN_QUERY_TOPICS = [
   ...EXISTING_DOMAIN_QUERY_TOPICS,
   ...GOVUK_DOMAIN_QUERY_TOPICS,
+  ...DE_FAHRERLAUBNISRECHT_DOMAIN_QUERY_TOPICS,
 ] as const;
 
 export type DomainQueryTopic = (typeof DOMAIN_QUERY_TOPICS)[number];

packages/relevance/src/load-domain-suite.ts

@@ -6,6 +6,7 @@ import { validateDomainSuite } from "./validate-domain-suite.js";
 export const KNOWN_DOMAIN_SUITES = [
   "searchable-docs",
   "govuk-learn-to-drive",
+  "de-fahrerlaubnisrecht",
 ] as const;
 export type KnownDomainSuite = (typeof KNOWN_DOMAIN_SUITES)[number];
 

packages/relevance/test/de-fahrerlaubnisrecht-fixture-policy.test.ts

@@ -0,0 +1,43 @@
+import { fileURLToPath } from "node:url";
+import { describe, expect, it } from "vitest";
+import { DE_FAHRERLAUBNISRECHT_DOMAIN_QUERY_TOPICS } from "../src/domain-schema.js";
+import { hashSnapshotContent } from "../src/govuk-normalize.js";
+import { loadDomainSuite } from "../src/load-domain-suite.js";
+
+const domainFixtureDirectory = fileURLToPath(
+  new URL("../fixtures/domains/", import.meta.url),
+);
+
+describe("committed German domain relevance fixture", () => {
+  it("is a well-formed draft with full judgment coverage", async () => {
+    const suite = await loadDomainSuite(
+      domainFixtureDirectory,
+      "de-fahrerlaubnisrecht" as never,
+    );
+    expect(suite.id).toBe("de-fahrerlaubnisrecht");
+    expect(suite.language).toBe("de");
+    expect(suite.corpus.kind).toBe("snapshot");
+    if (suite.corpus.kind !== "snapshot") throw new Error("unreachable");
+    expect(suite.corpus.documents.length).toBeGreaterThanOrEqual(20);
+    expect(suite.queries.length).toBeGreaterThanOrEqual(15);
+
+    expect(new Set(suite.queries.map((query) => query.topic))).toEqual(
+      new Set(DE_FAHRERLAUBNISRECHT_DOMAIN_QUERY_TOPICS),
+    );
+
+    for (const document of suite.corpus.documents) {
+      expect(document.body.trim()).not.toBe("");
+      expect(document.contentHash).toBe(hashSnapshotContent(document));
+    }
+
+    for (const query of suite.queries) {
+      const positiveIds = Object.entries(query.judgments)
+        .filter(([, grade]) => grade >= 1)
+        .map(([id]) => id);
+      expect(positiveIds.length).toBeGreaterThan(0);
+      expect(Object.keys(query.rationales).sort()).toEqual(
+        [...positiveIds].sort(),
+      );
+    }
+  });
+});

packages/relevance/test/domain-fixture-policy.test.ts

@@ -1,12 +1,19 @@
 import { fileURLToPath } from "node:url";
 import { describe, expect, it } from "vitest";
 import { DOC_SECTIONS } from "../../../showcase/docs-nav.js";
-import {
-  DOMAIN_QUERY_TOPICS,
-  GOVUK_DOMAIN_QUERY_TOPICS,
-} from "../src/domain-schema.js";
 import { loadDomainSuite } from "../src/load-domain-suite.js";
 
+// Topics for the searchable-docs fixture: existing documentation topics (excluding GOV.UK and domain-specific topics)
+const SEARCHABLE_DOCS_TOPICS = [
+  "setup",
+  "indexing-deployment",
+  "lexical-features",
+  "internationalization",
+  "offline-worker",
+  "relevance",
+  "vector-hybrid",
+] as const;
+
 const domainFixtureDirectory = fileURLToPath(
   new URL("../fixtures/domains/", import.meta.url),
 );
@@ -31,11 +38,8 @@ describe("committed documentation relevance fixture", () => {
     if (suite.corpus.kind !== "generated-index") throw new Error("unreachable");
     expect(suite.corpus.pages).toHaveLength(29);
     expect(suite.queries).toHaveLength(20);
-    const documentationTopics = DOMAIN_QUERY_TOPICS.filter(
-      (topic) => !GOVUK_DOMAIN_QUERY_TOPICS.includes(topic as never),
-    );
     expect(new Set(suite.queries.map((query) => query.topic))).toEqual(
-      new Set(documentationTopics),
+      new Set(SEARCHABLE_DOCS_TOPICS),
     );
     expect(
       suite.queries.filter(

packages/relevance/test/load-domain-suite.test.ts

@@ -59,17 +59,21 @@ it("loads an allowlisted domain suite", async () => {
   );
 });
 
-it("loads both committed allowlisted domain suites", async () => {
+it("loads all committed allowlisted domain suites", async () => {
   const fixtureDirectory = fileURLToPath(
     new URL("../fixtures/domains/", import.meta.url),
   );
   expect(KNOWN_DOMAIN_SUITES).toEqual([
     "searchable-docs",
     "govuk-learn-to-drive",
+    "de-fahrerlaubnisrecht",
   ]);
   await expect(
     loadDomainSuite(fixtureDirectory, "govuk-learn-to-drive"),
   ).resolves.toMatchObject({ id: "govuk-learn-to-drive", version: "1.0.0" });
+  await expect(
+    loadDomainSuite(fixtureDirectory, "de-fahrerlaubnisrecht"),
+  ).resolves.toMatchObject({ id: "de-fahrerlaubnisrecht", version: "1.0.0" });
 });
 
 it("rejects unknown suite names before filesystem access", async () => {