vektralabs · fvadicamo · Jul 12, 2026 · Jul 12, 2026 · Jul 12, 2026 · Jul 12, 2026
@@ -84,6 +84,7 @@
 # VEKTRA_QUERY_PIPELINE=advanced
 # VEKTRA_MIN_RELEVANCE_SCORE=0.15
 # VEKTRA_CHUNK_DEDUP_ENABLED=true
+# VEKTRA_PARENT_EXPANSION_ENABLED=false
 # VEKTRA_RESPONSE_TOKEN_RESERVE=2048
 # VEKTRA_CONTEXT_CHUNK_RATIO=0.6
 # VEKTRA_PROMPT_TEMPLATES_DIR=

@@ -283,19 +283,23 @@ The `title` field would contain `filename + page` (e.g., "Costituzione italiana.
 
 ### FEAT-017: Parent chunk expansion in query pipeline
 
-**Status**: planned | **Priority**: medium | **Created**: 2026-03-23
+**Status**: completed (2026-07-12) | **Priority**: medium | **Created**: 2026-03-23
 **Analysis**: `vektra-internal/stack/20260323-rag-prompt-chunk-confusion-analysis.md`
 
 **Context**: when a child chunk is retrieved via search, the pipeline should optionally expand it to the parent chunk for broader context. The infrastructure is already in place: `DualStrategyChunking` creates parent-child hierarchy (parent every 3000 tokens, children at 500 tokens with overlap), `DocumentChunkOrm` has `parent_id` column, and both are stored in the database. Missing: (1) filter parent chunks from default search results (search currently returns both), (2) parent expansion logic in AdvancedQueryPipeline when a child matches.
 
+**Premise corrections found during implementation**: the hierarchy was NOT actually persisted anywhere (`run_ingest` dropped `chunk.parent_id`, `ChunkEmbedding` had no field, Qdrant payload had none, `DocumentChunkOrm.parent_id` was always NULL); parent size is `chunk_size*3` (1500 tokens with Combo D 500, not 3000); pgvector `store()` ignored caller chunk ids (generated uuid4), so deterministic ids had to be plumbed there too.
+
 **Traceability**: ARCH-037 (ChunkingStrategy), ARCH-055 (token budget), core-pipeline-v2
 
 **Acceptance criteria**:
-- [ ] Search excludes parent chunks by default (WHERE parent_id IS NOT NULL for children only)
-- [ ] AdvancedQueryPipeline fetches parent chunk when child matches and includes it in context
-- [ ] Parent expansion is configurable (on/off, via env var)
-- [ ] Token budget accounts for expanded parent chunk size
-- [ ] Tested: truncated-context answers improve with parent expansion enabled
+- [x] Search excludes parent chunks by default (`chunk_level=parent` filtered: Qdrant `must_not`, pgvector `IS DISTINCT FROM`)
+- [x] AdvancedQueryPipeline fetches parent chunk when child matches and includes it in context (step 6.5, new `VectorStoreProvider.retrieve()`)
+- [x] Parent expansion is configurable (`VEKTRA_PARENT_EXPANSION_ENABLED`, default off)
+- [x] Token budget accounts for expanded parent chunk size (expansion runs before ARCH-055 allocation)
+- [x] Tested: 17 unit tests (linkage, exclusion, expansion, budget); measured on `eval-full` — expansion is grounding-neutral there (35/55 with and without, zero per-question flips, avg sources 1.4 → 1.0 via sibling merge). The multi-chunk collapse it targeted turned out to be upstream: the rerank+threshold funnel wipes all candidates before expansion (TECH-007). The corpus also understates expansion benefit (short self-contained articles — TECH-005 collection 1 is the real test bench).
+
+**Resolution (2026-07-12, Sprint 3)**: shipped default-off on `feat/feat-017-parent-chunk-expansion`. Measured A/B on `eval-full` reingested dual (105 points = 22 parents + 83 children): dual chunking alone costs ~6.5pp retrieval hit vs fixed (children stop rolling overlap across 1500-token parent boundaries; IT-F-13, IT-R-02, EN-F-03 flip to miss). Full numbers in plan `20260712-sprint3-rag-quality` Notes.
 
 ---
 
@@ -357,6 +361,25 @@ The `title` field would contain `filename + page` (e.g., "Costituzione italiana.
 
 ---
 
+### TECH-007: Multi-part questions wiped by rerank+threshold funnel (multi-chunk collapse root cause)
+
+**Status**: planned | **Priority**: high | **Created**: 2026-07-12
+**Origin**: FEAT-017 measurement (plan `20260712-sprint3-rag-quality`) - expansion turned out to be downstream of the real failure.
+
+**Context**: on `eval-full`, 9/10 multi-chunk questions end with `retrieval_filter before=5 after=0` → `no_relevant_context` → refusal, despite 90% raw retrieval hit for the category. Cause: bge-reranker-v2-m3 scores each partial-answer chunk of a comparative/multi-part question low (each chunk answers only one part), and `VEKTRA_MIN_RELEVANCE_SCORE=0.15` — calibrated in the tuning sprint on single-fact questions (DEBT-010) — wipes the entire candidate set. Evidence (MC-01, eval mode traces): max reranker score 0.088 on a candidate whose raw RRF score was 0.61. Parent expansion (FEAT-017) never runs because zero results survive the filter.
+
+**Candidate directions** (evaluate, do not assume): (a) floor semantics - keep top-N post-rerank chunks regardless of threshold when the raw retrieval score was strong (e.g. min(top_k, after_rerank) >= 2); (b) per-category or per-score-source thresholds (reranker scores are not calibrated on the same scale as RRF); (c) query decomposition for multi-part questions (rewrite step already exists, ARCH-061); (d) rescore against the parent text instead of the child (combines with FEAT-017).
+
+**Traceability**: ARCH-056 (retrieval quality controls), ADR-0021, DEBT-010, FEAT-017, TECH-005
+
+**Acceptance criteria**:
+- [ ] Reproduce with the eval harness and document the score distributions per category
+- [ ] Chosen mitigation implemented behind config, default preserving current single-fact behavior
+- [ ] `eval-full` multi-chunk grounded moves from 0-1/10 without regressing factual (19/21) or adversarial refusals (no answered-without-context)
+- [ ] Decision and numbers recorded in the sprint plan and vektra-internal
+
+---
+
 ### FEAT-024: Remote embedding and reranker providers (TEI)
 
 **Status**: planned | **Priority**: medium | **Created**: 2026-07-12

@@ -116,7 +116,7 @@ the backlog assumptions:
 - [x] Autouse fixture in `vektra-shared/tests/conftest.py` scrubs `VEKTRA_*` + external keys
 - [x] `make test` green with populated `.env` (638 passed) + `make lint` green
 - [x] Backlog entry updated (completed + resolution), changelog entry
-- [ ] PR created and merged
+- [x] PR created and merged (#84; #85 BUG-021, #86 backlog, #87 BUG-022 merged the same day)
 
 ### 2. Baseline eval (no branch; results recorded, not committed as code)
 - [x] Eval corpus verified intact in namespace `default` (excerpt corpus, 12 chunks — no reingest needed)
@@ -126,19 +126,23 @@ the backlog assumptions:
 - [x] Numbers logged in this plan (Notes) and in vektra-internal (`stack/20260712-sprint3-baseline-eval.md`)
 
 ### 3. FEAT-017 — parent chunk expansion (branch `feat/feat-017-parent-chunk-expansion`)
-- [ ] Propagate `parent_id` into `ChunkEmbedding` → Qdrant payload + pgvector column;
-      keep deterministic child/parent ids at store time
-- [ ] Search excludes `chunk_level=parent` by default (both providers)
-- [ ] `VEKTRA_PARENT_EXPANSION_ENABLED` (default false) in `QueryPipelineConfig`
+- [x] Propagate `parent_id` into `ChunkEmbedding` → Qdrant payload + pgvector column;
+      keep deterministic child/parent ids at store time (pgvector now honors
+      caller-provided uuid5 ids instead of generating uuid4)
+- [x] Search excludes `chunk_level=parent` by default (both providers)
+- [x] `VEKTRA_PARENT_EXPANSION_ENABLED` (default false) in `QueryPipelineConfig`
       + mirrored in `VektraSettings`
-- [ ] Expansion step in AdvancedQueryPipeline: fetch parent text by id via vector
-      store, replace child text **before** token budgeting (ARCH-055); dedup children
-      of the same parent (child text is a substring of parent → existing 80% overlap
-      dedup interacts)
-- [ ] Trace metadata records expansion (children expanded, parents fetched)
-- [ ] Reingest eval corpus with `dual` strategy; measure: dual+exclusion without
-      expansion (≈ fixed baseline expected), then with expansion; record both
-- [ ] Unit tests: store-time linkage, search filter, expansion logic, budget accounting
+- [x] Expansion step in AdvancedQueryPipeline: fetch parent text by id via new
+      `VectorStoreProvider.retrieve()`, replace child text **before** token
+      budgeting (ARCH-055); children of the same parent collapse into the
+      highest-scored one (runs after the 80% overlap dedup, so no interaction)
+- [x] Trace metadata records expansion (children_expanded, siblings_merged,
+      parents_fetched)
+- [x] Reingest eval corpus with `dual` strategy; measured both (see Notes):
+      A dual-no-expansion ≈ baseline on e2e but -6.5pp retrieval hit (boundary
+      effect); B expansion-on: zero e2e flips, sources 1.4 → 1.0
+- [x] Unit tests: store-time linkage, search filter, expansion logic, budget
+      accounting (17 new tests; suite 658 passed)
 
 ### 4. FEAT-018 — verification first (no branch unless justified)
 - [ ] Multi-turn scenario runner against `/api/v1/query` with `conversation_id`
@@ -247,3 +251,30 @@ the bundle and a manual smoke in the Moodle dev stack.
   - March numbers (factual 90 / reasoning 80 / multi-chunk 10) are not comparable:
     different pipeline (pre BUG-015/016/017, pre FEAT-020) and per-question
     results were never versioned (gitignored file, since overwritten).
+- 2026-07-12: **FEAT-017 measured** (full corpus `eval-full`, Combo D, same
+  dataset/config as baseline; corpus reingested with `dual` 500/100, parent 1500:
+  105 points = 22 parents + 83 children, parents excluded from search).
+  Baseline to compare (fixed 500/100, 78 chunks): retrieval hit 89.1% /
+  MRR 0.8062 / P@5 0.3174; e2e grounded 35/55, multi-chunk 0/10, avg sources 1.4.
+  - **Measure A** (dual + parent exclusion, expansion off): retrieval hit 82.6% /
+    MRR 0.7029 / P@5 0.3130; e2e grounded 35/55 (64%), multi-chunk 1/10, avg
+    sources 1.4, p50 3935ms. Dual chunking itself costs ~6.5pp hit rate on this
+    corpus: children no longer roll overlap across parent boundaries, and 3
+    questions whose keywords straddle a 1500-token section edge flip to miss
+    (IT-F-13, IT-R-02, EN-F-03).
+  - **Measure B** (expansion on): e2e grounded 35/55, multi-chunk 1/10, avg
+    sources 1.0 (sibling merge working: same grounding with a more compact
+    prompt), p50 3813ms. Zero per-question flips vs A. Expansion verified live
+    in traces (`parent_expansion` step, children_expanded/parents_fetched).
+  - **Root cause of the multi-chunk collapse found (and it is upstream of
+    FEAT-017)**: 9/10 multi-chunk questions end with `retrieval_filter
+    before=5 after=0` — bge-reranker-v2-m3 scores each partial-answer chunk
+    of a comparative/multi-part question below `min_relevance_score` 0.15
+    (MC-01 with eval mode: max rerank score 0.088 while the raw RRF candidate
+    was 0.61). The whole candidate set is wiped before expansion can run.
+    Filed as TECH-007. Parent expansion works as designed but cannot touch
+    this failure mode; the Costituzione corpus also understates its benefit
+    (short self-contained articles — see TECH-005 collection 1).
+  - Local dev `.env` now: `VEKTRA_CHUNKING_STRATEGY=dual`,
+    `VEKTRA_PARENT_CHILD_LEVELS=1`, `VEKTRA_PARENT_EXPANSION_ENABLED=true`,
+    `VEKTRA_EVAL_MODE=true` (left on for FEAT-018 trace inspection).
@@ -19,11 +19,20 @@ Convention (Keep a Changelog 1.1.0):
 
 <!-- Add entries under: Added, Changed, Deprecated, Removed, Fixed, Security -->
 
+### Added
+
+- **rag**: optional parent chunk expansion in the advanced query pipeline (FEAT-017, `VEKTRA_PARENT_EXPANSION_ENABLED`, default off). With `VEKTRA_CHUNKING_STRATEGY=dual`, retrieved child chunks are replaced with their parent chunk's text after the retrieval filter and before token budgeting; children of the same parent collapse into the highest-scored one. Parent-child linkage is now actually persisted (deterministic `uuid5(doc_id, position)` ids, `parent_id` in the Qdrant payload and in the pgvector column), a new `VectorStoreProvider.retrieve()` fetches chunks by id, and the trace records `children_expanded`/`siblings_merged`/`parents_fetched` in a `parent_expansion` step.
+
+### Changed
+
+- **index**: vector search now excludes parent-level chunks (`chunk_level=parent`) in both providers; parents are context material fetched by id during expansion, not retrieval targets. Only affects documents ingested with `dual` chunking, whose parents previously polluted search results.
+- **index**: pgvector `store()` honors caller-provided UUID chunk ids (deterministic ids from ingest) instead of always generating random ones; non-UUID ids still fall back to random.
+
 ### Fixed
 
 - **docker**: the `INSTALL_UNSTRUCTURED=true` image variant builds again (BUG-022). torchvision (transitive via unstructured-inference) resolved from PyPI with CUDA-built wheels while torch is pinned to the CPU index, crashing the build with `operator torchvision::nms does not exist`. It is now declared in the `ocr` extra and pinned to the pytorch-cpu index; a new path-filtered CI workflow builds the OCR variant so it cannot silently regress.
 - **index**: `/api/v1/search` now resolves the embedding, sparse-embedding, and vector-store providers from the ProviderRegistry instead of hardcoding pgvector and reading a never-populated `app.state` attribute (BUG-021). In Qdrant deployments the endpoint returned zero results (it searched the empty `document_chunks` table) and hybrid mode always fell back to dense; the RAG pipeline (`/api/v1/query`) was unaffected. Found by the Sprint 3 baseline `make eval-retrieval` run.
-- **tests**: unit tests are now hermetic against the developer's local `.env` (DEBT-025). Importing litellm during pytest collection loads `.env` into the process environment, which made 4 default-assertion tests in `vektra-shared` fail on dev machines while CI stayed green. An autouse fixture in `vektra-shared/tests/conftest.py` scrubs ambient `VEKTRA_*` variables; production settings loading is unchanged.
+- **tests**: unit tests are now hermetic against the developer's local `.env` (DEBT-025). Importing litellm during pytest collection loads `.env` into the process environment, which made 4 default-assertion tests in `vektra-shared` fail on dev machines while CI stayed green. An autouse fixture in `vektra-shared/tests/conftest.py` scrubs ambient `VEKTRA_*` variables; production settings loading is unchanged. The fixture is replicated in `vektra-core` and `vektra-ingest` (FEAT-017 surfaced the same leak there: ambient `VEKTRA_CHUNKING_STRATEGY=dual` broke 12 chunker and pipeline-default tests).
 
 ## [0.5.1] - 2026-07-12
 

@@ -81,6 +81,7 @@ The model name must match the vLLM `--model` path exactly (e.g., `/models/qwen35
 | `VEKTRA_QUERY_PIPELINE` | str | `advanced` | Pipeline implementation: `simple`, `advanced` |
 | `VEKTRA_MIN_RELEVANCE_SCORE` | float | `0.15` | Minimum relevance score for chunk inclusion (0.0-1.0). Safety net filter; top-k is the primary control. |
 | `VEKTRA_CHUNK_DEDUP_ENABLED` | bool | `true` | Deduplicate overlapping adjacent chunks from the same document |
+| `VEKTRA_PARENT_EXPANSION_ENABLED` | bool | `false` | Replace retrieved child chunks with their parent chunk text before prompt construction (advanced pipeline only). Requires documents ingested with `VEKTRA_CHUNKING_STRATEGY=dual`. |
 | `VEKTRA_RESPONSE_TOKEN_RESERVE` | int | `2048` | Tokens reserved for LLM response generation |
 | `VEKTRA_CONTEXT_CHUNK_RATIO` | float | `0.6` | Fraction of context window allocated to retrieved chunks (0.0-1.0) |
 | `VEKTRA_PROMPT_TEMPLATES_DIR` | str | - | Directory for custom Jinja2 prompt templates (`system.j2`, `context.j2`, `conversation.j2`). Uses built-in defaults if unset. |

@@ -0,0 +1,52 @@
+# RAG evaluation harness (TECH-002)
+
+Two-stage evaluation against a running Vektra stack. Both scripts are pure HTTP
+clients: they require `VEKTRA_API_URL` and `VEKTRA_API_KEY` in the environment.
+
+```bash
+make eval-retrieval    # /api/v1/search (no LLM) - hit rate, MRR, precision@k
+make eval-e2e          # /api/v1/query (full pipeline + LLM) - grounded rate, latency
+# extra args:
+make eval-retrieval EVAL_ARGS="--dataset tests/eval/dataset-full.jsonl --output tests/eval/results_retrieval_full.jsonl"
+```
+
+Results are written as JSONL next to the datasets (`results_*.jsonl`, gitignored:
+record aggregates in the active `.s2s/plans/` file and in vektra-internal).
+
+## Datasets
+
+| File | Namespace | Corpus |
+|------|-----------|--------|
+| `dataset.jsonl` | `default` | Excerpt corpus, 12 chunks: `costituzione_italiana.md` v2 (6), `udhr_excerpts.md` v2 (4), `sample.pdf` (2). Hit rate saturates here; useful for smoke/regression, not for tuning. |
+| `dataset-full.jsonl` | `eval-full` | Full-document corpus, 78 chunks: clean full Italian Constitution (72) + official UDHR English PDF (6). Same 55 questions, discriminative metrics. |
+
+Both files share the same 55 questions (21 factual, 15 reasoning, 10 multi-chunk,
+9 adversarial without ground truth; 37 IT / 18 EN). Entry shape:
+`{id, question, expected_keywords, namespace, category, language}`. Relevance is
+keyword-based (`expected_keywords`, diacritic-insensitive substring match), so it
+is chunking-independent and survives reingestion.
+
+## Corpus provenance (eval-full)
+
+- `costituzione-full-clean.md`: full text of the Italian Constitution converted
+  from Wikisource (`https://it.wikisource.org/api/rest_v1/page/html/Costituzione_della_Repubblica_italiana`,
+  CC BY-SA), metadata header stripped. Kept outside the repo at
+  `/mnt/ai/datasets/vektra-eval/` on the dev machine.
+- `udhr-en-ohchr.pdf`: official OHCHR English UDHR
+  (`https://www.ohchr.org/sites/default/files/UDHR/Documents/UDHR_Translations/eng.pdf`).
+- Do NOT use the senato.it combined PDF (`costituzione.pdf`, 506 pages): its print
+  layout breaks pdfplumber extraction (fused words like
+  `COSTITUZIONEDELLAREPUBBLICAITALIANA`, preserved hyphenation) and invalidates
+  keyword ground truth. Measured impact: IT hit rate 72% garbled vs 88% clean on
+  the same questions. Worth keeping in mind as a future "dirty extraction" test
+  case, but never as a retrieval-quality corpus.
+
+To rebuild the corpus: download the two sources, then
+`scripts/ingest.sh <file> eval-full` for each.
+
+## Adding questions
+
+Append JSONL entries with a unique `id`, the target `namespace`, and 1-3
+`expected_keywords` that only appear in the passages that truly answer the
+question. Adversarial entries (expected refusal) omit `expected_keywords` and are
+reported separately (`has_ground_truth: false`).