Skip to content

feat(sdd): add configurable search strategy for code-reading phases#267

Open
Lotto724 wants to merge 3 commits into
Gentleman-Programming:mainfrom
Lotto724:feat/sdd-search-strategy
Open

feat(sdd): add configurable search strategy for code-reading phases#267
Lotto724 wants to merge 3 commits into
Gentleman-Programming:mainfrom
Lotto724:feat/sdd-search-strategy

Conversation

@Lotto724
Copy link
Copy Markdown

@Lotto724 Lotto724 commented Apr 9, 2026

Summary

  • Add search_strategy config (grep | hybrid) to SDD skill system, enabling MCP RAG-based semantic code search with grep fallback
  • New Section E in sdd-phase-common.md defines the cascade protocol: RAG → Grep → targeted Read
  • Step 3.5 in sdd-init auto-detects MCP RAG tools (same pattern as strict_tdd — no interactive prompts)
  • Optional reindex_tool fires once per sdd-apply batch (fire-and-forget)
  • Backwards compatible: missing config = grep mode = current behavior unchanged

Closes #266

Changes

File What changed
_shared/sdd-phase-common.md Section E: Code Search Protocol (cascade, mode branching, reindex hook, fallback)
_shared/persistence-contract.md search_strategy schema in sub-agent context
sdd-init/SKILL.md Step 3.5 (MCP auto-detection) + config output + 3 summary variants
sdd-explore/SKILL.md Step 3 reads config + references Section E
sdd-apply/SKILL.md Step 2 reads+caches config via Section E + Step 6 reindex hook
sdd-verify/SKILL.md Step 4 reads config + references Section E
testdata/golden/sdd-*-init.golden Regenerated (7 files)

Design decisions

  • Two modes only (grep / hybrid) — no standalone rag mode (too risky without fallback)
  • Follows strict_tdd pattern: detect in sdd-init → persist → phases branch on config
  • MCP-agnostic: stores tool name, not product — works with any RAG MCP server
  • Reindex is optional: fire-and-forget after batch, never per-file; failure is non-fatal

Test plan

  • 37/37 Go test packages passing (golden files regenerated)
  • SDD verify: 20/20 spec scenarios compliant
  • Judgment Day: 3 rounds, both judges APPROVED
  • Manual: run sdd-init without RAG MCP server → confirm grep default, zero change
  • Manual: run sdd-init with RAG MCP server → confirm hybrid auto-detected
  • Manual: run sdd-explore in hybrid mode → confirm RAG → grep fallback cascade

Copy link
Copy Markdown
Contributor

@Alan-TheGentleman Alan-TheGentleman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: feat(sdd): add configurable search strategy for code-reading phases

Nice work on the overall design — the pattern reuse from strict_tdd, backwards compatibility, and graceful degradation are all solid. One critical gap before this can merge:

Critical: Missing orchestrator forwarding

The search_strategy config is detected in sdd-init and consumed by sdd-explore/sdd-apply/sdd-verify, but the orchestrator instruction files (internal/assets/*/sdd-orchestrator.md) are not updated. There is no "Search Strategy Forwarding" section analogous to the existing "Strict TDD Forwarding (MANDATORY)" block.

Without this, the orchestrator will never pass search_strategy to sub-agents — they start with a fresh context and can't discover the config on their own.

Fix: Add a forwarding section to all 7 engine orchestrator files, mirroring the strict TDD forwarding pattern. Search for sdd-init/{project}, extract search_strategy, inject it into prompts for the 3 code-reading phases.

Minor suggestions

  1. sdd-apply Step 2 item 3: Dense run-on sentence. Consider splitting into 3a (read search_strategy + cache) and 3b (read existing code via Section E).

  2. Reindex hook placement: Sits between Step 6 and Step 7 without a step number. A "Step 6b" label would be more consistent.

  3. Auto-detection heuristic: Consider noting that the tool description should mention "semantic"/"vector"/"embedding" to avoid false positives on generic code_search tools.

  4. Section E ">=2 relevant snippets": Threshold is subjective — consider clarifying what "relevant" means or dropping the number.

The critical fix is mechanical (7 orchestrator files + goldens), then this is good to go.

@Lotto724
Copy link
Copy Markdown
Author

Lotto724 commented Apr 13, 2026

Updated the PR to address all review feedback:\n- Added mandatory search_strategy forwarding in all orchestrator instructions (mirrors strict TDD forwarding).\n- Clarified sdd-apply Step 2 (split 3a/3b) and labeled reindex as Step 6b.

  • Tightened auto-detection heuristic for RAG tools.
  • Clarified hybrid search sufficiency criteria.
  • Regenerated all SDD golden files.

Ready for re-review.

Copy link
Copy Markdown
Contributor

@Alan-TheGentleman Alan-TheGentleman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This touches the SDD search architecture, so process alone is not enough here.

Please add the required type:feature label, refresh CI, and document the design boundary clearly: detection contract, fallback behavior, and why grep remains the safe default.

Add search_strategy config (grep|hybrid) to SDD skill system, enabling
MCP RAG-based semantic code search with grep fallback for code-reading
phases (sdd-explore, sdd-apply, sdd-verify).

- Section E in sdd-phase-common.md: cascade protocol (RAG → Grep → Read)
- Step 3.5 in sdd-init: auto-detection of MCP RAG tools
- Config schema: search_strategy.mode + rag.mcp_tool + rag.reindex_tool
- Consumer phases reference Section E for code search
- Reindex hook in sdd-apply: fire-and-forget after batch
- Backwards compatible: missing config = grep = current behavior

Closes Gentleman-Programming#266
@Lotto724 Lotto724 force-pushed the feat/sdd-search-strategy branch from 068589a to 537dfbe Compare May 10, 2026 13:57
@Lotto724
Copy link
Copy Markdown
Author

Updated the PR again after rebasing on the latest upstream main.\n\nWhat changed:\n- Rebased the branch onto current main and resolved the SDD asset refactors.\n- Moved the code search protocol references to Section F because Section E is now the Review Workload Guard.\n- Extended search_strategy forwarding coverage to all 11 orchestrator assets, including opencode, kiro, kimi, and qwen.\n- Added deterministic contract tests for the search strategy boundary and forwarding behavior.\n- Added docs/sdd-search-strategy.md documenting the detection contract, fallback behavior, why grep remains the safe default, and local grep-style baseline measurements.\n- Regenerated SDD golden files.\n\nValidation run:\n- go test ./internal/assets\n- go test ./internal/components -run TestGoldenSDD\n\nNote: I could not add the type:feature label from this fork due to GitHub label permissions on the upstream repository.

@Lotto724
Copy link
Copy Markdown
Author

Small permissions update:\n\n- I tried to add the requested type:feature label, but GitHub returned a permissions error for this fork: AddLabelsToLabelable is not allowed for Lotto724. A maintainer will need to add that label from the upstream repository.\n- The latest push did trigger the PR workflows, but both runs are currently marked action_required, which appears to require maintainer approval for forked PR workflows:\n - CI: https://github.com/Gentleman-Programming/gentle-ai/actions/runs/25630593454\n - PR Validation: https://github.com/Gentleman-Programming/gentle-ai/actions/runs/25630593455\n\nEverything else requested in the review has been addressed in the branch.

@Alan-TheGentleman
Copy link
Copy Markdown
Contributor

Heads-up: this PR is now showing a merge conflict against main. We shipped v1.31.0 (and a number of follow-up fixes) over the past two days — several files this PR touches were also modified upstream.

Could you rebase against current main and resolve the conflicts?

git fetch upstream main
git rebase upstream/main
# resolve any conflicts
git push --force-with-lease

Once it's green again I can review for merge. Sorry for the drift — it's the cost of the cleanup velocity. Thanks for sticking with it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:feature New feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(sdd): configurable search strategy for code-reading phases

3 participants