Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions docs/reference/ignore-eject.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,10 @@
`/factlog sync` re-extracts **every** source on each run. To keep specific
sources out of that — a draft, a work-in-progress, an external doc — add them to
the per-KB **sync-ignore list** (`policy/sync-ignore.md`). Ignored sources are
skipped by `/factlog sync`, `factlog ingest --scan`, and coverage gap reporting,
**even when modified**. Their already-merged facts are kept untouched (use
`factlog eject` to actually remove a fact).
skipped by `/factlog sync`, `factlog ingest --scan`, coverage gap reporting, and
the wiki-exploration evidence in `/factlog ask`, **even when modified**. Their
already-merged facts are kept untouched (use `factlog eject` to actually remove
a fact).

```bash
factlog ignore drafts/*.md sources/wip-notes.md # add pattern(s)
Expand Down
5 changes: 3 additions & 2 deletions docs/reference/ignore-eject.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@
`/factlog sync` 는 매 실행마다 **모든** 소스를 다시 추출합니다. 특정 소스를
그 대상에서 빼려면 — 초안, 작업 중인 문서, 외부 문서 등 — KB별 **sync-ignore
목록**(`policy/sync-ignore.md`)에 추가하십시오. 무시된 소스는 **수정되더라도**
`/factlog sync`, `factlog ingest --scan`, 커버리지 누락 보고에서 건너뜁니다. 이미
머지된 사실은 그대로 유지됩니다(사실을 실제로 제거하려면 `factlog eject` 사용).
`/factlog sync`, `factlog ingest --scan`, 커버리지 누락 보고와 `/factlog ask`의
wiki 탐색 근거에서 건너뜁니다. 이미 머지된 사실은 그대로 유지됩니다(사실을 실제로
제거하려면 `factlog eject` 사용).

```bash
factlog ignore drafts/*.md sources/wip-notes.md # add pattern(s)
Expand Down
16 changes: 9 additions & 7 deletions factlog/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -472,9 +472,10 @@ def cmd_doctor(_args: argparse.Namespace) -> int:
# Sync-ignore list
#
# Source files matching these glob patterns are SKIPPED by `/factlog sync`
# (re-extraction), `factlog ingest --scan`, and coverage gap reporting — even
# when modified. Their already-merged facts are KEPT (use `factlog eject` to
# remove those). Manage with `factlog ignore [--remove] <pattern>`.
# (re-extraction), `factlog ingest --scan`, coverage gap reporting, and
# `/factlog ask` wiki exploration — even when modified. Their already-merged
# facts are KEPT (use `factlog eject` to remove those). Manage with
# `factlog ignore [--remove] <pattern>`.
#
# One pattern per line; '#' comments and '-' bullets allowed; quote a pattern
# with spaces (or one starting with '#') in `backticks`. A pattern matches a
Expand Down Expand Up @@ -1373,12 +1374,13 @@ def _matches_extended(r: dict) -> bool:


def cmd_ignore(args: argparse.Namespace) -> int:
"""Manage policy/sync-ignore.md — glob patterns of sources excluded from sync.
"""Manage policy/sync-ignore.md — sources excluded from sync and wiki evidence.

No patterns: list current entries and the on-disk sources each matches.
With pattern(s): add them, or remove them with --remove. Excluding a source
only stops its re-extraction (ingest --scan / sync / coverage); its already-
merged facts are untouched (use `factlog eject` to remove those).
stops its re-extraction (ingest --scan / sync / coverage) and keeps it out
of `/factlog ask` wiki evidence; its already-merged facts are untouched
(use `factlog eject` to remove those).
"""
import re
import unicodedata
Expand Down Expand Up @@ -2760,7 +2762,7 @@ def build_parser() -> argparse.ArgumentParser:

ignore = sub.add_parser(
"ignore",
help="manage policy/sync-ignore.md: glob patterns of sources excluded from sync",
help="manage policy/sync-ignore.md: glob patterns excluded from sync and wiki evidence",
)
ignore.add_argument("patterns", nargs="*", help="glob/path pattern(s) to add (omit to list)")
ignore.add_argument("--remove", action="store_true", help="remove the given pattern(s) instead of adding")
Expand Down
5 changes: 3 additions & 2 deletions skills/factlog/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -385,8 +385,9 @@ Sources matching `policy/sync-ignore.md` are skipped. Then extract from **both**
source whose path matches one of its glob patterns — by full ref (`sources/...`
or `runs/sources/...`) or by the path within the source root (so `drafts/*.md`
matches `sources/drafts/x.md`). These sources are excluded from re-extraction on
purpose; their already-merged facts are left as-is. (Manage the list with
`factlog ignore`.)
purpose and are also excluded from `/factlog ask` wiki-exploration evidence;
their already-merged facts are left as-is. (Manage the list with `factlog
ignore`.)

For each *non-ignored* file under `sources/<name>` **and** `runs/sources/<name>`
in the KB root:
Expand Down
15 changes: 15 additions & 0 deletions tests/test_ask_router.sh
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,21 @@ if router search "WidgetX ToolA" | "$PYTHON" -c "import json,sys; d=json.load(sy
# pages/ candidate content must never appear in search citations
if router search "Datadog may_use" | grep -qE 'pages/|may_use|confidence=0\.40'; then bad "pages/ candidate content leaked into search results"; else ok "pages/ excluded from search (no candidate leak)"; fi

# Sync-ignored primary sources are deliberately excluded from wiki evidence;
# non-ignored sources and supplementary decisions retain their normal behavior.
mkdir -p "$KB/sources/drafts" "$KB/runs/sources/drafts"
printf 'syncignoretoken appears only in an ignored original.\n' > "$KB/sources/drafts/ignored.md"
printf 'syncignoretoken appears only in an ignored conversion.\n' > "$KB/runs/sources/drafts/ignored.md"
printf 'syncignoretoken appears in the retained source.\n' > "$KB/sources/retained.md"
printf -- '- drafts/**\n' >> "$KB/policy/sync-ignore.md"
ignored_search="$(router search "syncignoretoken")"
if printf '%s' "$ignored_search" | grep -qE 'sources/drafts/ignored\.md|runs/sources/drafts/ignored\.md'; then bad "sync-ignored sources leaked into search results"; else ok "sync-ignored sources excluded from search results"; fi
if printf '%s' "$ignored_search" | grep -qF 'sources/retained.md'; then ok "non-ignored source remains in search results"; else bad "non-ignored source missing from search results"; fi
if router wiki "syncignoretoken" | grep -qE 'sources/drafts/ignored\.md|runs/sources/drafts/ignored\.md'; then bad "sync-ignored sources leaked into rendered wiki evidence"; else ok "sync-ignored sources excluded from rendered wiki evidence"; fi
rm -rf "$KB/sources/drafts" "$KB/runs/sources/drafts" "$KB/sources/retained.md"
# Keep later search cases independent from this filtering fixture.
sed -i.bak '/^- drafts\/\*\*$/d' "$KB/policy/sync-ignore.md" && rm -f "$KB/policy/sync-ignore.md.bak"

wiki_out="$(router wiki "what uses FastAPI" --reason "unknown entity")"
if printf '%s' "$wiki_out" | grep -qF "UNVERIFIED — wiki exploration"; then ok "wiki answer carries UNVERIFIED marker"; else bad "wiki answer missing UNVERIFIED marker"; fi
if printf '%s' "$wiki_out" | grep -qF "sources/acme.md:"; then ok "wiki answer cites a source path:line"; else bad "wiki answer missing citation"; fi
Expand Down
11 changes: 10 additions & 1 deletion tools/ask_router.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@
policy_predicates,
relation_aliases,
run_wirelog,
is_sync_ignored,
sync_ignore_patterns,
)
from factlog import literal_types # noqa: E402

Expand Down Expand Up @@ -653,6 +655,7 @@ def search(question: str, root: Path, *, limit: int = 10) -> list[dict[str, obje
if not patterns:
return []
scored: list[tuple[tuple[int, int], dict[str, object]]] = []
ignored_patterns = sync_ignore_patterns(root)
for rel, label in _wiki_corpus():
base = root / rel
if not base.is_dir():
Expand All @@ -662,6 +665,12 @@ def search(question: str, root: Path, *, limit: int = 10) -> list[dict[str, obje
# Stay within the corpus root: never follow a symlink out of the KB.
if not path.resolve().is_relative_to(base_resolved):
continue
ref = path.relative_to(root).as_posix()
# Sync-ignore means this primary source is not evidence for wiki
# exploration either. Supplementary decisions remain searchable:
# they are explicitly labeled and are not source files.
if rel in WIKI_SOURCE_DIRS and is_sync_ignored(ref, ignored_patterns):
continue
try:
text = path.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
Expand All @@ -681,7 +690,7 @@ def search(question: str, root: Path, *, limit: int = 10) -> list[dict[str, obje
last_end = end - 1
excerpt = "\n".join(_sanitize(line_text) for line_text in lines[start:end])
result = {
"file": path.relative_to(root).as_posix(),
"file": ref,
"line": i + 1,
"excerpt": excerpt,
"dir": label,
Expand Down