Skip to content

feat(learner): collapse cross-OS duplicate projects in the registry#22

Open
Sergio-LPA wants to merge 2 commits into
Luispitik:mainfrom
Sergio-LPA:feat/crossos-registry
Open

feat(learner): collapse cross-OS duplicate projects in the registry#22
Sergio-LPA wants to merge 2 commits into
Luispitik:mainfrom
Sergio-LPA:feat/crossos-registry

Conversation

@Sergio-LPA

Copy link
Copy Markdown

Problem

The per-project id is sha256(remote || root). For a project with no git remote, the id is derived from the root — which differs per OS:

  • macOS/Linux: /Users/me/Proj
  • Windows: C:/Users/Me/Proj

So the same project gets a different id on each machine. When the registry (_sinapsis-projects.json) is shared across machines — e.g. a Nextcloud/Dropbox/iCloud-synced ~/.claude — that one project shows up as two entries, duplicating it in /projects, /eod, and cross-project instinct search.

Projects with a remote are unaffected (the remote is identical across machines, so the id already matches).

Fix

Match on a cross-OS-stable key instead of the raw id: the git remote when present, otherwise the project name. The two sightings then collapse into a single entry:

  • each per-OS id is preserved in aliases[] (so future sessions resolve without a re-scan)
  • each observed root is stored under roots.{posix,windows}
  • the remote key takes precedence over the name, so two genuinely different projects that happen to share a folder name never merge

The change is purely additivealiases, roots and crossKey are new optional fields; existing readers that don't use them are unaffected. Behaviour for remote-backed projects is unchanged.

Tests

New tests/test-crossos-registry.sh — 5 hermetic tests driving the real hook via a sandbox HOME:

  1. two-OS, no-remote sighting collapses into one entry
  2. both roots.posix and roots.windows recorded
  3. the second per-OS id registered as an alias
  4. remote key beats name (same-named distinct projects stay separate)
  5. idempotent on re-run

Existing suites re-run clean:

  • test-install-upgrade 21/21
  • test-registry-isolation 4/4
  • test-eod-gather 8/8

Notes

  • The name-based fallback could, in theory, merge two different projects that share a folder name across machines and have no remote on either. The remote key takes precedence to minimise this; documented in the code comment.
  • Context: surfaced on a real Mac + Windows setup syncing ~/.claude via Nextcloud, where non-git folders (the same logical project) appeared twice in /eod. Follow-on to the v4.5.1 cross-OS /eod work.

@Luispitik Luispitik left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The diagnosis is right and in scope: project ids are sha256(remote || root), so a no-remote project gets a different id per OS and duplicates in synced registries. The diff applies clean on v4.6.1 and the new code is safe (no eval, execFileSync with args). But I reproduced three functional problems by running the patched hook, so this needs another iteration:

  1. Same-machine false merge (regression, data loss). Two different no-remote projects that share a folder name on the same machine (think clientA/scripts and clientB/scripts) collapse silently into one entry, and one root disappears from the registry (roots.windows overwritten — same os-family). Reproduced with the real hook. /eod resolves roots by name, so it would attribute git commits to the wrong project. Fix: only collapse by name when the os-family actually differs, e.g.
    registry.projects.find(p => p && p.crossKey === crossKey && p.root && projectRoot
      && osFamily(p.root) !== osFamily(projectRoot))
  2. Pre-existing duplicates never merge. The lookup matches by id first, so a registry that already contains both entries (the synced-via-Nextcloud scenario that motivates this PR) keeps both forever — each machine just updates its own entry. Reproduced. Add a post-upsert migration pass that merges entries with identical crossKey (union of aliases/roots, oldest created, newest last_seen) — or hook it into _dream.sh hygiene. As written, the PR only prevents future duplicates from a clean registry; it doesn't cure the reported symptom.
  3. The new test fails on Windows Git Bash (suite exit 1; 1 of the 5 cases): MSYS converts the posix fixture path /Users/tester/CrossProj passed as argv into C:/Program Files/Git/Users/..., so roots.posix never registers. Export MSYS_NO_PATHCONV=1 / MSYS2_ARG_CONV_EXCL='*' at the top of the test, or pass fixture paths via env vars instead of argv. Also drop the || r.projects[0].id==='bbbbbbbbbbbb' clause in test 3 — as written it passes even when the posix run was never processed, which is exactly what happens on Git Bash today.

Please also:

  • Add an anti-regression test: two same-name no-remote projects with different roots in the same os-family must stay 2 entries.
  • Teach the consumers about aliases: commands/projects.md (step 3) should also count observations from homunculus/projects/{alias}/; _eod-gather.sh should register aliases in its registry[] map (for (const a of p.aliases||[]) registry[a] = {name:p.name, root:p.root}).
  • Document the known asymmetry in a code comment: if remote detection fails on one machine (git not on PATH for execFileSync), that sighting falls back to a name: crossKey and won't match the remote: entry — duplicate persists.

Good direction — with the cross-OS restriction and the migration pass this becomes a clean merge. Windows Git Bash compatibility is non-negotiable in this repo (it's where most users hit bugs).

@Sergio-LPA

Copy link
Copy Markdown
Author

Hola Luis, ¿pudiste echarle un ojo a este PR? Lo llevo corriendo en local desde principios de junio (Mac y PC Windows con el registry compartido por Nextcloud) sin ningún problema, y la regresión de upstream me salió verde (install 21/21, registry-isolation 4/4, eod-gather 8/8).

Si le ves algo raro o prefieres otro enfoque para el aliasing de roots, me dices y lo ajusto. Sin prisa 🙂

@Luispitik Luispitik left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cross-OS diagnosis is correct and the hermetic test harness (real hook, sandbox HOME, no mocks) is exactly how this repo tests — thank you. One design flaw blocks the merge as-is:

The name-key fallback merges two genuinely different projects — and not only across machines. crossKey = "name:" + projectName matches ANY other no-remote project with the same folder name, including on the same machine: ~/work/app and ~/personal/app (both no remote) get different ids (roots differ), but the second sighting finds the first entry via crossKey === "name:app" and collapses them — aliases them together, overwrites root/roots.posix, and from then on /projects, /eod and cross-project search treat two projects as one. Your Notes section flags the cross-machine variant as theoretical; the same-machine variant is the common case (demo, test, scratch, api…), and a false merge is worse than the duplicate it replaces — a duplicate is cosmetic, a merge corrupts the registry.

Suggested fix — restrict the name-key match to the actual cross-OS scenario: only accept a crossKey name match when the candidate root's OS family is NOT already recorded on the entry, e.g.:

|| (crossKey !== "name:" ? registry.projects.find(p => p && p.crossKey === crossKey
     && !(p.roots && p.roots[osFamily(projectRoot)])
     && osFamily(p.root || "") !== osFamily(projectRoot)) : null)

That still collapses the Mac+Windows sighting you're targeting (families differ), but two same-family same-name projects stay separate. Please add the regression test: two same-named no-remote projects on the SAME OS remain 2 entries.

Also needed after the fix:

  1. Rebase onto v4.6.2 — main now routes registry reads through a BOM-safe readJson() in _session-learner.sh (#25); your hunk sits right below it.
  2. Register tests/test-crossos-registry.sh in .github/workflows/tests.yml — the suite list is explicit, new suites don't run otherwise.
  3. Minor: osFamily classifies Git-Bash /c/... as windows but observe_v3.py normally emits C:/... cwds — fine, just add one test line covering the /c/ branch since you handle it.

Happy to merge once the name-key guard is in.

Luimarcabai pushed a commit to Luimarcabai/synapis that referenced this pull request Jul 20, 2026
Critical fixes:
- Luispitik#1-3: install.sh preserves user data on upgrade (instincts, rules, projects)
- Luispitik#4/5A: execSync → execFileSync in 3 hooks (command injection prevention)
- Luispitik#5: auto-promote works — drafts track occurrences without injecting
- Luispitik#6: race condition — instinct-activator checks dream lock before write

Security fixes:
- #5B: +4 secret scrubbing patterns (JWT, GitHub, AWS, PEM) in observe_v3.py
- #5C/Luispitik#12: ReDoS protection — reject nested quantifiers before RegExp
- #5D: chmod 600 on data files during install
- #5E/Luispitik#7: fcntl.flock() on JSONL append and rotation
- #5F: inject sanitization — 500 char limit + blocked prompt injection patterns

Bug fixes:
- Luispitik#8: token catalog corrected (9,995 vs 2,700 declared)
- Luispitik#9: install.bat synced to v4.3.1 with missing hooks
- Luispitik#10-11: instinct-status + passive-status use actual schemas
- Luispitik#13: Jaccard Unicode support (\p{L}\p{N})
- Luispitik#14: contradiction detection — \bdo\b scoped to actionable verbs
- Luispitik#15: session-end documents relationship with /eod
- Luispitik#16: tmpdir flag cleanup (>24h stale files)
- Luispitik#17: session-learner selects most recent project by mtime
- Luispitik#18: operator-state schema validation flag
- Luispitik#19: KNOWLEDGE_FILE dead code removed
- Luispitik#20: synapis → sinapsis directory naming + catalog IDs
- Luispitik#22: SINAPSIS_DEBUG=1 redirects stderr to debug log

Tests: 51/51 GREEN (15 install + 11 security + 25 dream cycle)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sergio-LPA and others added 2 commits July 24, 2026 19:34
The per-project id is sha256(remote || root). A project with no git remote
therefore gets a DIFFERENT id on each OS (its root differs: /Users/me/Proj
vs C:/Users/Me/Proj), so with a shared registry (e.g. a synced folder) the
same project appears twice — duplicated in /projects, /eod and cross-project
instinct search.

Match on a cross-OS-stable key instead: the git remote when present, else
the project name. The two sightings collapse into one entry — each per-OS id
kept in aliases[], each observed root under roots.{posix,windows}. Projects
with a remote already shared one id and are unaffected; the remote key beats
the name so two distinct same-named projects never merge.

Purely additive: aliases, roots and crossKey are optional fields ignored by
readers that don't use them.

New tests/test-crossos-registry.sh (5 hermetic tests). Existing suites pass:
test-install-upgrade 21/21, test-registry-isolation 4/4, test-eod-gather 8/8.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… (review)

Addresses both review rounds on Luispitik#22:

- Name-key fallback now matches ONLY in the genuine cross-OS scenario:
  the candidate entry's root must belong to a different OS family and the
  entry must have no root of the sighting's family recorded. Two same-named
  no-remote projects on the same machine (~/work/app vs ~/personal/app)
  stay separate — a false merge corrupts the registry where a duplicate is
  merely cosmetic.
- Post-upsert migration pass cures PRE-EXISTING duplicates (the synced-via-
  Nextcloud symptom motivating the PR): same-crossKey entries merge under
  the same family guard, union of aliases/roots, oldest created, newest
  last_seen; crossKey backfilled for legacy entries.
- Consumers learn aliases: _eod-gather.sh maps aliased per-OS ids to the
  merged project; /projects counts observations from alias directories.
- Known asymmetry documented in a comment: a sighting whose remote
  detection fails falls back to a name: key and won't match the remote:
  entry until the key upgrade heals it.
- Tests: suite now 10 hermetic cases — adds same-name same-family
  anti-regression (both roots intact), pre-existing duplicate migration
  (merge + union semantics), and the Git Bash /c/ osFamily branch.
  MSYS_NO_PATHCONV / MSYS2_ARG_CONV_EXCL exported so posix fixtures
  survive Git Bash; the id-fallback clause in the alias assertion removed.
  Suite registered in .github/workflows/tests.yml.
- Rebased onto v4.8.1 (readJson BOM-safe base).

All affected suites green: crossos-registry 10/10, install-upgrade 21/21,
registry-isolation, eod-gather, bom-decay-python, v45-opus47, v46-opus48.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Sergio-LPA
Sergio-LPA force-pushed the feat/crossos-registry branch from 8d5c38d to c4316cd Compare July 24, 2026 19:39
@Sergio-LPA

Copy link
Copy Markdown
Author

Hecho todo lo de la review, Luis:

  • Guard cross-OS en el name-key: el fallback por nombre ahora solo matchea cuando la familia de SO del root difiere Y la entrada no tiene ya un root de mi familia registrado (tu snippet, con early-return para las keys remote:). Dos proyectos homónimos sin remote en la misma máquina quedan separados — test de regresión incluido, con assert de que ninguno de los dos roots se pisa.
  • Pasada de migración post-upsert: cura los duplicados pre-existentes (el síntoma del registry sincronizado que motivó el PR). Merge por crossKey bajo el mismo guard de familias que el lookup, unión de aliases/roots, created más viejo, last_seen más nuevo. Con backfill de crossKey para entradas legacy, así los registries escritos antes de la feature también se curan.
  • Consumers con aliases: _eod-gather.sh resuelve ids aliased al proyecto fusionado, y commands/projects.md suma las observaciones de cada homunculus/projects/{alias}/.
  • Asimetría documentada en comentario en el código: si la detección de remote falla en una máquina, ese sighting cae a key name: y no matchea la entrada remote: — el duplicado persiste hasta que el key upgrade lo cura.
  • Git Bash: MSYS_NO_PATHCONV=1 + MSYS2_ARG_CONV_EXCL='*' al principio de la suite, fuera la cláusula || id==='bbbb...' del test 3 (el assert del alias ahora es estricto), y test nuevo cubriendo el branch /c/ de osFamily.
  • Rebase sobre v4.8.1 (el hunk asienta sobre el readJson BOM-safe) y suite registrada en .github/workflows/tests.yml.

La suite queda en 10 casos, 10/10 en local (Linux). Re-corridas las afectadas, todas en verde: install-upgrade 21/21, registry-isolation, eod-gather, bom-decay-python, v45-opus47 y v46-opus48.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants