Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,8 @@ Full instructions:
|---|---|
| `cmd/claude-memsync` | Daemon binary: lifecycle + watcher + sync loop |
| `cmd/claude-memmerge` | Git custom merge driver for `MEMORY.md` |
| `internal/sync` | Mirror, reconcile, watcher loop, manifest |
| `internal/sync` | Mirror, reconcile, watcher loop, manifest, layout migration |
| `internal/project` | Project fingerprinting (path resolution, key derivation, per-PC index) |
| `internal/merge` | Section-block parser + 3-way semantic merge |
| `internal/distill` | Distilled-memory catalog index + reconcile |
| `internal/gitwt` | Wrapper around the `git` CLI scoped to the sync work-tree |
Expand Down
20 changes: 19 additions & 1 deletion cmd/claude-memsync/init.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (

"github.com/MarimerLLC/claude-utils/internal/config"
"github.com/MarimerLLC/claude-utils/internal/gitwt"
"github.com/MarimerLLC/claude-utils/internal/project"
syncpkg "github.com/MarimerLLC/claude-utils/internal/sync"
)

Expand Down Expand Up @@ -130,14 +131,31 @@ func bootstrap(cfg config.Config, force bool) error {
return err
}
roots := syncpkg.Roots{Claude: cfg.ClaudeProjectsDir, Mirror: mirrorProjects}

// Build the fingerprint index and migrate any legacy path-keyed dirs the
// remote already carries (from a pre-fingerprint version or another machine)
// to their fingerprint keys before reconciling local memories in.
idx, err := project.Build(cfg.ClaudeProjectsDir)
if err != nil {
return fmt.Errorf("build project index: %w", err)
}
if migRep, err := syncpkg.MigrateLayout(roots, idx); err != nil {
return fmt.Errorf("migrate layout: %w", err)
} else {
printReport(migRep)
}

// First-time init: no manifest yet. Pass nil so Reconcile doesn't try
// to propagate deletes (with no prior state, "missing in Claude" can't
// be distinguished from "this PC never had it").
report, err := syncpkg.Reconcile(roots, nil)
report, err := syncpkg.Reconcile(roots, idx, nil)
if err != nil {
return fmt.Errorf("reconcile: %w", err)
}
printReport(report)
if err := idx.Save(cfg.SyncDir); err != nil {
fmt.Fprintln(os.Stderr, "save project index:", err)
}

// Step 6: stage + commit + push.
if err := repo.AddAll(); err != nil {
Expand Down
65 changes: 51 additions & 14 deletions docs/claude-memsync.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,12 @@ of its own — just a git remote you can push to.
inherits your normal git credentials (Git Credential Manager on Windows, SSH
agent or `~/.gitconfig` credential helpers on Linux/macOS). Confirm `git push`
works against the remote before installing.
- The same project paths on each PC. Claude's project-hash directory names are
derived from absolute paths, so if your repos live at the same drive letter /
mount point on every PC, you're set. (See [Limitations](#limitations--known-issues).)
- Repos checked out at **different paths on different PCs still merge**: the
daemon fingerprints each project (by its git remote URL, falling back to the
root-commit SHA) and keys the sync repo by that fingerprint rather than the
local path. Non-git directories keep the old path-based key and only merge
across PCs that use identical paths. (See [How it works](#how-it-works) and
[Limitations](#limitations--known-issues).)

## Setup

Expand Down Expand Up @@ -133,22 +136,45 @@ and SSH keys. None require root.
## How it works

```
~/.claude/projects/<hash>/memory/ (Claude reads + writes here — authoritative)
~/.claude/projects/<localHash>/memory/ (Claude reads + writes here — authoritative)
localHash = path with / → -, per machine
│ fsnotify watcher, 3s debounce
│ fingerprint the project → stable key
~/.claudesync/projects/<hash>/memory/ (mirror — git work-tree)
~/.claudesync/projects/<key>/memory/ (mirror — git work-tree, keyed by fingerprint)
│ git add -A, commit, pull --rebase, push
<remote> (private GitHub repo)
<remote> (private GitHub repo)
│ ls-remote check on 1h timer (or after local push);
│ full pull only when remote SHA actually moved
propagate pull-driven changes back to ~/.claude/projects/...
propagate pull-driven changes back to ~/.claude/projects/<localHash>/...
```

**Project fingerprinting.** Claude names each project directory by dash-encoding
its absolute path (`/Users/me/code/foo` → `-Users-me-code-foo`), which differs
per machine. Keying the sync repo by that name would split one repo into several.
Instead the daemon resolves each project's real path (from the `cwd` Claude
records in its session transcripts) and derives a machine-independent **key**:

- `g-<hash>` — from the normalized `origin` remote URL (all of
`git@github.com:Acme/app.git`, `https://github.com/acme/app`, and
`…/app.git` collapse to the same key).
- `r-<hash>` — from the git root-commit SHA, when the repo has no remote.
- the dash-encoded path itself — fallback for non-git directories (unchanged
behavior; only merges across PCs with identical paths).

A per-PC index at `~/.claudesync/.state/index.json` (gitignored) records the
`localHash → key` mapping for observability and to route inbound changes back to
the right local directory.

**Lazy materialization.** A project synced from another PC but never opened on
this one stays pending in the mirror — there's no local path to write it to yet.
The first time you open that project here, its directory appears, fingerprints to
the pending key, and the next reconcile copies its memories in.

Idle ticks cost a single `git ls-remote` round-trip — no pull, no push, no merge
driver invocation. The full pull/push cycle runs only when origin's branch SHA
differs from the local `refs/remotes/origin/<branch>` or when there are unpushed
Expand Down Expand Up @@ -176,6 +202,7 @@ commits.
`~/.claude/history.jsonl`, `~/.claude/settings.json`, etc.
- `~/.claudesync/config.json` — per-PC (paths embed your machine layout)
- `~/.claudesync/.state/manifest.json` — per-PC (delete-detection state)
- `~/.claudesync/.state/index.json` — per-PC (localHash → fingerprint key map)
- `~/.claudesync/daemon.pid` — runtime state
- `~/.claudesync/distilled/DISTILLED.md` — derived index, regenerated per-PC
(the distilled `<slug>.md` entry files themselves **are** synced)
Expand Down Expand Up @@ -206,7 +233,7 @@ This means:
## On-disk layout

```
~/.claude/projects/<hash>/memory/ # what Claude reads + writes
~/.claude/projects/<localHash>/memory/ # what Claude reads + writes
├── MEMORY.md
└── <topic>.md ...

Expand All @@ -217,10 +244,11 @@ This means:
├── .gitattributes # MEMORY.md merge=claude-memory-index
├── .gitignore # excludes config.json, .state/, etc.
├── .state/manifest.json # per-PC delete-detection state
├── .state/index.json # per-PC localHash → fingerprint key map
├── distilled/ # shared distilled-memory catalog
│ ├── DISTILLED.md # derived index (gitignored)
│ └── <slug>.md ... # distilled entries (synced)
└── projects/<hash>/memory/... # git work-tree mirror
└── projects/<key>/memory/... # git work-tree mirror (keyed by fingerprint)
```

## Auth notes
Expand All @@ -246,10 +274,19 @@ If that works without prompting, the daemon will too.

## Limitations / known issues

- **Path consistency required**: Claude derives the per-project memory directory
name by escaping the project's absolute path. PCs that open the same repo at
different paths (e.g. `C:\src\foo` vs. `D:\dev\foo`) will see them as different
projects.
- **Git projects merge across paths; non-git ones don't**: git repos are keyed
by their remote URL (or root-commit SHA), so opening the same repo at different
paths on different PCs still merges. A non-git directory falls back to a
path-based key, so it only merges across PCs that use the identical absolute
path.
- **Upgrading from a path-keyed layout**: the first daemon run after upgrading
migrates existing mirror directories to fingerprint keys, merging any that
collapse to the same key. Both PCs must be upgraded for a project's previously
divergent histories to converge on the shared key.
- **Fingerprint edge cases**: a shallow clone (`git clone --depth 1`) has no true
root commit, so a remote-less shallow repo falls back to the path key. A repo
whose `origin` URL is later changed (renamed org, new host) re-keys and will
re-merge under the new key.
- **No live conflict UI**: when the merge driver emits actual conflict markers
(rare; only on overlapping line edits within the same `MEMORY.md` section
body), the file is committed and pushed with markers. The next time you edit on
Expand Down
133 changes: 133 additions & 0 deletions internal/project/fingerprint.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
package project

import (
"crypto/sha256"
"encoding/hex"
"sort"
"strings"

"github.com/MarimerLLC/claude-utils/internal/gitwt"
)

// Method records how a project's key was derived, for diagnostics in the index.
type Method string

const (
MethodRemote Method = "remote" // normalized origin remote URL (preferred)
MethodRoot Method = "root" // git root-commit SHA (git repo, no remote)
MethodPath Method = "path" // fallback: Claude's dash-encoded dir name
)

// Key prefixes distinguish fingerprint keys from legacy path keys (which start
// with "-"), so migration can tell what to move and the two never collide.
const (
remotePrefix = "g-"
rootPrefix = "r-"
)

// Fingerprint computes the machine-independent mirror key for the project at
// realPath. localHash is Claude's directory name, used verbatim as the path
// fallback so non-git projects keep today's per-machine behavior (they simply
// won't merge across machines, but never merge with an unrelated project).
//
// Preference order: normalized origin remote URL, then git root-commit SHA when
// there is no remote, then the path fallback.
func Fingerprint(realPath, localHash string) (key string, method Method) {
if realPath != "" {
repo := gitwt.New(realPath)
if repo.IsRepo() {
if url, ok := originURL(repo); ok {
return remotePrefix + shortHash(NormalizeRemoteURL(url)), MethodRemote
}
if root, ok := rootCommit(repo); ok {
return rootPrefix + shortHash(root), MethodRoot
}
}
}
return localHash, MethodPath
}

func originURL(repo *gitwt.Repo) (string, bool) {
out, err := repo.Run("config", "--get", "remote.origin.url")
if err != nil {
return "", false
}
url := strings.TrimSpace(out)
return url, url != ""
}

// rootCommit returns a stable identifier built from the repository's root
// commit(s). A shallow clone's history is truncated, so its "root" is a
// per-clone boundary commit rather than the true first commit — useless as an
// identity — so we bail and let the caller fall back to the path key. Repos with
// multiple root commits (merged orphan histories) are handled by sorting and
// joining so the result is order-independent.
func rootCommit(repo *gitwt.Repo) (string, bool) {
if out, err := repo.Run("rev-parse", "--is-shallow-repository"); err == nil {
if strings.TrimSpace(out) == "true" {
return "", false
}
}
out, err := repo.Run("rev-list", "--max-parents=0", "HEAD")
if err != nil {
return "", false
}
roots := strings.Fields(out)
if len(roots) == 0 {
return "", false
}
sort.Strings(roots)
return strings.Join(roots, ","), true
}

// NormalizeRemoteURL reduces the many equivalent spellings of a git remote to a
// single canonical "host/path" form so the same repository yields one key across
// machines and protocols. All of git@github.com:Acme/app.git,
// https://github.com/acme/app, and ssh://git@github.com:22/acme/app.git collapse
// to github.com/acme/app.
func NormalizeRemoteURL(raw string) string {
s := strings.TrimSpace(raw)

if i := strings.Index(s, "://"); i >= 0 {
// scheme://[user@]host[:port]/path
s = s[i+3:]
if at := strings.Index(s, "@"); at >= 0 {
if slash := strings.Index(s, "/"); slash < 0 || at < slash {
s = s[at+1:]
}
}
} else if colon := strings.Index(s, ":"); colon >= 0 {
// scp-like: [user@]host:path
host := s[:colon]
path := s[colon+1:]
if at := strings.LastIndex(host, "@"); at >= 0 {
host = host[at+1:]
}
s = host + "/" + path
}

// Strip a :port from the host segment (everything before the first slash).
if slash := strings.Index(s, "/"); slash > 0 {
host, rest := s[:slash], s[slash:]
if c := strings.Index(host, ":"); c >= 0 {
host = host[:c]
}
s = host + rest
}

s = strings.TrimSuffix(strings.TrimSuffix(strings.TrimSuffix(s, "/"), ".git"), "/")
return strings.ToLower(s)
}

// IsFingerprintKey reports whether name is a fingerprint key (g-/r- prefixed)
// rather than a legacy path key. A fingerprint key can't be reversed to a local
// path, so a mirror dir with one and no matching local project must wait to
// materialize until that project is opened here.
func IsFingerprintKey(name string) bool {
return strings.HasPrefix(name, remotePrefix) || strings.HasPrefix(name, rootPrefix)
}

func shortHash(s string) string {
sum := sha256.Sum256([]byte(s))
return hex.EncodeToString(sum[:])[:12]
}
Loading