Implement copy-on-write overlay filesystem and three-way merge

.github/workflows/pr-tests.yml

@@ -7,15 +7,28 @@ on:
       - reopened
       - synchronize
       - ready_for_review
+  push:
+    branches:
+      - core-improvements
 
 concurrency:
-  group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
   cancel-in-progress: true
 
 jobs:
   swift-tests:
-    name: Swift Tests
-    runs-on: macos-latest
+    name: Swift Tests (${{ matrix.os }})
+    runs-on: ${{ matrix.os }}
+    # Linux runs inside the official Swift image; macOS installs a toolchain via setup-swift.
+    container: ${{ matrix.container }}
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - os: macos-latest
+            container: ""
+          - os: ubuntu-24.04
+            container: swift:6.2-noble
     timeout-minutes: 15
 
     permissions:
@@ -28,13 +41,18 @@ jobs:
         uses: actions/checkout@v5
 
       - name: Set up Swift
+        if: runner.os == 'macOS'
         uses: swift-actions/setup-swift@v3
         with:
           swift-version: "6.2"
 
       - name: Show Swift version
         run: swift --version
 
+      - name: Mark workspace safe for git in container
+        if: runner.os == 'Linux'
+        run: git config --global --add safe.directory "$GITHUB_WORKSPACE"
+
       - name: Run test suite
         shell: bash
         run: |
@@ -49,7 +67,7 @@ jobs:
         if: ${{ !cancelled() }}
         uses: dorny/test-reporter@v3
         with:
-          name: Swift Test Report
+          name: Swift Test Report (${{ matrix.os }})
           path: .build/test-results/xunit.xml
           reporter: swift-xunit
           fail-on-error: false
@@ -59,7 +77,8 @@ jobs:
           list-tests: failed
 
       - name: Publish coverage summary
-        if: always()
+        # The Swift Linux container does not ship python3; the macOS job covers the summary.
+        if: always() && runner.os == 'macOS'
         shell: bash
         run: |
           python3 <<'PY'
@@ -93,7 +112,7 @@ jobs:
         if: always()
         uses: actions/upload-artifact@v6
         with:
-          name: pr-test-results-${{ github.event.pull_request.number }}
+          name: pr-test-results-${{ matrix.os }}-${{ github.event.pull_request.number || github.run_id }}
           if-no-files-found: warn
           include-hidden-files: true
           path: |

README.md

@@ -33,7 +33,7 @@ Many agent and tooling flows need more than plain disk I/O:
 - `FileSystem`: low-level protocol for custom filesystem backends
 - `ReadWriteFilesystem`: real disk access rooted to a configured directory
 - `InMemoryFilesystem`: fully in-memory filesystem for isolated workspaces and tests
-- `OverlayFilesystem`: snapshot a disk root and keep writes in memory
+- `OverlayFilesystem`: lazy copy-on-write view of a disk root — reads pass through, writes stay in memory
 - `MountableFilesystem`: compose multiple filesystems under one virtual tree
 - `PermissionedFileSystem`: wrap any filesystem with operation-level approvals
 - `SandboxFilesystem`: convenience wrapper for app sandbox roots
@@ -96,7 +96,7 @@ let workspace = Workspace(
 )
 ```
 
-`Storage.directory(at:)` writes checkpoint and snapshot JSON plus a `mutations.jsonl` append log (one JSON record per line) under `<url>/<workspaceId>/`. A legacy `mutations.json` array is migrated to JSONL on first access. The store assigns monotonic `sequence` numbers while holding `mutations.lock` (advisory `flock` where the OS supports it), so multiple `Workspace` instances that share a `workspaceId` and store do not collide on mutation sequence. Appends read only the log's final record to derive the next sequence, and a partial trailing line left by a crashed append is skipped on read and truncated before the next append. The current checkpoint head is derived from the parent-id graph (unparented tips), not only from `createdAt` ordering, which reduces surprises when wall clocks differ between processes. Listing mutations still reads the full log; very long histories may need application-level rotation. Coordinating multiple hosts or network disks that do not honor `flock` may still require extra synchronization.
+`Storage.directory(at:)` writes checkpoint metadata, snapshot manifests, and a `mutations.jsonl` append log (one JSON record per line) under `<url>/<workspaceId>/`. Snapshots are content-addressed: file bytes are stored once per unique content in `blobs/<sha256>`, and each snapshot manifest references them by hash — so consecutive checkpoints share unchanged file content instead of re-serializing the whole tree, and contents are stored raw rather than base64-inflated. Legacy inline-snapshot JSON files and a legacy `mutations.json` array are still read (the latter is migrated to JSONL on first access). Blobs are retained until pruned by future GC tooling. The store assigns monotonic `sequence` numbers while holding `mutations.lock` (advisory `flock` where the OS supports it), so multiple `Workspace` instances that share a `workspaceId` and store do not collide on mutation sequence. Appends read only the log's final record to derive the next sequence, and a partial trailing line left by a crashed append is skipped on read and truncated before the next append. The current checkpoint head is derived from the parent-id graph (unparented tips), not only from `createdAt` ordering, which reduces surprises when wall clocks differ between processes. Listing mutations still reads the full log; very long histories may need application-level rotation. Coordinating multiple hosts or network disks that do not honor `flock` may still require extra synchronization.
 
 ### Reads
 
@@ -111,6 +111,16 @@ let tree = try await workspace.walkTree("/")
 let summary = try await workspace.summarizeTree("/")
 ```
 
+Glob wildcards use shell semantics: `*` and `?` match within a single path component, `**` matches recursively across components, and character classes support negation (`[!abc]`).
+
+Ranged reads avoid loading whole files when the backing filesystem supports seeking:
+
+```swift
+let slice = try await workspace.readData(from: "/blob.bin", offset: 1024, length: 4096)
+```
+
+At the `FileSystem` level, implementations also provide `readFileChunks(path:chunkSize:)` for streaming reads, `createFile(path:data:)` for exclusive creation (fails with `EEXIST`), and `capabilities()` to query optional features (symlinks, hard links, permissions, real-path resolution) without probe-and-catch.
+
 JSON helpers encode and decode through `Codable`:
 
 ```swift
@@ -126,7 +136,20 @@ print(config.enabled) // true
 
 ### Tracked Writes
 
-Every public write records an internal mutation:
+Every public write records a mutation. The history is public — `mutationRecords()` returns the ordered records including per-file effects and text diffs — and the amount of detail is configurable per workspace:
+
+```swift
+let workspace = Workspace(
+    filesystem: InMemoryFilesystem(),
+    tracking: .fullDiffs(maxDiffBytes: 1_000_000) // cap diff computation to 1 MB files
+)
+// .full        — full diffs, no size cap (default)
+// .pathsOnly   — touched paths and effects, no text diffs
+// .disabled    — no history at all; change events still fire, but merge cannot
+//                detect uncheckpointed edits made in this mode
+```
+
+Tracked writes:
 
 ```swift
 try await workspace.writeFile("/notes/todo.txt", content: "one")
@@ -206,7 +229,7 @@ print(all.count)
 print(snapshot.rootPath)
 ```
 
-Public `restoreSnapshot(_:)` is also tracked as a workspace mutation. Checkpoint rollback uses an internal untracked restore so the rollback is represented by the rollback checkpoint, not by a second restore mutation.
+Public `restoreSnapshot(_:)` is also tracked as a workspace mutation. Checkpoint rollback records the tree changes it performs as a `rollback` mutation in addition to the rollback checkpoint, so the mutation log remains a complete account of filesystem changes. Long-running workspaces can bound the log with `pruneMutationHistory(throughSequence:)`, which always retains the newest record so sequence numbers stay monotonic.
 
 ### Branch And Merge
 
@@ -229,6 +252,20 @@ print(merged.mergedFromCheckpointId == branchHead.id)       // true
 
 `merge(_:)` is optimistic. If the parent workspace head changed after `branch()` was created, merge throws `WorkspaceError.mergeConflict(parentWorkspaceId:expectedBase:actualHead:)`. If the parent made tracked writes after `branch()` without creating a checkpoint, merge throws `WorkspaceError.mergeUncheckpointedChanges(parentWorkspaceId:baseMutationCursor:currentMutationCursor:)` instead of silently overwriting those edits; create a checkpoint or roll the parent back first.
 
+`mergeThreeWay(_:label:)` merges even when both sides advanced. Each path is resolved against the branch's base checkpoint: the side that changed wins, identical changes merge cleanly, and when both sides changed the same path differently the merge applies nothing and returns structured conflicts instead of throwing:
+
+```swift
+let result = try await workspace.mergeThreeWay(branch, label: "merge draft")
+if result.applied {
+    print("merged as checkpoint \(result.checkpoint!.id)")
+} else {
+    for conflict in result.conflicts {
+        print("conflict at \(conflict.path): \(conflict.kind)")
+        // conflict.oursDiff / conflict.theirsDiff carry base→side diffs for text files
+    }
+}
+```
+
 ## Common Filesystem Patterns
 
 ### Rooted Disk Workspace
@@ -253,6 +290,8 @@ let preview = try await workspace.summarizeTree("/Sources", maxDepth: 2)
 try await workspace.writeFile("/SCRATCH.md", content: "overlay-only change\n")
 ```
 
+The overlay is lazy: nothing is copied at configuration time, reads pass through to the source directory (so files that change on disk are visible immediately), and only mutated entries are copied up into memory. Deletions are recorded as whiteouts that hide the source entry without touching the disk. `reload()` discards all overlay writes and whiteouts.
+
 ### Mounted Workspaces
 
 ```swift
@@ -308,6 +347,7 @@ let workspace = Workspace(filesystem: filesystem)
 ## Security Notes
 
 - Jail and root enforcement belong to the underlying filesystem implementation.
+- `ReadWriteFilesystem` uses `lstat` semantics for entry-level operations (`stat`, `exists`, `remove`, `move` source, `readSymlink`): a symlink is handled as the link itself, never its target, so links pointing outside the root can be inspected and deleted but not read or written through.
 - Permission checks are additive. They do not replace path normalization or jail enforcement.
 - If you expose `Workspace` to model-driven or remote callers, the host still needs to define what roots, mounts, and permissions are acceptable.