Find and remove photos in one iCloud/Photos library that are perceptual duplicates of photos in another — without ever opening both libraries at the same time.
Working name. The repo is
PhotoLibraryDedupe; the app target isPhotoDedupe. Rename freely.
iCloud makes it easy to receive a large batch of someone else's photos — through a Shared Photo Library, Shared Albums, or AirDrop/Messages saves. When that sharing relationship ends (you leave the shared library, or the other person turns sharing off), the photos that were copied into your library stay there. They're now indistinguishable from your own photos, they count against your iCloud storage, and there can be thousands of them.
Deleting them by hand is impractical, and Apple's "Duplicates" feature in Photos only finds duplicates within a single library — it cannot tell you "these 3,800 photos in my library are the same as the photos in that other library, so they're safe to remove."
PhotoDedupe solves exactly that: it compares two separate libraries and removes, from one of them, the photos that already exist in the other.
macOS Photos / PhotoKit can only operate on one Photos library at a time — the one Photos.app currently has open, and only one library can sync to iCloud. So "compare two libraries live" isn't possible.
PhotoDedupe sidesteps this with a two-pass, portable-fingerprint workflow:
┌─────────────────────────┐ carry the file ┌──────────────────────────┐
│ Mac / account A │ (USB stick, AirDrop, │ Mac / account B │
│ the "KEEP" library │ iCloud Drive, etc.) │ the "TARGET" library │
│ │ │ │
│ Pass 1: INDEX │ library.photodedupe ───► │ Pass 2: DEDUPE │
│ • read every photo │ (hashes + thumbnails) │ • read every photo │
│ • compute a perceptual │ │ • match against the │
│ fingerprint │ │ imported index │
│ • write a portable │ │ • move matches to a │
│ index file │ │ "To Delete" album │
│ │ │ • one-click bulk delete │
└─────────────────────────┘ └──────────────────────────┘
- Pass 1 — Index mode runs on the library you want to keep (the source of truth). It computes a compact perceptual fingerprint for every photo and writes a single portable index file:
library.photodedupe. - You carry that file to the other Mac / sign into the other account.
- Pass 2 — Dedupe mode runs on the target library (the one that has the unwanted copies). It fingerprints those photos, matches them against the imported index, moves every confident match into a dedicated "PhotoDedupe — To Delete" album, and gives you a single bulk-delete button that sends the whole album to Recently Deleted.
Only one library is ever open at a time. The index file is the bridge.
PhotoDedupe never deletes anything silently.
- Matches are collected into a normal Photos album named "PhotoDedupe — To Delete". You can open Photos.app and browse it like any album.
- When you're satisfied, you click Delete all in album in PhotoDedupe. This uses PhotoKit to delete every asset in the album in one system confirmation dialog.
- Deleted photos go to Recently Deleted, where macOS keeps them recoverable for 30 days.
Why an in-app bulk-delete button? If you select photos inside an album in Photos.app and press Delete, Photos only removes them from the album — it does not delete them from your library. Deleting from the library requires ⌘-Delete per item or a right-click. PhotoDedupe's button does the real library deletion for the entire album in a single action, which is the "easily bulk-remove from the account" behavior you want.
Each photo is reduced to a small perceptual fingerprint that survives resizing, re-compression, and format changes (HEIC ↔ JPEG):
- dHash (64-bit difference hash) and pHash (64-bit DCT hash) — compact, fast to compare with Hamming distance.
- An aspect-ratio bucket used as a cheap pre-filter.
- For borderline cases, an optional Vision feature print (
VNGenerateImageFeaturePrintRequest→computeDistance) gives a neural-network similarity score.
Strictness is a configurable preset (default Strict):
| Preset | Catches | Trade-off |
|---|---|---|
| Strict (default) | The same image across different resolution, format, or recompression | Minimal false positives — safest for a tool that can delete |
| Moderate | Also edited, cropped, or Live-Photo-vs-still versions of the same shot | A few more matches to review |
| Loose | Also burst frames and near-identical shots | Highest recall, most review |
Thresholds for each preset are defined in CLAUDE.md and are meant to be calibrated against a labeled test set (see the issues).
- macOS 14 (Sonoma) or later. Built and tested against macOS 26 "Tahoe" with the macOS 26 SDK.
- Xcode 26 (Swift 6).
- Full Photos Library access must be granted to the app (it needs to read every asset and to delete).
- Photos should be downloaded locally on each Mac. With "Optimize Mac Storage", some originals live only in iCloud; PhotoDedupe can either skip those or fetch them over the network (see Settings). Fetching can be slow and use bandwidth.
The Xcode project is generated with XcodeGen — project.yml is the source of truth and PhotoDedupe.xcodeproj is generated (and git-ignored). Generate it once after cloning:
git clone <your-repo-url> PhotoLibraryDedupe
cd PhotoLibraryDedupe
brew install xcodegen # one-time, if not already installed
xcodegen generate # creates PhotoDedupe.xcodeproj from project.yml
open PhotoDedupe.xcodeproj # then ⌘R in XcodeOr from the command line:
xcodegen generate
xcodebuild -project PhotoDedupe.xcodeproj -scheme PhotoDedupe -configuration Debug buildOn first launch the app will request Photos access — choose Full Access.
On the Mac with the library you want to keep:
- Launch PhotoDedupe and choose Index this library (Pass 1).
- Let it scan. Progress shows photos fingerprinted / total.
- Save the resulting
library.photodedupefile somewhere you can carry it.
Move the file to the other Mac (or sign into the other iCloud account on the same Mac and let Photos finish syncing).
On the Mac with the library to clean up:
- Launch PhotoDedupe and choose Find duplicates (Pass 2), then select the
library.photodedupefile. - Let it scan and match. Review the matched pairs side by side — each shows the target photo and the keep-library thumbnail it matched, with a confidence score.
- Uncheck anything you want to spare. Click Move matches to "To Delete" album.
- Browse the album in Photos.app if you like, then click Delete all in album in PhotoDedupe.
The portable index contains small thumbnails and metadata of the keep-library photos so you can visually verify matches on the other machine. Treat library.photodedupe as sensitive — it is a low-resolution visual copy of that library. Everything runs on-device; PhotoDedupe makes no network calls except optional iCloud asset fetches performed by the system Photos framework. Delete the index file when you're done.
- Two libraries can't be compared live; you must run two passes and carry the index. By design.
- Photos stored only in iCloud (not downloaded) must be fetched to be fingerprinted, which can be slow.
- Perceptual matching is heuristic. Strict minimizes false positives but no threshold is perfect — that's why review + Recently Deleted recovery exist.
- Very large libraries (100k+) are supported but Pass 1/Pass 2 may take a while; matching uses a BK-tree to stay efficient.
- Screenshots, scans, and heavily edited images may match less reliably.
See ISSUES.md for the full, phased backlog. High level:
- Project scaffold, Photos permissions, library reading.
- Pass 1 — fingerprinting + portable index writer.
- Pass 2 — index importer + matching engine.
- Review UI + album + bulk delete.
- Settings (strictness, iCloud fetch), performance, threshold calibration.
- Packaging, signing, tests.
TBD (suggest MIT for a personal tool).