feat: complete profile intelligence and knowledge controls

ARCHITECTURE.md

@@ -63,10 +63,12 @@ Main schema groups:
 
 - Library: `titleids`, `title_updates`, `covers`, `download_history`
 - Knowledge: sources, documents, revisions, entities, names, identifiers,
-  facts, citations, relationships, import runs, and conflicts
+  facts, citations, relationships, import runs, conflicts, source priorities,
+  conflict decisions, and scheduled sync state
 - Backups: targets, scans, inventory, and operations
-- Profiles: scan runs, profiles, saves, snapshots, snapshot files, and
-  auditable operations
+- Profiles: scan runs, profiles, saves, snapshots, snapshot files, GPD
+  inventories, achievements, comparisons, Xenia migration runs, and auditable
+  operations
 
 Schema initialization is idempotent. New migrations should preserve existing
 data and be covered by tests.
@@ -105,6 +107,22 @@ user-selected file or ZIP
 Game payloads are not stored in SQLite. Inventory records contain paths,
 identifiers, sizes, statuses, and notes.
 
+### Profile Intelligence
+
+```text
+standalone/extracted XDBF file
+  -> bounded table and offset validation
+  -> read-only achievement/setting parsing
+  -> local inventory and profile comparison
+
+indexed saves + Xenia content root
+  -> non-mutating migration preview
+  -> verified automatic snapshot
+  -> .partial copy and SHA-256 verification
+  -> skip identical / retain conflicts
+  -> migration audit record
+```
+
 ### Profile Snapshot
 
 ```text

CHANGELOG.md

@@ -7,6 +7,18 @@ Notable changes to UnityScraper are documented here. The project follows
 
 ### Added
 
+- Read-only XDBF/GPD inspection with bounded entry parsing, achievement state,
+  gamerscore summaries, extracted-folder discovery, and local inventory.
+- Profile comparison for save hashes and imported achievement state.
+- Snapshot-first Xenia migration previews with atomic verified copies and
+  non-overwriting conflict handling.
+- Additive schema migration 7 for GPDs, achievements, comparisons, Xenia
+  migration runs, source priorities, conflict decisions, sync schedules, and
+  opt-in remote hash verification.
+- Field-specific knowledge source priorities, auditable conflict resolution,
+  and opt-in app-start knowledge refresh scheduling.
+- Feature-detected remote SHA-256 verification for compatible console FTP
+  dashboards.
 - Profiles & Saves workspace with read-only Content-tree discovery, masked
   profile inventory, STFS ownership metadata, save search, duplicate and
   mismatch reporting, verified snapshots, manifests, and conflict-safe restore.

CONSOLE_SYNC.md

@@ -11,6 +11,8 @@ whose FTP server the user explicitly configures.
 - FTP `REST` is used when the server supports ranged transfer.
 - Uploads are published by renaming the completed partial file.
 - Final sizes are verified; downloads can also require a SHA-256.
+- Uploads can optionally require a remote SHA-256 when the dashboard exposes a
+  compatible read-only hash command.
 - Each job can have a bytes-per-second bandwidth limit.
 - Passwords remain in memory and are never stored.
 
@@ -22,6 +24,11 @@ files. A snapshot can be compared with a local directory to find files only
 on the PC, only on the console, different-sized files, and matching files.
 Discovery has a default 100,000-entry safety limit.
 
+Remote hash verification probes `XSHA256`, standardized `HASH`, and compatible
+`SITE SHA256` commands. When the option is enabled, a server without one of
+those commands fails verification instead of silently falling back to size
+only.
+
 ```powershell
 python main.py --ftp-host 192.168.1.50 --ftp-user xbox --ftp-snapshot /Hdd1
 

DOCS_INDEX.md

@@ -7,6 +7,8 @@
   verification, and external conversion
 - [Profiles and Saves](PROFILES_AND_SAVES.md) - profile inventory, save
   snapshots, privacy, restore behavior, and Le Fluffie attribution
+- [Profile Intelligence and Xenia](PROFILE_INTELLIGENCE.md) - read-only GPD
+  achievements, profile comparison, and snapshot-first Xenia migration
 - [Collection Intelligence](COLLECTION_INTELLIGENCE.md) - storage discovery,
   XEX identity, Title Update compatibility, preservation, and repair previews
 - [Console Sync](CONSOLE_SYNC.md) - persistent transfers, resume, snapshots,

KNOWLEDGE_SOURCES.md

@@ -111,7 +111,11 @@ The **Knowledge** page includes:
 - source license, document count, fact count, and latest import status;
 - ConsoleMods ID sync and whole-wiki sync;
 - Redump and No-Intro file import;
-- conflicting-claim review.
+- per-property source priorities, where lower numbers are preferred for
+  display;
+- conflicting-claim review with recorded prefer-existing, prefer-incoming,
+  and dismiss decisions;
+- an opt-in app-start refresh schedule with a minimum six-hour interval.
 
 ## Remaining Boundaries
 

PROFILES_AND_SAVES.md

@@ -95,6 +95,17 @@ incorrectly. Future editing and migration support should only ship with
 complete package verification, automatic pre-change snapshots, and
 well-tested cross-platform signing support.
 
+## Profile Intelligence and Xenia
+
+The workspace also imports standalone or already-extracted XDBF/GPD files for
+read-only achievement views, compares two indexed profiles, and previews Xenia
+save mappings. Xenia migration always creates a verified save snapshot first,
+copies through `.partial` staging, verifies SHA-256, skips identical files, and
+never overwrites a different destination.
+
+See [PROFILE_INTELLIGENCE.md](PROFILE_INTELLIGENCE.md) for the parser bounds,
+comparison fields, Xenia paths, and migration safety model.
+
 ## Le Fluffie Attribution
 
 The profile/STFS field model is informed by Dalavin, also known as

PROFILE_INTELLIGENCE.md

@@ -0,0 +1,80 @@
+# Profile Intelligence and Xenia
+
+The **Profiles & Saves** workspace includes read-only profile intelligence and
+a snapshot-first bridge for Xenia save folders.
+
+## GPD and Achievements
+
+UnityScraper reads standalone or already-extracted Xbox 360 XDBF/GPD files.
+Choose **Import GPD** for one file or **Scan Extracted Folder** to find files
+whose first four bytes are the `XDBF` signature.
+
+The bounded parser validates:
+
+- XDBF magic, version, table capacity, and active counts;
+- every entry offset and size before reading it;
+- achievement record minimum sizes;
+- variable-length setting sizes;
+- a 512 MiB per-file safety limit.
+
+For game GPDs, the application displays achievement ID, title, gamerscore,
+locked/unlocked state, and a valid online unlock timestamp when present. It
+also records totals for unlocked achievements and earned/possible gamerscore.
+
+The parser never writes to the source file. It does not unlock achievements,
+alter sync records, extract images, edit account settings, or repair malformed
+databases.
+
+UnityScraper currently reads standalone or extracted GPD files. It does not
+silently unpack or rewrite the profile's STFS container.
+
+## Profile Comparison
+
+Choose two indexed profiles on the **Compare** tab. The report identifies:
+
+- save TitleIDs present on only one profile;
+- TitleIDs whose indexed save hashes differ;
+- TitleIDs with identical indexed save hashes;
+- imported achievements unlocked by only one profile;
+- achievements unlocked by both profiles.
+
+Comparison history is stored locally. Profile identifiers remain masked in the
+normal inventory interface and no profile information is uploaded.
+
+## Xenia Migration
+
+Xenia normally keeps saves in a `content` directory. Common locations are
+suggested on Windows and Linux, and any Xenia folder or content root can be
+selected manually.
+
+The migration workflow is:
+
+1. Select an indexed source profile.
+2. Choose the Xenia folder and target profile ID.
+3. Preview every destination and conflict.
+4. Create an automatic verified snapshot.
+5. Copy only new files through `.partial` staging.
+6. Verify each copied file with SHA-256.
+
+Identical destination files are skipped. Different files and non-file
+destinations are conflicts and are never overwritten. Migration runs, counts,
+plans, and their pre-change snapshot IDs are recorded in SQLite.
+
+Xenia's folder guidance is based on the official
+[Xenia Canary FAQ](https://github.com/xenia-canary/xenia-canary/wiki/FAQ) and
+[Quickstart](https://github.com/xenia-canary/xenia-canary/wiki/Quickstart).
+UnityScraper does not bundle or modify the emulator.
+
+## Deliberate Boundary
+
+This release still does not:
+
+- edit GPD achievements, settings, gamertags, or account blocks;
+- rewrite ownership identifiers;
+- rehash or resign a modified CON package;
+- store signing material, CPU keys, passwords, or Xbox Live credentials;
+- write raw FATX devices.
+
+Those operations require complete package extraction, mutation, rehashing,
+signature verification, and recovery testing across real profiles before they
+can be offered responsibly.

PROJECT_STATUS.md

@@ -42,6 +42,12 @@ backup-management, and source-attributed knowledge application.
   reporting, verified snapshots, manifests, and conflict-safe restore.
 - Credited GPLv3 technical lineage from Dalavin / DJ SkunkieButt's X360
   library and Le Fluffie without bundling its updater, keys, or executable.
+- Bounded read-only XDBF/GPD achievement inspection, profile comparison, and
+  snapshot-first Xenia migration with non-overwriting verified copies.
+- Per-property knowledge source priorities, recorded conflict decisions, and
+  opt-in scheduled app-start refreshes.
+- Feature-detected, opt-in remote SHA-256 verification for console FTP servers
+  that expose a compatible read-only command.
 - Local-by-default REST API with token-required remote binding, restricted
   browser origins, validated settings, and current version reporting.
 - Cross-platform CI, Windows packaging checks, tagged release archives,
@@ -83,9 +89,9 @@ backup-management, and source-attributed knowledge application.
 
 ## Future Work
 
-- Add field-specific source-priority controls and conflict resolution actions.
-- Add optional scheduled knowledge refreshes.
-- Validate console resume behavior against a broader matrix of dashboard FTP
-  servers and add opt-in remote hash verification where servers expose it.
-- Add read-only GPD/achievement views, Xenia save mapping, and audited
-  profile-migration previews before considering package mutation.
+- Validate console resume and optional hash behavior against a broader matrix
+  of real dashboard FTP servers.
+- Expand read-only GPD coverage with dashboard title-history and safe image
+  previews after adding a decompression and image validation boundary.
+- Consider package mutation only after complete STFS extraction, rehashing,
+  signing, verification, and automatic recovery have independent test vectors.

README.md

@@ -38,6 +38,9 @@ service exposes. UnityScraper does not silently substitute HTTPS URLs.
 - Imports user-supplied Redump and No-Intro XML DAT files.
 - Stores entities, identifiers, facts, citations, revisions, import runs, and
   conflicts with provenance.
+- Supports per-property source priorities and records explicit conflict
+  decisions without deleting competing claims.
+- Offers an opt-in app-start refresh schedule with a minimum six-hour interval.
 - Fills blank or unknown local metadata without replacing better known values.
 
 ### Backup Management
@@ -53,6 +56,8 @@ service exposes. UnityScraper does not silently substitute HTTPS URLs.
   manifest.
 - Queues resumable uploads and downloads to a configured Aurora-style FTP server.
 - Captures read-only console inventories and compares PC and console content.
+- Can opt into remote SHA-256 verification when the selected FTP dashboard
+  advertises a compatible read-only hash command.
 - Runs a user-selected external ISO converter without bundling converter code.
 
 ### Profiles and Save Data
@@ -67,10 +72,14 @@ service exposes. UnityScraper does not silently substitute HTTPS URLs.
   verified atomic copies.
 - Restores snapshots without overwriting different existing files.
 - Exports portable JSON preservation manifests.
+- Reads extracted XDBF/GPD achievement databases without modifying them.
+- Compares save hashes and imported achievement state across two profiles.
+- Previews Xenia save mappings and creates a verified snapshot before migration.
 
 The profile/package model is informed by Dalavin, also known as DJ
 SkunkieButt, and the GPLv3 X360 library and Le Fluffie source. See
-[PROFILES_AND_SAVES.md](PROFILES_AND_SAVES.md) and
+[PROFILES_AND_SAVES.md](PROFILES_AND_SAVES.md),
+[PROFILE_INTELLIGENCE.md](PROFILE_INTELLIGENCE.md), and
 [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).
 
 ### External Tools
@@ -167,7 +176,7 @@ Linux source setup:
 | Add Games | Search cached game names, select TitleIDs, or import lists |
 | Downloads | Review and manage download activity |
 | Backup Manager | Scan, install, verify, export, convert, and transfer owned content |
-| Profiles & Saves | Inventory profiles, back up saves, and restore snapshots |
+| Profiles & Saves | Inventory profiles, inspect achievements, compare, snapshot, restore, and migrate to Xenia |
 | External Tools | Run XeXTool and other user-supplied command-line utilities |
 | Collections | Identify storage, compare Title Updates, verify preservation data, and preview repairs |
 | Knowledge | Search sources, facts, citations, imports, and conflicts |

SECURITY.md

@@ -37,6 +37,12 @@ filesystem contents in a report.
 - Profile and save identifiers are masked in the GUI by default.
 - Profile scans are read-only. Snapshot restores preserve different existing
   files and write the restored copy alongside them.
+- GPD parsing validates bounded entry tables and file offsets and never writes
+  achievement, setting, sync, or image records.
+- Xenia migrations require a preview and verified snapshot, publish through
+  partial files, and never overwrite different destination data.
+- Optional remote hashes use read-only FTP commands and fail closed when the
+  selected dashboard does not expose a supported SHA-256 response.
 - Profiles, saves, gamertags, XUIDs, console IDs, and device IDs are not sent
   to metadata sources.
 - External converters run only when explicitly configured by the user.

UnityScraper.spec

@@ -27,12 +27,16 @@ a = Analysis(
         'dat_adapters',
         'knowledge_gui',
         'knowledge_service',
+        'knowledge_scheduler',
         'knowledge_sync',
         'plugins',
+        'gpd_parser',
         'profile_gui',
+        'profile_intelligence',
         'profile_manager',
         'updater',
         'wiki_adapters',
+        'xenia_bridge',
     ],
     hookspath=[],
     hooksconfig={},

backup_gui.py

@@ -184,6 +184,12 @@ def _build_transfer(self) -> None:
         ttk.Spinbox(
             limit_row, from_=0, to=102400, textvariable=self.ftp_limit_var, width=10
         ).pack(side=tk.LEFT, padx=(8, 0))
+        self.ftp_remote_hash_var = tk.BooleanVar(value=False)
+        ttk.Checkbutton(
+            limit_row,
+            text="Require remote SHA-256 when supported",
+            variable=self.ftp_remote_hash_var,
+        ).pack(side=tk.LEFT, padx=(18, 0))
         self.queue_var = tk.StringVar(value="Persistent queue: empty")
         ttk.Label(tab, textvariable=self.queue_var, style="Subheader.TLabel").grid(
             row=9, column=0, columnspan=2, sticky=tk.W, pady=(8, 0)
@@ -458,6 +464,7 @@ def queue_upload(self) -> None:
                 source,
                 remote,
                 bandwidth_limit=max(0, int(self.ftp_limit_var.get() or "0")) * 1024,
+                verify_remote_hash=self.ftp_remote_hash_var.get(),
             )
         except Exception as exc:
             self._failed(exc)

console_sync.py

@@ -6,6 +6,7 @@
 import hashlib
 import json
 import posixpath
+import re
 import sqlite3
 import threading
 import time
@@ -88,6 +89,7 @@ def enqueue(
         priority: int = 100,
         bandwidth_limit: int = 0,
         expected_sha256: str = "",
+        verify_remote_hash: bool = False,
     ) -> int:
         if direction not in {"upload", "download"}:
             raise ValueError("direction must be upload or download")
@@ -100,8 +102,8 @@ def enqueue(
                 INSERT INTO console_transfer_jobs
                     (target_id, direction, local_path, remote_path, total_bytes,
                      status, priority, bandwidth_limit, expected_sha256,
-                     created_at, updated_at)
-                VALUES (?, ?, ?, ?, ?, 'queued', ?, ?, ?, ?, ?)
+                    verify_remote_hash, created_at, updated_at)
+                VALUES (?, ?, ?, ?, ?, 'queued', ?, ?, ?, ?, ?, ?)
                 """,
                 (
                     target_id,
@@ -112,6 +114,7 @@ def enqueue(
                     priority,
                     max(0, bandwidth_limit),
                     expected_sha256.lower(),
+                    int(verify_remote_hash),
                     now,
                     now,
                 ),
@@ -314,7 +317,13 @@ def _checkpoint(self, job_id, current, total, started, job, progress) -> None:
     def _verify(self, job: dict, target: FtpTarget, total: int) -> None:
         if job["direction"] == "upload":
             with _ftp(target) as ftp:
-                actual = _remote_size(ftp, _remote_path(job["remote_path"]))
+                remote_path = _remote_path(job["remote_path"])
+                actual = _remote_size(ftp, remote_path)
+                if bool(job.get("verify_remote_hash")):
+                    remote_digest = _remote_sha256(ftp, remote_path)
+                    local_digest = _sha256(Path(job["local_path"]))
+                    if remote_digest != local_digest:
+                        raise IOError("Remote SHA-256 verification failed")
             if actual != total:
                 raise IOError(f"Remote verification failed: {actual} != {total}")
         else:
@@ -499,3 +508,24 @@ def _sha256(path: Path) -> str:
         for chunk in iter(lambda: handle.read(1024 * 1024), b""):
             digest.update(chunk)
     return digest.hexdigest()
+
+
+def _remote_sha256(ftp: ftplib.FTP, path: str) -> str:
+    """Use read-only FTP hash extensions when a dashboard exposes one."""
+    attempts = (
+        f"XSHA256 {path}",
+        "OPTS HASH SHA-256",
+        f"HASH {path}",
+        f"SITE SHA256 {path}",
+    )
+    for command in attempts:
+        try:
+            response = ftp.sendcmd(command)
+        except ftplib.all_errors:
+            continue
+        match = re.search(r"(?i)\b[0-9a-f]{64}\b", response)
+        if match:
+            return match.group(0).lower()
+    raise IOError(
+        "The console FTP server does not expose a supported SHA-256 command"
+    )