Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ filesystem, FTP, package, or database behavior directly.
- `knowledge_gui.py` renders knowledge search, imports, sources, and conflicts.
- `backup_gui.py` renders inventory, package, FTP, and converter workflows.
- `profile_gui.py` renders privacy-aware profile/save inventory and snapshots.
- `package_gui.py` renders STFS, SVOD, GDF/XISO, FATX-image, and GPD workflows.
- `collection_gui.py` renders collection analysis, matching, reports, and
repair previews.
- `community_gui.py` renders unified search and the cross-domain community
Expand Down Expand Up @@ -82,7 +83,9 @@ results to Tk's main loop.

- `main.py` contains the XboxUnity collector and shared configuration.
- `resume.py` handles partial download state and verification.
- `backup_manager.py` parses public STFS/XBE/XEX fields and performs safe
- `unityscraper/domains/packages` owns bounded package/image parsing,
verification, extraction, and transactional mutation.
- `backup_manager.py` retains compatibility exports and performs safe
filesystem or FTP operations.
- `knowledge_base.py` defines normalized knowledge records and resolution.
- `consolemods_adapters.py`, `wiki_adapters.py`, and `dat_adapters.py` parse
Expand Down
47 changes: 47 additions & 0 deletions PACKAGE_LAB.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Package Lab

Package Lab is UnityScraper's built-in Xbox package and image workspace. The
desktop page is available from the sidebar and the Tools menu.

## Supported Workflows

- **STFS (`CON`, `LIVE`, `PIRS`)**: inspect metadata, follow consecutive or
fragmented file chains, inventory and extract files, verify data-block
SHA-1 records, replace a file within its existing allocation, edit bounded
public text metadata, and rebuild the hash tree.
- **Games on Demand / SVOD**: inspect a package header and adjacent `.data`
directory, verify Data#### block hashes, and reconstruct the payload.
- **GDF/XISO**: inventory the directory tree and safely extract files from
supported Xbox disc images.
- **FATX images**: discover known partitions, follow FATX16/FATX32 chains,
inventory and extract files, and replace a file into a separate image when
it fits the existing allocation.
- **GPD/XDBF**: inspect achievements, settings, title history, and images;
update existing achievement or setting records into a separate output.

## Write Safety

Writes use a temporary file and atomic publication. FATX and GPD edits require
a separate output path. STFS replacement does not allocate new blocks: a
replacement must fit the file's existing chain. FATX replacement follows the
same rule. Source files remain unchanged when an operation fails.

STFS rehashing is available without signing. A caller can provide a signer
through the packages-domain callback interface; UnityScraper does not ship or
store private signing keys. The callback must return the signature bytes in
the package-specific representation expected by the caller's lawful signing
material.

## Deliberate Exclusions

UnityScraper does not bundle X360's embedded key resources, Le Fluffie's
updater or artwork, account credential modification, or DLC license bypasses.
It also does not write directly to a physical device. FATX work targets image
files, and destructive changes require an explicit output image.

## Attribution

Format geometry and field layouts were informed by Dalavin's GPLv3 X360
library and Le Fluffie lineage. UnityScraper uses a new Python implementation
with explicit bounds, transactional writes, and cross-platform paths. See
[Third-Party Notices](THIRD_PARTY_NOTICES.md).
15 changes: 7 additions & 8 deletions PROFILES_AND_SAVES.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,19 +81,18 @@ device.

## Current Safety Boundary

This release deliberately does not:
Profile inventory and restore deliberately do not:

- edit achievements, GPD records, gamertags, or account blocks
- edit gamertags or account credential blocks
- change profile, console, or device ownership fields
- rehash or resign modified CON packages
- automatically resign modified CON packages
- authenticate to Xbox Live or Microsoft accounts
- store CPU keys, account credentials, or signing material
- write raw FATX disks
- write physical FATX disks

Those operations can make a profile or save unusable when implemented
incorrectly. Future editing and migration support should only ship with
complete package verification, automatic pre-change snapshots, and
well-tested cross-platform signing support.
Package Lab separately provides transactional edits for existing GPD records
and STFS files. It writes to a chosen output, rebuilds STFS hashes, and accepts
only caller-supplied signing callbacks. See [Package Lab](PACKAGE_LAB.md).

## Profile Intelligence and Xenia

Expand Down
12 changes: 7 additions & 5 deletions THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,16 @@ different lawfully obtained XeXTool build or another command-line utility.
- **Archived source:** [mtolly/X360](https://github.com/mtolly/X360)
- **License:** GNU General Public License version 3

UnityScraper's profile and save implementation is new Python code informed by
the public package/profile model and field layout documented in the X360
library and Le Fluffie source. The original GPL text is preserved at
UnityScraper's STFS, SVOD, GDF/XISO, FATX-image, GPD, profile, and save
implementation is new Python code informed by the public format geometry and
field layouts documented in the X360 library and Le Fluffie source. The
original GPL text is preserved at
`assets/references/lefluffie/X360-GPL-3.0.txt`.

UnityScraper does not include Le Fluffie's executable, updater, embedded key
resources, account-modification code, or artwork. The application credits
Dalavin prominently and links to the archived corresponding source.
resources, account credential modification, license-bypass behavior, or
artwork. The application credits Dalavin prominently and links to the archived
corresponding source.

## Tool Center Interoperability

Expand Down
34 changes: 22 additions & 12 deletions backup_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,20 @@
from pathlib import Path, PurePosixPath
from typing import Any, Callable, Iterable, Iterator, Optional

from unityscraper.domains.packages.errors import (
InvalidPackageError,
PackageError as BackupError,
UnsafeArchiveError,
)
from unityscraper.domains.packages.executables import (
inspect_xbe as _domain_inspect_xbe,
inspect_xex as _domain_inspect_xex,
)
from unityscraper.domains.packages.stfs import (
extract_stfs_files as _domain_extract_stfs_files,
inspect_stfs as _domain_inspect_stfs,
list_stfs_entries as _domain_list_stfs_entries,
)

STFS_MAGICS = {b"CON ", b"LIVE", b"PIRS"}
CONTENT_TYPES = {
Expand All @@ -38,18 +52,6 @@
FATX_INVALID_RE = re.compile(r'[<>:"/\\|?*]')


class BackupError(RuntimeError):
"""Base error for backup operations."""


class InvalidPackageError(BackupError):
"""Raised when a file is not a supported Xbox package."""


class UnsafeArchiveError(BackupError):
"""Raised when an archive attempts to escape its extraction directory."""


class ConflictError(BackupError):
"""Raised when a destination conflict cannot be resolved automatically."""

Expand Down Expand Up @@ -496,6 +498,14 @@ def format_version(value: int) -> str:
)


# Compatibility exports. Package-format ownership lives in the packages domain.
inspect_stfs = _domain_inspect_stfs
list_stfs_entries = _domain_list_stfs_entries
extract_stfs_files = _domain_extract_stfs_files
inspect_xbe = _domain_inspect_xbe
inspect_xex = _domain_inspect_xex


def sha256_file(path: str | Path) -> str:
digest = hashlib.sha256()
with Path(path).open("rb") as handle:
Expand Down
11 changes: 11 additions & 0 deletions modern_gui.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@
from unityscraper.domains.library import GameSummary, LibraryService
from platform_support import open_path
from profile_gui import ProfileSavePage
from package_gui import PackageLabPage
from profile_manager import ProfileSaveManager
from setup_wizard import run_first_run_wizard
from title_catalog import TitleSuggestion, XboxUnityTitleCatalog
Expand Down Expand Up @@ -416,6 +417,7 @@ def _build_menubar(self) -> None:
)),
("Tools", (
("Backup Manager", self.show_backups),
("Package Lab", self.show_package_lab),
("Tool Center", self.show_external_tools),
("Archive Health", self.show_health),
("Settings", self.show_settings),
Expand Down Expand Up @@ -482,6 +484,7 @@ def _build_shell(self) -> None:
(t("nav_downloads"), self.show_downloads),
(t("nav_backups"), self.show_backups),
(t("nav_profiles"), self.show_profiles),
("Package Lab", self.show_package_lab),
(t("nav_tools"), self.show_external_tools),
(t("nav_collections"), self.show_collections),
(t("nav_knowledge"), self.show_knowledge),
Expand Down Expand Up @@ -562,6 +565,14 @@ def show_external_tools(self) -> None:
CONFIG_PATH,
)

def show_package_lab(self) -> None:
self._clear_content()
self.package_lab_page = PackageLabPage(
self.root,
self.content,
self._page_header,
)

def show_profiles(self) -> None:
self._clear_content()
self.profile_save_page = ProfileSavePage(
Expand Down
Loading