Skip to content

Store a migration manifest (CID-to-piece receipt) on FOC at the end of every flow #74

Description

@SgtPooki

Description

A completed migration leaves no durable, self-contained record of what landed where. The mapping from source CID to piece CID, data set, piece id, and transaction exists only in the local migrate.db, which the PRD's "receipt"/"manifest" claims and the campaign runbook cannot point at.

Generate a migration manifest at the end of every flow (upload and the legacy pdp-submit path) and store it on FOC itself as one more piece in the same run:

  • Content: per source CID — the piece CID it was packed into, raw size, sha256; per piece — each copy's provider id, data set id, piece id, and addPieces tx hash; run metadata — network, payer, source tag, tool version, timestamps; and every held-out or failed CID with its reason.
  • Format: a CAR whose root is the manifest file, uploaded with withIPFSIndexing, so the receipt has its own IPFS CID and resolves from the providers like the migrated data. The manifest cannot list itself; its own CID and piece info go to stdout and migrate.db.
  • Size: dominated by per-CID rows (~150–200 bytes each) — tiny for most runs, ~10 MB raw at 56k CIDs; a single small piece either way. Below a provider's advisory minimum piece size it warns and proceeds.

Impact

Closes the gap between the landing page's "receipt with transaction links" / "manifest mapping every IPFS CID" claims and what the tool produces, and gives report/verification (#73) a portable artifact instead of a local SQLite file.

Next steps

Implement for upload first (the campaign path), then pdp-submit. Decide whether the manifest lands in the existing data sets or a dedicated one per run.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

Status
⌨️ In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions