Remove dependency from erigon-snapshot repo

[wmitsuda]

There is a coupling between erigon and erigon-snapshot which has historical reasons, but IMHO nowadays it is:

• unnecessary
• creates confusion, it is very coupled to erigon (the company) infrastructure layout
• requires changing multiple points every time we need to change something

Recent example (this is the 2nd time that happens to me, and I always need to spend 1 afternoon remembering/AI assisting now how the code works and which code change I forgot):

• performance branch was created in erigon-snapshots
• sync stops working
• it is serving torrents correctly
• manifest looks fine in erigon-snapshot
• actually need to update constant inside erigon code

That is only one variation/possibility, but in general I think we should (in no particular order, only things I remember right now):

• remove the dependency with erigon-snapshot (as go mod).
  • the historical reason was fallback (as I remember): if getting remote hashes fail, then use embedded hashes.
    • but in the past we only had github (now we have R2, and github as a fallback)
    • even if R2/github fails, due to merging, etc., (1) there is a high probability that some files in the embedded hashes don't exist anymore in the torrent network (2) if the user manage to get such torrent, it'll be far from the tip. in both cases UX will be better if the user just wait for connectivity to R2/github in order to get up-to-date-to-tip chain.toml
    • last time I tested the fallback was not working, i.e., R2/github down => erigon halts
  • there is some code inside erigon-snapshot, but AFAIK they are all to interpret the embedded hashes or temporary hacks to connect to R2 (which we did ASAP when github started to block us but we didn't cleanup), hence they can be removed or internalized into erigon.
• internalize the webseeds URLs from erigon-snapshot into erigon (they are constants and there are no fallbacks, AFAIK they don't need to be inside erigon-snapshot).
• turn erigon-snapshot into a release-target-only .toml repo for our automated process
• remove the hardcoded constant with the erigon-snapshots branch name
  • this is a source of confusion, it does a lookup to webseed URL (from embedded erigon-snapshot go mod) + hardcoded branch name. if you create a branch in erigon-snapshot, you need to remember to point erigon binary to the new branch name. in practice we never did publish multiple branches for the same chain and if we forget to do that, then we publish to the correct place, but the erigon "feels" like it is working, but it doesn't, bc it downloads torrent from the wrong branch. not fail fast.
  • we should publish to/read from only 1 place: the webseed determined to the domain hardcoded inside erigon binary and the chain name which is dynamic.
  • if we need some variation, then we can provide some env var manifest override to provide a full URL.
• remove non-valid chain.toml inside erigon-snapshot repo: e.g., the performance branch should contain only bloatnet.toml, everything else can be removed; we should do similar policy to all active branches in order to failfast if any process tries to publish/read by mistake a wrong chain in a wrong branch.

[wmitsuda]

another random thought:

• if you by mistake compile erigon from the performance branch and run it to sync mainnet, it is going to use torrents published in the performance branch, which will be stale. it may look like it is working, but then suddenly it does not complete. then you spend the entire day trying to figure it out, we should fail fast.

[AskAlexSharov]

make sense
also mark has plans to decentralize .toml in future (means Erigon will able to get it from ENR or Bittorrent DHT)

[wmitsuda]

cc: @mh0lt

[mh0lt]

cc: @mh0lt
This is the background issue around this. Once you have merged your changes I will add the webseeds to the published toml.
#20617

As general background I'm working on having erigon nodes independently publish thier chain.toml manifests using this format via diskv5 discovery.

This is a background project I've (well ok I have had claude) working on for a while. Where we have got to:

1. Have an upto 10 node swarm locally starting swapping manifests and launching downloads
2. I have integrated the various validations in the integration lib, applied at a step level as they are published/consumed
3. The nodes all seem to start and get to the tip.

Once this process is finished I'll do at leaste a couple of weeks local swarm testing, befor trying to work out how to run a cross machine test.

I think this is getting close to stable - this will hit main I think in 1-2 months - as a feature flagged option.

It's main net gain is that we'll have constant shorter start-up times as new nodes will always get the latest snapshots, which will then eventually make thier way onto webseeds, but by that stage these become a secondary source.

[wmitsuda]

Surfacing some related merged work that doesn't currently get a mention in the umbrella, for cross-referencing:

Preparatory PRs that broke the embedded-data ↔ runtime-data coupling, paving the way for #21089 and #21197:

• snapcfg, downloader: lazy-parse registry, per-chain remote loading #19641 — snapcfg, downloader: lazy-parse registry, per-chain remote loading (merged 2026-03-04)
• snapcfg: lazy-parse EmbeddedWebseeds, only parse the chain in use #19722 — snapcfg: lazy-parse EmbeddedWebseeds, only parse the chain in use (merged 2026-03-10)
• snapcfg: load preverified hashes for single chain only #19785 — snapcfg: load preverified hashes for single chain only (merged 2026-03-11)

Upstream cleanup in erigontech/erigon-snapshot:

• Cleanup dead code erigon-snapshot#1266 — Cleanup dead code (merged to release/3.4 on 2026-05-08). Removes LoadSnapshots, fetchSnapshotHashes, cloudflareHeaders, insertCloudflareHeaders from embed.go and deletes embed_test.go (−154 lines). These helpers were already mirrored into erigontech/erigon during the 3.4 cycle. Done on release/3.4 first to keep the 3.5 cycle (planned to consume 3.4 snapshots) building.
• Cherry-pick #1266: Cleanup dead code (main) erigon-snapshot#1267 — cherry-pick of TG HasState stage: doesn't gracefully stop on Ctrl+C #1266 to main (open).
• Cherry-pick #1266: Cleanup dead code (performance) erigon-snapshot#1268 — cherry-pick of TG HasState stage: doesn't gracefully stop on Ctrl+C #1266 to performance (open).

Worth flagging:

• db/version: default snapshot git branch to performance #21048 — db/version: default snapshot git branch to performance (merged 2026-05-08). This is actually a counter-example to the "remove the hardcoded constant with the erigon-snapshots branch name" bullet: rather than removing DefaultSnapshotGitBranch, it changed its value on the performance branch — exactly the kind of churn that bullet is intended to eliminate.

— Comment drafted with AI assistance.