Arc Testnet v0.6.0 fails to bootstrap from a fresh public snapshot: Receipts static-file / DB inconsistency during unwind

[Bela1401]

Environment:

• OS: Ubuntu 24.04
• Network: Arc Testnet
• arc-node version: v0.6.0
• install method: built from source from the public repository
• execution binary commit reported at startup: 567e6c5

Summary:
A clean Arc Testnet bootstrap from the current public snapshot fails deterministically in the execution layer during startup consistency checks. The failure occurs before the node becomes usable and appears to be caused by an inconsistency between the Receipts static files and the database state.

This is reproducible after multiple full resets and fresh snapshot downloads, so it does not appear to be caused by stale local state.

Steps to reproduce:

1. Build and install arc-node v0.6.0 from source
2. Remove all previous node state:
   • /root/.arc
   • /root/.cache/reth
   • /run/arc
3. Recreate clean directories
4. Download a fresh snapshot using the official bootstrap flow:
   arc-snapshots download
   --chain=arc-testnet
   --execution-path /root/.arc/execution
   --consensus-path /root/.arc/consensus
5. Initialize consensus home:
   arc-node-consensus init --home /root/.arc/consensus
6. Start the execution layer with the documented command:
   arc-node-execution node
   --chain arc-testnet
   --datadir /root/.arc/execution
   --ipcpath /run/arc/reth.ipc
   --auth-ipc --auth-ipc.path /run/arc/auth.ipc
   --http --http.addr 127.0.0.1 --http.port 8545
   --http.api eth,net,web3,txpool,trace,debug
   --rpc.forwarder https://rpc.quicknode.testnet.arc.network/
   --metrics 127.0.0.1:9001
   --disable-discovery
   --enable-arc-rpc

Observed behavior:
The execution layer fails during startup while checking consistency and attempting an unwind for the Receipts segment.

Relevant log excerpt:
INFO check_consistency: Healing static file inconsistencies.
INFO check_consistency:check_consistency{read_only=false}: Verifying storage consistency.
INFO check_consistency:check_consistency{read_only=false}: Checking consistency for segment=Receipts ...
INFO Executing unwind after consistency check. unwind_target=Unwind(36161573) inconsistency_source=static file
ERROR failed to run unwind unwind_target=Unwind(36161573) inconsistency_source=static file err=trying to append row to Receipts at index #321414214 but expected index #321359853
ERROR shutting down due to error
Error: trying to append row to Receipts at index #321414214 but expected index #321359853

Location:
crates/node/builder/src/launch/common.rs:565:13

Expected behavior:
A node bootstrapped from the published Arc Testnet snapshot should start successfully, complete consistency checks, and expose a healthy local RPC endpoint.

What I ruled out:

• stale local state
• partial cleanup
• one-off run failure
• systemd-related behavior
• a single interrupted session

This has been reproduced after multiple full wipes and fresh snapshot downloads with the same failure signature.

Question:
Can you confirm whether the currently published Arc Testnet snapshot is valid for arc-node v0.6.0?

If not:

• is there a newer recommended commit on main that should be used instead of v0.6.0?
• or is the current public snapshot known to be inconsistent/corrupted for EL bootstrap?

If useful, I can provide:

• full startup logs
• directory sizes after snapshot extraction
• exact binary versions
• system details

[Bela1401]

I also checked the current open PRs, including #18, and I do not see changes in the execution bootstrap / snapshot / reth startup path that would obviously explain or fix this issue. This still looks like either a snapshot integrity problem or a v0.6.0 bootstrap incompatibility.

[romac]

Thank you for the report, we are looking into this. May I ask what CPU architecture you are running on? Arm or x64?

[Bela1401]

Thanks for looking into it.

This is running on a dedicated bare-metal server, not a VPS.
The CPU is Intel Core i7-8700, and the architecture is x86_64.

For reference, lscpu reports:
Architecture: x86_64

OS is Ubuntu 24.04.

[ZhiyuCircle]

Thanks for the detailed report, we were able to reproduce and identify the root cause.

The currently published pruned snapshot was produced by a node running an older version. That node's pruning left a gap in the Receipts segment between where static files end and where MDBX data begins. When v0.6.0 starts without --full, its consistency check detects this gap and fails during the unwind.

Workaround: Start the execution layer once with the --full flag:

arc-node-execution node \
  --chain arc-testnet \
  --datadir /root/.arc/execution \
  --full \
  ... (rest of your flags)

This will reconcile the tables. After that initial run, you can restart without --full and it should work normally.

We are working on updating the published snapshot and documentation to prevent this from happening going forward.

[Bela1401]

Confirmed, the workaround resolved it.

Starting the execution layer once with --full, then restarting without --full, allowed the node to bootstrap successfully. After that, both EL and CL started normally and the node began syncing/finalizing blocks.

Thanks for identifying the root cause.