Add utxo set soft validation.

[gdassori]

Do checkpoints based soft validation (no scripts evaluation) and track the utxo set.

[gdassori]

• Drop the existing UTXO storage.
• Redesign the blockchain .repository.
• Deserialize Blocks to the UTXO.
• Design the UTXO repository w/ UTXO sharding.
• Avoid validation before the checkpoints.
• Save the UTXO status when saving a block.
• Ensure a spent UTXO exists (soft validation) when saving a block.

[dakk]

A tiny hint for performance improvement: if the block is old in the chain, the "Ensure a spent UTXO exists (soft validation) when saving a block." can be neglected without loosing on security. Btw when saving a block you need to mark all spent utxo as spent

[gdassori]

Let me see if I understand your point... are you suggesting to avoid the READ on the UTXO, and just hit a DELETE, basically to spend it, when the block height is, let's say... 100 blocks behind? (magic number).

So, when I receive a block with height = maxheight - 100:

for spent_utxo in block:
   repo.delete(spent_utxo)

else, with a recent block:

for spent_utxo in block:
   repo.ensure_utxo_exists(spent_utxo)
   repo.delete(spent_utxo)

Sort of? Sounds good to me.