Align optimized SMT appendix with hashed-zero algorithm and fix example encoding

[jintekc]

Closes #317
Closes #318

[parasyte]

One thing that might be useful to avoid some confusion is specifying that the concatenation operator (+) and concat() function takes a byte array, not a string. And that the encoded hashes in SMT proofs must be decoded before operating on them. If this is already specified somewhere and I missed it, please point it out.

I can imagine a scenario where the word "concatenation" is interpreted as "string concatenation" instead of "byte array concatenation".

Also, the SMT Proof Verification algorithm description and pseudocode currently do not work with "non-updates" (i.e., with a proof of non-inclusion). It's a simple fix; make the concatenation of hashed-nonce and update hash optional (while maintaining the outer hash - very important). The updateId property on the SMT Proof data structure is already optional, but the algorithm and pseudocode do not acknowledge that.

---

Still reviewing the changes. This is just a thought that came up while going over it.

[parasyte]

Looks great! I found only one minor suggestion.

The new diagram reads better (IMHO) with every level of the tree covered and each node receiving a meaningful identifier. I'll create a separate PR for fixing the unrelated SMT Proof Verification algorithm issue I noticed.

[jintekc]

Looks great! I found only one minor suggestion.

The new diagram reads better (IMHO) with every level of the tree covered and each node receiving a meaningful identifier. I'll create a separate PR for fixing the unrelated SMT Proof Verification algorithm issue I noticed.

Thanks for the prompt review! Addressed here:

1. added a note that hash()/concat()/+ operate on byte arrays (not strings) and that the base64url proof fields must be decoded to raw bytes first
2. the hashes bullet now states each hash MUST be base64url without padding. Left the non-update (proof-of-non-inclusion) algorithm fix for your separate PR, as you suggested.

[parasyte]

The + operator is also used in the Optimized SMT chapter. But this works for me.

[danpape]

I also approve, looks great! Like Jay mentioned though, maybe you can also add the "Throughout this section, hash() denotes SHA-256..." paragraph from src/algorithms.md to src/appendix/optimized-smt.md?

[jintekc]

@danpape @parasyte - pushed one follow-up commit addressing the review: added the hash()/+ byte-array and base64url-decoding note to src/appendix/optimized-smt.md, and linked the appendix to the normative [SMT Proof Verification] algorithm. No algorithm changes. Needs two more approvals.