One optimization bytebox has done is moving the check for if the stack will overflow from instructions to function calls. The validation pass for each function tracks the maximum number of values it has seen, and records that as the max number of values that stack frame will need. However, if there is a loop that pushes multiple values onto the stack, the validation pass can't detect that. Instead, we should record the number of values pushed inside each block, and whenever a new block is pushed or jumped to in the case of a loop, determine if enough stack space is available.
Ideally there would be a test validating this behavior as well.
One optimization bytebox has done is moving the check for if the stack will overflow from instructions to function calls. The validation pass for each function tracks the maximum number of values it has seen, and records that as the max number of values that stack frame will need. However, if there is a loop that pushes multiple values onto the stack, the validation pass can't detect that. Instead, we should record the number of values pushed inside each block, and whenever a new block is pushed or jumped to in the case of a loop, determine if enough stack space is available.
Ideally there would be a test validating this behavior as well.