Try function

[DoubleNom]

Problem

Currently, there is no easy way to check an expectation without failing the test. The current workaround is using the ExpectToBeXXX functions, then provide the result to the actual Expectation to not make the test fail.

Proposition

I propose the introduction of a Try function, much like the already existing tryFunction, but that will allows to revert the test expectation to their when-called state between each tries.

Example

Sequence("MySequence", function()
  Test("MyTest", function()    
    Expect(doMeasure(), "My first measure"):ToBeInPercentage(3.3, 10)
    Try(
      function(args) Expect(doMeasure(), "My second measure"):ToBeInPercentage(3.3, 10) end,
      {
        max_retries = 5, -- Max number of tries to do, default to 1
        cooldown_ms = 100, -- Delay between each tries, default to 0
        args = {"arg1", "arg2"} -- Args to forward to function, can be nil
      }
    )
  end)
end)

Implementation

The following pseudocode provide an implementation example

1. Check function is called inside a Test Scope, raise error if not.
2. Store the current number of expectation in scope.
3. Check if has remaining tries, if not, exit function.
4. Call provided function, while forwarding arguments, if any.
5. For each new expectation
   a. If success, continue
   b. If failure
   1. delete all new expectation
   2. sleep for cooldown_ms
   3. increment try counter
   4. jump back to step 3

Limitations

• The Try function can only be used when inside a Test scope

[smartel99]

We'd have to modify the behavior of Expect to get this behavior to work I think, iirc, Expect immediately commits the result in the log and we do not have a way to remove it from the log.

I love the idea of a try block tho, I think this is a great feature

[DoubleNom]

It stores the expectation result in a table, which will then be store in the log file.

[smartel99]

Won't we end up with duplicates then? One for each attempt?

[DoubleNom]

The function will erase the newly added expectation, if it is allowed to retry. Otherwise, it keeps it as-is