test_runner: support test order randomization #3801

Job	Run time
Build slim tarball	2m 1s
aarch64-darwin: with shared libraries	19m 43s
x86_64-linux: with shared libraries	25m 55s
aarch64-linux: with shared libraries	20m 51s
x86_64-darwin: with shared libraries	52m 8s
	2h 0m 38s

Provide feedback