On Wed, Oct 16, 2019 at 05:26:29PM -0600, Shuah Khan wrote: > > I don't really buy the argument that unit tests should be deterministic > Possibly, but I would opt for having the ability to feed test data. I strongly believe that unit tests should be deterministic. Non-deterministic tests are essentially fuzz tests. And fuzz tests should be different from unit tests. We want unit tests to run quickly. Fuzz tests need to be run for a large number of passes (perhaps hours) in order to be sure that we've hit any possible bad cases. We want to be able to easily bisect fuzz tests --- preferably, automatically. And any kind of flakey test is hell to bisect. It's bad enough when a test is flakey because of the underlying code. But when a test is flakey because the test inputs are non-deterministic, it's even worse. - Ted
