> -----Original Message----- > From: Brendan Higgins > > On Thu, Oct 17, 2019 at 3:25 PM <Tim.Bird@xxxxxxxx> wrote: > > > > > -----Original Message----- > > > From: Theodore Y. Ts'o on October 17, 2019 2:09 AM > > > > > > On Wed, Oct 16, 2019 at 05:26:29PM -0600, Shuah Khan wrote: > > > > > > > > I don't really buy the argument that unit tests should be deterministic > > > > Possibly, but I would opt for having the ability to feed test data. > > > > > > I strongly believe that unit tests should be deterministic. > > > Non-deterministic tests are essentially fuzz tests. And fuzz tests > > > should be different from unit tests. > > > > I'm not sure I have the entire context here, but I think deterministic > > might not be the right word, or it might not capture the exact meaning > > intended. > > > > I think there are multiple issues here: > > 1. Does the test enclose all its data, including working data and expected > results? > > Or, does the test allow someone to provide working data? This alternative > > implies that either the some of testcases or the results might be different > depending on > > the data that is provided. IMHO the test would be deterministic if it always > produced > > the same results based on the same data inputs. And if the input data was > deterministic. > > I would call this a data-driven test. > > > > Since the results would be dependent on the data provided, the results > > from tests using different data would not be comparable. Essentially, > > changing the input data changes the test so maybe it's best to consider > > this a different test. Like 'test-with-data-A' and 'test-with-data-B'. > > That kind of sound like parameterized tests[1]; ... > > [1] https://dzone.com/articles/junit-parameterized-test Both Iurii and you pointed me at parameterized tests. This sounds like what I am talking about. We have a lot of infrastructure for these concepts in Fuego, but it's called something different. I'm not sure if I should feel smart for having developed some of these concepts independently or dumb for not having known about this (apparently) standard nomenclature. ;-) Thanks for the reference! -- Tim
