On 05/17, Junio C Hamano wrote: > Ævar Arnfjörð Bjarmason <avarab@xxxxxxxxx> writes: > > >> Well, it is one thing to place git-annex under CI to make sure its > >> latest and greatest works together well with our latest and greatest > >> (and it may be something we want to see happen), but driving its > >> tests from our testsuite sounds like a tail wagging the dog, at > >> least to me. > > > > To me this is just a question of: > > > > * Is it the case that git-annex tests for a lot of edge cases we don't > > test for: Yes, probably. As evidenced by them spotting this > > regression, and not us. > > And I'd encourage them to keep doing so. > > > * We can (and should) add a test for the specific breakage we caused > > in 2.13.0, but that's no replacement for other things annex may be > > covering & we may be missing which'll catch future breakages. > > > > * It's a pretty established practice to test a library (git) along > > with its consumers (e.g. annex) before a major release. > > I am not so sure about the division of labor. What you are > advocating would work _ONLY_ if we test with a perfect & bug-free > version of the consumers. If they are also a moving target, then > I do not think it is worth it. After all, we are *not* in the > business of testing these consumers. I agree with this. It makes no sense to test consumers of git, its downstream's job to do that. Though I do think that its perfectly reasonable to test that our API works as advertised such that consumer's can rely on git. > > Unless I misunderstood you and you were saying that we freeze a > version, or a set of versions, of customer that is/are known to pass > their own tests, and test the combination of that frozen version of > the customer with our daily development. If that is the case, then > I would agree that we are using their test to test us, not them. > But I somehow didn't get that impression, hence my reaction. -- Brandon Williams