Another aspect that I have not seen discussed yet is the handling of testcases that expose a bug, and which are usually integrated as non-reg tests afterwards. If we're going the way of using git-note's, we can easily link the commit which adds the testcase to the non-reg testsuite. But we could be interested in running the testcase against an arbitrary commit, thus allowing some sort of automated bisect (modulo applicability of the testcase to other commits, which may cause false negatives, but also false positives, and thus may need to be hand-driven or at least hand-reviewed). For this, however, would it be sufficient to have the testcase committed somewhere in the history tree ? Our hypothetic automatic bisect tool could cherry-pick that commit and run the testcase, but at would place an arbitrary restriction of "one testcase per commit", which would probably a bad idea. And having the annotation point to the blob does not seem a good idea either, at least because currently in git tools we have test scripts that excercise several testcases each. Even an annotation to both a commit and a blob changed by that commit would not be sufficiently accurate, when several testcases are added to the same script in a single commit. Another options would be to embed the testcase in an annotation: I can't really see that being used, since that would duplicate the testcase. That makes me wonder whether it would not be required to have testcases described in seperate files ? Best regards, -- Yann. - To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
