Hi, On Sun, May 03, 2009 at 11:24:07AM +0200, Robin Rosenberg wrote: > söndag 03 maj 2009 10:31:30 skrev "Ferry Huberts (Pelagic)" <ferry.huberts@xxxxxxxxxx>: > > > 3) We should have a convention like C Git for marking known breakages. > > > One option is FIXME, another it so go JUnit 4 and abuse the expected exception > > > annotation (using it for declaring OK exceptions is pretty bad use anyway I think, > > > so we might use it for something better), or perhaps the @Ignore annotation which > > > is meant specifically for this and other cases. A FIXME can be implemented right > > > away. > > > > standard pratice for junit would be to write a test case on what you would > > expect to be _correct_ behaviour. obviously that test would then fail. > > it would be a know failure in the test suite. do not go ignoring it. it's > > better to keep being reminded that stuff doesn't work :-) > > What I've see so far is that people start ignoring almost any failure, including new ones, when the test suites contains fails with "known" failues. The assumption is that the failed tests were the same as before. > > Worse, automated tests have a hard time telling the difference. Currently I ran > the jgit tests as part of the Eclipse plugin build and I want it to stop if there is a problem that we don't know of. > > "Annotation" of different kinds can be "grepped" for so we can find the broken > cases separately and even refuse completion of release builds if we decide > on that. > > Our primary UI right now is the Eclipse JUnit tests runner and I don't want > to be remined of Shawn's or whoever's bugs when trying to make sure I don't > break anything. Red = *I* broke something or found something new. > > TestNG has a nice way of classifying tests so, we could mark failures as "known failures" and specifically exclude/include them when invoking the > JUnit tests. you could use test suites to easily circumvent this in JUnit, even in JUnit 3.x. Just set up two test suites: one for the tests that should pass and one for the tests with known breakages. That way you can run either only the "good" tests or only the broken ones. Or even both, and you can easily discern failures caused by known breakages from your new breakages by looking at the tree of tests in eclipse's JUnit view. Regards, Gábor -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html