On Fri, 26 Aug 2011, Christopher Li wrote:
Your patch now makes the runner verbose for "known to fail" tests
which is definitely not something we should do. If someone tagged
the test as "known to fail", we should treat it just like we treat
passed test cases.
That is intentional. I haven't seen you reply on my comment so I take
the liberty to change it. I do want to see the "known to fail" test
case.
Sorry about that. I somehow missed the email.
The whole point of my patch was to make "make check" pinpoint
*unexpected* breakage so that anyone who bothers to do "make check"
on their patches can never cause regressions.
How to you know your new patch did not break "known to fail" test
case in a different way than originally? It could be regression as well.
I think that's irrelevant, really. I happen to think it's completely
pointless to even run test cases that are "known to fail" because it
creates confusion and adds little value.
And if we really care that a "known to fail" test case fails in a certain
way, we can turn it into a "known _not_ to fail" and assert the exact
failure case.
You can diff the aggregate output of "make check" before and after
the new patch to see if the new patch affect any test case at all.
That is better than blindly skip the "known to fail" test case.
No, that doesn't scale at all if you run "make check" every few minutes
like I do.
Yes, I have a different usage case in mind. I want to look at what currently
breaks. In the long run, hopefully we fix all the known issues so this
wouldn't be any issue at all.
Agreed.
You can submit a patch to add a config for it if you are so insist on
the "I don't care about already broken test cases" usage case.
To me, diff the "make check" output is straightly better.
OK, a flag would definitely work for me.
Pekka
--
To unsubscribe from this list: send the line "unsubscribe linux-sparse" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
