On 30/07/20 13:32, Andrew Jones wrote: >> This should have practical implications. I remember, for example, that I had >> a discussion with Paolo in the past regarding “xpass” being reported as a >> failure. The rationale was that if a test that is expected to fail on KVM >> (since KVM is known to be broken) surprisingly passes, there is some problem >> that should be reported as a failure. I would argue that if the project is >> hypervisor-agnostic, “xpass” is not a failure. > We can use compile-time or run-time logic that depends on the target to > decide whether a test should be a normal test (pass/fail) or an > xpass/xfail test. Yeah, that would be basically the old "errata" mechanism. Probably we should have some kind of configure script that builds "errata" files based on hypervisor, uname or whatnot. (Upstream should do the bare minimum, just the hypervisor). Paolo
