On Sun, Apr 15, 2018 at 15:29:23 +0200, Ján Tomko wrote: > On Wed, Apr 11, 2018 at 11:16:27AM -0400, John Ferlan wrote: > > > > > > On 04/04/2018 04:13 AM, Peter Krempa wrote: > > > Allow testing of XML->argv conversion with using a real capability map > > > as used in the qemucapabilitiestest. This allows specifying the required > > > qemu version with the test rather than having to enumerate all the > > > required capabilities. > > > > > > Signed-off-by: Peter Krempa <pkrempa@xxxxxxxxxx> > > > --- > > > tests/qemuxml2argvtest.c | 31 +++++++++++++++++++++++++++++++ > > > 1 file changed, 31 insertions(+) > > Perhaps the macro(s) should be DO_TEST_VERS[ION] or DO_TEST_<ARCH>_VERS > > instead since DO_TEST runs the test with specific capabilities and this > > new model runs the test with a specific version for a specific platform. > > > > How do or should we enforce that when adding a new test from this point > > forward that the DO_TEST_CAPS model is chosen over DO_TEST? I suppose > > there is still value in DO_TEST since one can pick and choose which CAP > > is needed - although I'll freely admit that's usually a cut-n-paste from > > some other test and change the name type activity. > > The problem with that approach is that we're testing against > capability combinations that might never occur in the wild. > > I actually remember fixing some bug that was supposed to be covered by > a unit test, but it was missing the important capability. Actually a very good example is the recent series of adding the 'discard-data' attribute to memory devices. Since the new feature is decided with capability bits, all existing tests were still passing, but the attribute would be added to NVDIMM memory devices which would delete the contents. I think we also need to add a way to take the latest caps present in the repo to be used with the bulk of tests rather than binding it to a specific version. That way we can see whether the command line is stable when we add a capability based test. If the command line will change, the test case then can be split for the specific version of qemu capabilities, so that the contemporary test case still captures the new case, but the old behaviour is kept too.
Attachment:
signature.asc
Description: PGP signature
-- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list
