On 01/25/2016 02:05 AM, Martin Kletzander wrote: > On Fri, Jan 22, 2016 at 02:09:18PM -0500, Cole Robinson wrote: >> Most qemuxml2xml tests expect that the input XML is unchanged after >> parsing. This is unlike 99% of new qemu configs in the wild, which after >> initial parsing end up with stable PCI device addresses. The xml2xml bit >> doesn't currently hit that code path though, so most XML testing indeed >> does not change. >> >> Future patches will add that PCI address bits, which means most test cases >> will have different output. So let's do away with the hardcoded same vs >> different test split, and always track a separate output file. Tests can >> still have same input and output, it just necessitates 2 separate XML files. > > I don't know whether there was a discussion behind this, if there was, I > regret missing it. There wasn't you, you didn't miss anything :) I don't like this very much as it's not visible from > the test if that file should change or not. It is visible in the files, > but you have to diff them. If we really need this, I suggest using > symlinks to save some space and also time as it will be clearer which > XMLs are the same and which are not. I agree that this method is not space efficient, and that it loses some visibility into testing that XML should not change. And the patch isn't necessary for this patch series so I can drop it However the point I made above still stands: nearly all these test files are going to need to change, because we should be testing qemu device address assignment unconditionally. In that future case, we _could_ change all the input to match the output and maintain the current state... however I don't think that's a good idea, since that means we will have very little xml2xml testing for the device address My opinion is that the way we do xml2xml testing right now is not ideal and it deserves a larger rework, of which this is a step in the right direction. xml2xml tests should take a few forms: - testing generic XML is parsed and formatted correctly - testing generic XML validation failures - testing driver specific XML handling Right now the vast majority of the xml2xml testing is handled by qemuxml2xml, when the tests are not driver specific. I think we should aim to move the generic XML tests to (the new) genericxml2xml. If we want to keep the explicit sameness tracking I think that's the place to do it. That would also be a good opportunity to consolidate a lot of the xml2xml tests... I figure we could have genericxml2xml-disks.xml and genericxml2xml-interfaces.xml and rather than drop in a whole new file we just extend those for new XML testing specific to the device type. Personally though I don't think we need to enforce the 'sameness' checking too hard. Any regressions should stick out clearly whether they share the XML or not: make changes, do VIR_TEST_REGENERATE_OUTPUT=1 make check, then carefully review the diff is the same pattern either way Thanks, Cole -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list