On Mon, Jun 27, 2016 at 01:23:46PM +0200, Michal Privoznik wrote: > On 27.06.2016 10:05, Daniel P. Berrange wrote: > > On Sat, Jun 25, 2016 at 04:32:21PM -0600, Eric Blake wrote: > >> [moderator note: the mail server balked at 1.3 megabytes of attachments, > >> so I've stripped them in my reply] > >> > >> On 12/31/1969 05:00 PM, wrote: > >>> Hello everybody, > >>> > >>> I set up a clean libvirt repo with: > >>> git clone > >>> ./autogen.sh > >>> make -j5 > >>> > >>> and then executed tests with various values of LIBVIRT_DEBUG using > >>> the following command: > >>> LIBVIRT_DEBUG=0…5 VIR_TEST_DEBUG=1 make check > >>> > >>> The logs are in this mail's attachments. > >>> > >>> > >>> With LIBVIRT_DEBUG=0 (or =5 or probably any invalid value), the logs say: > >>> "Ignoring invalid log level setting” > >>> but still, 10 tests fail due to various reasons. > >>> > >>> With LIBVIRT_DEBUG=1, there is a slight difference. The number of tests > >>> that fail is also 10, but here, commandtest fails instead of eventtest. > >>> > >>> With LIBVIRT_DEBUG=2, only 9 tests fail, because commandtest passes. > >>> > >>> With LIBVIRT_DEBUG=3, only 1 test fails: qemuargv2xmltest. The > >>> reason is the same as in the previous tests. > >>> > >>> With LIBVIRT_DEBUG=4, it’s just qemuargv2xmltest that is failing, > >>> but for a different reason than before: > >>> "qemuParseCommandLineString should have logged a warning”. > >>> > >>> > >>> Perhaps it’s okay that these tests fail, but still there's no harm > >>> in letting people know. > >>> > >>> Thank you rbogorodskiy for guidance on the mailing list :) > >>> > >>> Have a nice day, > >>> Tomasz > >>> > >> > >> I don't know that anyone ever expected the testsuite to work with > >> LIBVIRT_DEBUG set to anything (because turning on debug can change > >> output of libvirt itself). Things are different with VIR_TEST_DEBUG, > > > > I'd certainly expect the test suite to pass 100% no matter what the > > debug level is set to. Debug logging should not have a functional > > effect on our code, besides outputting the log messages. > > That's not 100% true. If we enable the debug messages, those are printed > to stdout. Now we have some tests that expect certain output or some > exact message to be logged. If we enable debug logs, the outputs are > poisoned and don't match the expected values. > I'd say this is a weakness of our test suite, not actual libvirt code. Hmm, yes, unit tests really should not be expecting precise output on stderr/stdout. Any that do are broken by design IMHO. At most they should be doing a sub-string match if they want to look for something and certainly not fail if extra output appears. Not least because I'll often set LIBVIRT_DEBUG in order to debug the unit tests and would not expect that to have side effects on them. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list