When the test suites fail, we have a DEBUG_TESTS=1 environment variable
which can be set to show the full XML doc or string pair that failed to
match. Unfortunately these docs can be quite large, so it is hard to
spot the difference between the actual and expected output.
So this patch introduces a virtTestDifference(actual, expect) method
which will trim the head & tail of the actual & expected strings until
the point at which they differ. This gives immediate clear feedback on
where the bug is
Second, it will enumerate each test condition & pretty print align things
a little better.
When I apply this to a test which fails, i now see output like this:
19) Xen SEXPR-2-XML fv-usbmouse -> fv-usbmouse ...
Expect [localtime'/>
<devices>
<emulator>/usr/lib64/xen/bin/qemu-dm</emulator>
<disk type='file' device='disk'>
<driver name='file'/>
<source file='/root/foo.img'/>
<target dev='hda'/>
</disk>
<interface type='network]
Actual [utc'/>
<devices>
<emulator>/usr/lib64/xen/bin/qemu-dm</emulator>
<disk type='file' device='disk'>
<driver name='file'/>
<source file='/root/foo.img'/>
<target dev='hda'/>
</disk>
<interface type='bridge]
... FAILED
So you can see immediately that they start to differ in the 'localtime' vs
'utc' value, and the last place at which they differ is the network interface
which has type of 'network' vs 'bridge'.
This has dramatically eased my debugging of tests when adding the serial
device options :-)
This patch is merely the helper routines. I'll include the actualy test
suite changes later.
Dan.
Index: testutils.h
===================================================================
RCS file: /data/cvs/libvirt/tests/testutils.h,v
retrieving revision 1.6
diff -u -p -r1.6 testutils.h
--- testutils.h 10 Apr 2008 16:53:29 -0000 1.6
+++ testutils.h 18 Apr 2008 00:22:55 -0000
@@ -32,6 +32,11 @@ extern "C" {
char **buf,
int buflen);
+
+ int virtTestDifference(FILE *stream,
+ const char *expect,
+ const char *actual);
+
#ifdef __cplusplus
}
#endif
Index: testutils.c
===================================================================
RCS file: /data/cvs/libvirt/tests/testutils.c,v
retrieving revision 1.11
diff -u -p -r1.11 testutils.c
--- testutils.c 10 Apr 2008 16:53:29 -0000 1.11
+++ testutils.c 18 Apr 2008 00:22:55 -0000
@@ -19,6 +19,7 @@
#include <sys/stat.h>
#include <sys/wait.h>
#include <unistd.h>
+#include <string.h>
#include <fcntl.h>
#include <limits.h>
#include "testutils.h"
@@ -58,6 +59,12 @@ virtTestRun(const char *title, int nloop
{
int i, ret = 0;
double *ts = NULL;
+ static int counter = 0;
+
+ counter++;
+
+ fprintf(stderr, "%2d) %-65s ... ", counter, title);
+ fflush(stderr);
if (nloops > 1 && (ts = calloc(nloops,
sizeof(double)))==NULL)
@@ -76,12 +83,12 @@ virtTestRun(const char *title, int nloop
}
}
if (ret == 0 && ts)
- fprintf(stderr, "%-50s ... OK [%.5f ms]\n", title,
+ fprintf(stderr, "OK [%.5f ms]\n",
virtTestCountAverage(ts, nloops));
else if (ret == 0)
- fprintf(stderr, "%-50s ... OK\n", title);
+ fprintf(stderr, "OK\n");
else
- fprintf(stderr, "%-50s ... FAILED\n", title);
+ fprintf(stderr, "FAILED\n");
free(ts);
return ret;
@@ -206,3 +213,57 @@ int virtTestCaptureProgramOutput(const c
}
}
}
+
+
+/**
+ * @param stream: output stream write to differences to
+ * @param expect: expected output text
+ * @param actual: actual output text
+ *
+ * Display expected and actual output text, trimmed to
+ * first and last characters at which differences occur
+ */
+int virtTestDifference(FILE *stream,
+ const char *expect,
+ const char *actual)
+{
+ const char *expectStart = expect;
+ const char *expectEnd = expect + (strlen(expect)-1);
+ const char *actualStart = actual;
+ const char *actualEnd = actual + (strlen(actual)-1);
+
+ if (getenv("DEBUG_TESTS") == NULL)
+ return 0;
+
+ /* Skip to first character where they differ */
+ while (*expectStart && *actualStart &&
+ *actualStart == *expectStart) {
+ actualStart++;
+ expectStart++;
+ }
+
+ /* Work backwards to last character where they differ */
+ while (actualEnd > actualStart &&
+ expectEnd > expectStart &&
+ *actualEnd == *expectEnd) {
+ actualEnd--;
+ expectEnd--;
+ }
+
+ /* Show the trimmed differences */
+ fprintf(stream, "\nExpect [");
+ if ((expectEnd - expectStart + 1) &&
+ fwrite(expectStart, (expectEnd-expectStart+1), 1, stream) != 1)
+ return -1;
+ fprintf(stream, "]\n");
+ fprintf(stream, "Actual [");
+ if ((actualEnd - actualStart + 1) &&
+ fwrite(actualStart, (actualEnd-actualStart+1), 1, stream) != 1)
+ return -1;
+ fprintf(stream, "]\n");
+
+ /* Pad to line up with test name ... in virTestRun */
+ fprintf(stream, " ... ");
+
+ return 0;
+}
--
|: Red Hat, Engineering, Boston -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
--
Libvir-list mailing list
Libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list