On 2/10/20 4:01 PM, Collin Walling wrote:
Hello,
I am working on implementing libvirt test cases for the hypervisor-cpu-compare
and -baseline commands. Ideally, I would like to take advantage of the
qemuProcess API to spin up a QEMU instance and run these commands to test real
data queried from the hypervisor.
The tests in libvirt's tests directory are all unit tests, and don't run
external binaries like qemu - they attempt to test various pieces of
code within libvirt in a limited, controlled environment, not requiring
any special privileges, and without being subject to the vagaries of
external programs. qemuxml2argvtest, for example, calls the code that
converts a libvirt domain xml into a qemu commandline, but rather than
starting up a qemu process with that commandline, it then compares the
generated commandline string to the "known correct" commandline that is
stored in a file in a subdirectory of the tests directory (for this
example, look at tests/qemuxml2argvdata/*.xml vs
tests/qemuxml2argvdata/*.args); this way the success of the test doesn't
depend on qemu being installed, the user having the proper privileges to
perform all the other things that qemu needs to run, or be subject to a
failure due to a bug in the version of qemu running on the build system.
Other examples of tests are *xml2xmltest, which parse a test input file,
then format the resulting object back to xml, and compare that result
with known-correct XML that, again, is stored in a file.
Sometimes the code that is being tested calls a function that can't be
realistically called from a unit test that is run on random hardware by
an unprivileged user (e.g. unbinding a PCI device from its host driver
and re-binding it to the vfio-pci driver). In that case, various library
functions are "mocked" (by linking in the .o created from *mock.c files
- virpcimock.c in the case of PCI device driver binding) to return
reasonable values that will allow the libvirt code to be tested without
actually needing to perform the "impossible" operation.
Anyway, the short answer to your questions is that libvirt's tests
directory isn't the proper location (nor is "make check" the proper
time) for a test that starts up a qemu process and sends commands to it.
Something like that would be better suited for an integration test
suite, like the ones that are in the libvirt-tck package (which is a bit
dated, but still very useful, as long as you don't mind writing perl
scripts :-P)
For a unit test in the tests directory, you would want to setup a test
harness (blahtest.c) that links in the files containing your code, calls
those functions with a fixed set of inputs, then compares the results to
fixed known outputs. Along the way you may need to "stub-out" or "mock"
various functions that are called by the code under test but can't
actually be called in the environment of a unit test.
However, I am having issues with my libvirt
tests communicating with a QEMU instance. The API can successfully spin an
instance, but no commands can be sent to QEMU -- not even the qmp_capabilities
handshake. The test case hangs forever with no indication that something went
wrong.
The hang occurs specifically within the qemuProcessQMPLaunch phase of the
qemuProcessQMPStart function. Eventually the libvirt API will get to
qemuMonitorSend, and at this loop...
while (!mon->msg->finished) {
if (virCondWait(&mon->notify, &mon->parent.lock) < 0) {
virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
_("Unable to wait on monitor condition"));
goto cleanup;
}
}
...the program will hang at conditional wait. I kept the QEMU instance alive
after killing my test and connected to the monitor socket that was still
lingering. The qmp_capabilities command was never sent, thus leading me to
believe that the libvirt tests cannot communicate with a QEMU instance.
As s390x is currently the only arch to have implemented these commands, I
believe it would be beneficial to have an easy way to test both the expected
QEMU response and libvirt parsing within a single test case if/when other archs
decide to implement these commands.
I'd like to ask two questions:
1: does it make sense for libvirt tests to communicate with an external binary
(specifically QEMU)?
2: if so, is there some sort of conflict between a libvirt test and a QEMU
binary? I afraid to say that I am at a loss how to repair this or perhaps how
to use the API properly.
I appreciate anyone's help with looking into this.
Note: in case I am not clear, by "libvirt test" I am referring to a test
implemented in the tests directory of the libvirt project.
