On 2/14/25 02:06, Akio Kakuno (Fujitsu) via Devel wrote:
> Hi, all!
>
> I'm adding three test for CCA compatibility:
> domaincapstest, qemucapabilitiestest, and qemuxmlconftest.
> This is because SEV-SNP added these three tests.
>
> I have three questions regarding these tests:
> 1. How to add tests to qemuxmlconftest
> 2. How to create launch-security-cca.xml
> 3. About the file output with VIR_TEST_REGENERATE_OUTPUT=1
>
> 1. How to add tests to qemuxmlconftest
> Following the example of launch-security-sev-snp tests, I've done the following.
> Is this correct?
>
> (1) Placed the following three files in tests/qemuxmlconfdata:
> launch-security-cca.xml
> launch-security-cca.aarch64-latest.xml
> launch-security-cca.aarch64-latest.args
>
The placement is correct, yes. BUT ...
> (2) Added the test processing to qemuxmlconftest.c's mymain() function:
> DO_TEST_CAPS_ARCH_LATEST_FULL("launch-security-cca",
> "aarch64",
> ARG_QEMU_CAPS,
> QEMU_CAPS_CCA_GUEST,
> QEMU_CAPS_LAST);
... this can be simplified to:
DO_TEST_CAPS_ARCH_LATEST("launch-security-cca", "aarch64");
>
> 2. How to create launch-security-cca.xml
> Do I need to handmade this from scratch or is there an automated method?
Basically it's hand written. What I usually do is I copy-paste the
domain XML I used when developing and testing a feature. And then cut
off all unnecessary elements.
>
> 3. About the file output with VIR_TEST_REGENERATE_OUTPUT=1
> I created launch-security-cca.aarch64-latest.* using the method described in
> doc/advanced-tests.rst.
> And, I created the test data for qemucapabilitiestest and domaincapstest using
> the method described in tests/qemucapabilitiesdata/README.rst.
> VIR_TEST_REGENERATE_OUTPUT=1 ./qemuxmlconftest
> VIR_TEST_REGENERATE_OUTPUT=1 ./domaincapstest
> VIR_TEST_REGENERATE_OUTPUT=1 ./qemucapabilitiestest
>
> Can I use the generated file for testing as is?
> Because doc/advanced-tests.rst says:
> "VERY CAREFULLY to ensure they are correct"
> > I assume that automatically generated expected values are
checked for accuracy.
Not really. It takes a machine brain to decide whether those files
follow some syntax (e.g. whether JSON is valid), but it takes human
brain to decide whether full combination of cmd line arguments actually
makes sense. I'd say - if you're able to start generated cmd line
(modulo some FD passing stuff - see my point above about cutting off
unnecessary elements), then you're probably fine.
> If correct, they are adopted; otherwise, investigation and remediation are undertaken.
> However, due to the lack of explicit documentation, we require confirmation.
>
> Also, it appears to be generating expected values based on the execution environment.
> Do we need to worry about variations in execution environments?
No. Our tests should generate stable enough (and reproducible!) environment.
> For example, executing qemuxmlconftest detects the following comparison errors,
> such as with aarch64-virt-minimal.aarch64-latest, etc.
> Expect [sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
> -m]
> Actual [m]
Yeah, printing diffs is not very userfriendly. You can get better
results with VIR_TEST_DEBUG=2.
>
> Best Regards.
>
Michal
