Dear Michal
Thank you for your reply.
I will create a test based on your comment and post v2.
-----Original Message-----
From: Michal Prívozník <mprivozn@xxxxxxxxxx>
Sent: Monday, February 17, 2025 11:34 PM
To: Kakuno, Akio/角野 秋男 <fj3333bs@xxxxxxxxxxx>; 'devel@xxxxxxxxxxxxxxxxx' <devel@xxxxxxxxxxxxxxxxx>
Subject: Re: [PATCH 1/1] RFC: Add Arm CCA support for getting capability information and running Realm VM
On 2/14/25 02:06, Akio Kakuno (Fujitsu) via Devel wrote:
> Hi, all!
>
> I'm adding three test for CCA compatibility:
> domaincapstest, qemucapabilitiestest, and qemuxmlconftest.
> This is because SEV-SNP added these three tests.
>
> I have three questions regarding these tests:
> 1. How to add tests to qemuxmlconftest
> 2. How to create launch-security-cca.xml
> 3. About the file output with VIR_TEST_REGENERATE_OUTPUT=1
>
> 1. How to add tests to qemuxmlconftest
> Following the example of launch-security-sev-snp tests, I've done the following.
> Is this correct?
>
> (1) Placed the following three files in tests/qemuxmlconfdata:
> launch-security-cca.xml
> launch-security-cca.aarch64-latest.xml
> launch-security-cca.aarch64-latest.args
>
The placement is correct, yes. BUT ...
> (2) Added the test processing to qemuxmlconftest.c's mymain() function:
> DO_TEST_CAPS_ARCH_LATEST_FULL("launch-security-cca",
> "aarch64",
> ARG_QEMU_CAPS,
> QEMU_CAPS_CCA_GUEST,
> QEMU_CAPS_LAST);
... this can be simplified to:
DO_TEST_CAPS_ARCH_LATEST("launch-security-cca", "aarch64");
>
> 2. How to create launch-security-cca.xml
> Do I need to handmade this from scratch or is there an automated method?
Basically it's hand written. What I usually do is I copy-paste the domain XML I used when developing and testing a feature. And then cut off all unnecessary elements.
>
> 3. About the file output with VIR_TEST_REGENERATE_OUTPUT=1
> I created launch-security-cca.aarch64-latest.* using the method described in
> doc/advanced-tests.rst.
> And, I created the test data for qemucapabilitiestest and domaincapstest using
> the method described in tests/qemucapabilitiesdata/README.rst.
> VIR_TEST_REGENERATE_OUTPUT=1 ./qemuxmlconftest
> VIR_TEST_REGENERATE_OUTPUT=1 ./domaincapstest
> VIR_TEST_REGENERATE_OUTPUT=1 ./qemucapabilitiestest
>
> Can I use the generated file for testing as is?
> Because doc/advanced-tests.rst says:
> "VERY CAREFULLY to ensure they are correct"
> > I assume that automatically generated expected values are
checked for accuracy.
Not really. It takes a machine brain to decide whether those files follow some syntax (e.g. whether JSON is valid), but it takes human brain to decide whether full combination of cmd line arguments actually makes sense. I'd say - if you're able to start generated cmd line (modulo some FD passing stuff - see my point above about cutting off unnecessary elements), then you're probably fine.
> If correct, they are adopted; otherwise, investigation and remediation are undertaken.
> However, due to the lack of explicit documentation, we require confirmation.
>
> Also, it appears to be generating expected values based on the execution environment.
> Do we need to worry about variations in execution environments?
No. Our tests should generate stable enough (and reproducible!) environment.
> For example, executing qemuxmlconftest detects the following comparison errors,
> such as with aarch64-virt-minimal.aarch64-latest, etc.
> Expect [sandbox
> on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -m]
> Actual [m]
Yeah, printing diffs is not very userfriendly. You can get better results with VIR_TEST_DEBUG=2.
>
> Best Regards.
>
Michal
