On 06/09/2019 11.43, David Hildenbrand wrote: > On 30.08.19 20:45, Thomas Huth wrote: >> Currently the tests at the end of the .travis.yml script are ignored, >> since we can not use KVM in the Travis containers. But we can actually >> run of some of the kvm-unit-tests with TCG instead, to make sure that >> the binaries are not completely broken. >> Thus introduce a new TESTS variable that lists the tests which we can >> run with TCG. Unfortunately, the ppc64 and s390x QEMUs in Ubuntu also >> need some extra love: The ppc64 version only works with the additional >> "cap-htm=off" setting. And the s390x package lacks the firmware and >> refuses to work unless we provide a fake firmware file here. Any file >> works since the firmware is skipped when "-kernel" is used, so we can >> simply use one of the pre-existing files in the source tree. >> >> Signed-off-by: Thomas Huth <thuth@xxxxxxxxxx> >> --- >> .travis.yml | 19 ++++++++++++++++++- >> 1 file changed, 18 insertions(+), 1 deletion(-) >> >> diff --git a/.travis.yml b/.travis.yml >> index a4a165d..6c14953 100644 >> --- a/.travis.yml >> +++ b/.travis.yml >> @@ -20,24 +20,40 @@ env: >> matrix: >> - CONFIG="" >> BUILD_DIR="." >> + TESTS="vmexit_cpuid vmexit_mov_from_cr8 vmexit_mov_to_cr8 vmexit_ipi >> + vmexit_ple_round_robin vmexit_tscdeadline vmexit_tscdeadline_immed" >> - CONFIG="" >> BUILD_DIR="x86-builddir" >> + TESTS="ioapic-split ioapic smptest smptest3 eventinj msr port80 syscall >> + tsc rmap_chain umip intel_iommu vmexit_inl_pmtimer vmexit_ipi_halt" >> - CONFIG="--arch=arm --cross-prefix=arm-linux-gnueabihf-" >> BUILD_DIR="." >> + TESTS="selftest-vectors-kernel selftest-vectors-user selftest-smp" >> - CONFIG="--arch=arm --cross-prefix=arm-linux-gnueabihf-" >> BUILD_DIR="arm-buildir" >> + TESTS="pci-test pmu gicv2-active gicv3-active psci selftest-setup" >> - CONFIG="--arch=arm64 --cross-prefix=aarch64-linux-gnu-" >> BUILD_DIR="." >> + TESTS="selftest-vectors-kernel selftest-vectors-user selftest-smp" >> - CONFIG="--arch=arm64 --cross-prefix=aarch64-linux-gnu-" >> BUILD_DIR="arm64-buildir" >> + TESTS="pci-test pmu gicv2-active gicv3-active psci timer selftest-setup" >> - CONFIG="--arch=ppc64 --endian=little --cross-prefix=powerpc64le-linux-gnu-" >> BUILD_DIR="." >> + TESTS="spapr_hcall emulator rtas-set-time-of-day" >> + ACCEL="tcg,cap-htm=off" >> - CONFIG="--arch=ppc64 --endian=little --cross-prefix=powerpc64le-linux-gnu-" >> BUILD_DIR="ppc64le-buildir" >> + TESTS="rtas-get-time-of-day rtas-get-time-of-day-base" >> + ACCEL="tcg,cap-htm=off" >> - CONFIG="--arch=s390x --cross-prefix=s390x-linux-gnu-" >> BUILD_DIR="." >> + TESTS="diag10 diag308" >> + ACCEL="tcg,firmware=s390x/run" >> - CONFIG="--arch=s390x --cross-prefix=s390x-linux-gnu-" >> BUILD_DIR="s390x-builddir" >> + TESTS="sieve" >> + ACCEL="tcg,firmware=s390x/run" > > What about the other s390x tests? (is the QEMU binary too old to make > them pass?) Yes. All the others are failing. > The issue with TCG is that you can easily get false negatives in case > the QEMU binary changes (e.g., new Ubuntu release). That should not be a big deal, since the Ubuntu version is locked to a certain release in the travis.yml file. So we should not get a new version of QEMU by accident here, only if we explicitly update the "dist:" line in the yml file. > "to make sure that the binaries are not completely broken" - while I > understand the intuition behind that, I wonder if this is relevant in > practice Well, the test coverage here is certainly far from being perfect, but it's still better than nothing. Imagine one of the patches changes common code in the lib/ folder and it only works on one architecture that the author tested. Chances are quite good that we detect that problem with this patch now. Thomas
