On 12/14/23 10:41, Alex Bennée wrote: > Eric Auger <eric.auger@xxxxxxxxxx> writes: > >> Hi Cleber, >> >> On 12/13/23 21:08, Cleber Rosa wrote: >>> Alex Bennée <alex.bennee@xxxxxxxxxx> writes: >>> >>>> Cleber Rosa <crosa@xxxxxxxxxx> writes: >>>> >>>>> Based on many runs, the average run time for these 4 tests is around >>>>> 250 seconds, with 320 seconds being the ceiling. In any way, the >>>>> default 120 seconds timeout is inappropriate in my experience. >>>> I would rather see these tests updated to fix: >>>> >>>> - Don't use such an old Fedora 31 image >>> I remember proposing a bump in Fedora version used by default in >>> avocado_qemu.LinuxTest (which would propagate to tests such as >>> boot_linux.py and others), but that was not well accepted. I can >>> definitely work on such a version bump again. >>> >>>> - Avoid updating image packages (when will RH stop serving them?) >>> IIUC the only reason for updating the packages is to test the network >>> from the guest, and could/should be done another way. >>> >>> Eric, could you confirm this? >> Sorry for the delay. Yes effectively I used the dnf install to stress >> the viommu. In the past I was able to trigger viommu bugs that way >> whereas getting an IP @ for the guest was just successful. >>>> - The "test" is a fairly basic check of dmesg/sysfs output >>> Maybe the network is also an implicit check here. Let's see what Eric >>> has to say. >> To be honest I do not remember how avocado does the check in itself; my >> guess if that if the dnf install does not complete you get a timeout and >> the test fails. But you may be more knowledged on this than me ;-) > I guess the problem is relying on external infrastructure can lead to > unpredictable results. However its a lot easier to configure user mode > networking just to pull something off the internet than have a local > netperf or some such setup to generate local traffic. > > I guess there is no loopback like setup which would sufficiently > exercise the code? I don't think so. This test is a reproducer for a bug I encountered and fixed in the past. Besudes, I am totally fine moving the test out of the gating CI and just keep it as a tier2 test, as suggested by Phil. Thanks Eric > >> Thanks >> >> Eric >>>> I think building a buildroot image with the tools pre-installed (with >>>> perhaps more testing) would be a better use of our limited test time. >>>> >>>> FWIW the runtime on my machine is: >>>> >>>> ➜ env QEMU_TEST_FLAKY_TESTS=1 ./pyvenv/bin/avocado run ./tests/avocado/intel_iommu.py >>>> JOB ID : 5c582ccf274f3aee279c2208f969a7af8ceb9943 >>>> JOB LOG : /home/alex/avocado/job-results/job-2023-12-11T16.53-5c582cc/job.log >>>> (1/4) ./tests/avocado/intel_iommu.py:IntelIOMMU.test_intel_iommu: PASS (44.21 s) >>>> (2/4) ./tests/avocado/intel_iommu.py:IntelIOMMU.test_intel_iommu_strict: PASS (78.60 s) >>>> (3/4) ./tests/avocado/intel_iommu.py:IntelIOMMU.test_intel_iommu_strict_cm: PASS (65.57 s) >>>> (4/4) ./tests/avocado/intel_iommu.py:IntelIOMMU.test_intel_iommu_pt: PASS (66.63 s) >>>> RESULTS : PASS 4 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 >>>> JOB TIME : 255.43 s >>>> >>> Yes, I've also seen similar runtimes in other environments... so it >>> looks like it depends a lot on the "dnf -y install numactl-devel". If >>> that can be removed, the tests would have much more predictable runtimes. >>>
