On 7/24/24 15:40, Nícolas F. R. A. Prado wrote:
Introduce a new test to identify regressions causing devices to go
missing on the system.
For each bus and class on the system the test checks the number of
devices present against a reference file, which needs to have been
generated by the program at a previous point on a known-good kernel, and
if there are missing devices they are reported.
Can you elaborate on how to generate reference file? It isn't clear.
Signed-off-by: Nícolas F. R. A. Prado <nfraprado@xxxxxxxxxxxxx>
---
Hi,
Key points about this test:
* Goal: Identify regressions causing devices to go missing on the system
* Focus:
* Ease of maintenance: the reference file is generated programatically
* Minimum of false-positives: the script makes as few assumptions as possible
about the stability of device identifiers to ensure renames/refactors don't
trigger false-positives
* How it works: For each bus and class on the system the test checks the number
of devices present against a reference file, which needs to have been
generated by the program at a previous point on a known-good kernel, and if
there are missing devices they are reported.
* Comparison to other tests: It might be possible(*) to replace the discoverable
devices test [1] with this. The benefits of this test is that it's easier
to setup and maintain and has wider coverage of devices.
Additional detail:
* Having more devices on the running system than the reference does not cause a
failure, but a warning is printed in that case to suggest that the reference
be updated.
* Missing devices are detected per bus/class based on the number of devices.
When the test fails, the known metadata for each of the expected and detected
devices is printed and some simple similitarity comparison is done to suggest
the devices that are the most likely to be missing.
* The proposed place to store the generated reference files is the
'platform-test-parameters' repository in KernelCI [2].
How would a user run this on their systems - do they need to access
this repository in KernelCI?
This is what I see when I run the test on my system:
make -C tools/testing/selftests/devices/exist/ run_tests
make: Entering directory '/linux/linux_6.11/tools/testing/selftests/devices/exist'
TAP version 13
1..1
# timeout set to 45
# selftests: devices/exist: exist.py
# TAP version 13
# # No matching reference file found (tried './LENOVO,20XH005JUS.yaml')
# # Totals: pass:0 fail:0 xfail:0 xpass:0 skip:0 error:0
not ok 1 selftests: devices/exist: exist.py # exit=1
make: Leaving directory '/linux/linux_6.11/tools/testing/selftests/devices/exist'
thanks,
-- Shuah
