On Tue, Sep 24, 2024 at 06:56:26PM GMT, Jani Nikula wrote: > On Tue, 24 Sep 2024, Guenter Roeck <linux@xxxxxxxxxxxx> wrote: > >>>> On Tue, Sep 24, 2024 at 12:06:28PM GMT, Simona Vetter wrote: > >>>>> Yeah I think long-term we might want a kunit framework so that we can > >>>>> catch dmesg warnings we expect and test for those, without those warnings > >>>>> actually going to dmesg. Similar to how the lockdep tests also reroute > >>>>> locking validation, so that the expected positive tests don't wreak > >>>>> lockdep for real. > >>>>> > >>>>> But until that exists, we can't have tests that splat in dmesg when they > >>>>> work as intended. > > > > FWIW, that is arguable. More and more tests are added which do add such splats, > > and I don't see any hesitance by developers to adding more. So far I counted > > two alone in this commit window, and that does not include new splats from > > tests which I had already disabled. I simply disable those tests or don't > > enable them in the first place if they are new. I did the same with the drm > > unit tests due to the splats generated by the scaling unit tests, so any > > additional drm unit test splats don't make a difference for me since the > > tests are already disabled. > > What's the point of having unit tests that CI systems routinely have to > filter out of test runs? Or filter warnings generated by the tests, > potentially missing new warnings. Who is going to run the tests if the > existing CI systems choose to ignore them? If we turn this argument around, that means we can't write unit test for code that will create a warning. IMO, this creates a bad incentive, and saying that any capable CI system should reject them is certainly opiniated. Maxime
Attachment:
signature.asc
Description: PGP signature
