Hi David, On Tue, Jul 25, 2023 at 05:54:32PM +0800, David Gow wrote: > On Tue, 25 Jul 2023 at 16:38, Maxime Ripard <mripard@xxxxxxxxxx> wrote: > > On Thu, Apr 27, 2023 at 03:14:39PM +0200, Maxime Ripard wrote: > > > Hi, > > > > > > On Fri, Jul 08, 2022 at 05:30:52PM -0300, Maíra Canal wrote: > > > > From: Arthur Grillo <arthur.grillo@xxxxxx> > > > > > > > > Considering the current adoption of the KUnit framework, convert the > > > > DRM mm selftest to the KUnit API. > > > > > > > > Signed-off-by: Arthur Grillo <arthur.grillo@xxxxxx> > > > > Tested-by: David Gow <davidgow@xxxxxxxxxx> > > > > Acked-by: Daniel Latypov <dlatypov@xxxxxxxxxx> > > > > Reviewed-by: Javier Martinez Canillas <javierm@xxxxxxxxxx> > > > > Signed-off-by: Maíra Canal <maira.canal@xxxxxx> > > > > > > I'm very late to the party, but I'd like to discuss that patch some more. > > > > > > Two tests (drm_test_mm_reserve, drm_test_mm_insert) in it take a super > > > long time to run (about 30s each on my machine). > > > > > > If we run all the DRM tests and VC4 tests, each of those two are longer > > > to run than all the ~300 tests combined. About 100 times longer. > > > > > > I don't think that running for so long is reasonable, and for multiple > > > reasons: > > > > > > - While I don't know drm_mm well, it doesn't look like any of those > > > tests do something that really should take this long. I'm especially > > > skeptical about the fact that we test each operation 8192 times by > > > default. > > > > > > - It makes using kunit more tedious than it should be. Like I said, on > > > a very capable machine, running the all the DRM and VC4 tests takes > > > about 50s with those two tests, ~0.4s without. > > > > > > - The corollary is that it will get in the way of people that really > > > want to use kunit will just remove those tests before doing so, > > > defeating the original intent. > > > > > > > > > I understand that it came from selftests initially, but I think we > > > should rewrite the tests entirely to have smaller, faster tests. It's > > > not clear to me why those tests are as complicated as they are though. > > > > > > Also, going forward we should probably put disencourage tests running > > > that long. Could Kunit timeout/warn after a while if a test is taking > > > more than X seconds to run? > > > > I'd still like to address this. We spend ~90% of the DRM kunit tests > > execution time executing those two tests, which doesn't seem like a > > reasonable thing to do. > > FWIW, KUnit is going to add a "speed" attribute for tests, so that > it's easy to skip tests which are slow: > https://lore.kernel.org/linux-kselftest/20230724162834.1354164-3-rmoar@xxxxxxxxxx/T/#u > > This would allow the slow tests to be marked using KUNIT_CASE_SLOW(), > and then be run via kunit.py --filter "speed>slow". > > It obviously doesn't make the tests themselves any faster, but could > at least make it possible to run only the fast tests during > development, and the full, slower set before sending the patches out > (or in CI), for example. That's awesome, thanks Speaking of which, should we detect in kunit.py tests that should be marked as (super) slow but aren't? Maxime
Attachment:
signature.asc
Description: PGP signature
