On 9/30/22 06:11, Michał Winiarski wrote: > On Fri, Sep 30, 2022 at 02:50:43PM +0800, David Gow wrote: >> On Fri, Sep 30, 2022 at 6:33 AM Michał Winiarski >> <michal.winiarski@xxxxxxxxx> wrote: >>> >>> On Tue, Sep 27, 2022 at 07:12:06PM -0300, Maíra Canal wrote: >>>> The drm_test_dp_mst_sideband_msg_req_decode repeats the same test >>>> structure with different parameters. This could be better represented >>>> by parameterized tests, provided by KUnit. >>>> >>>> In order to convert the tests to parameterized tests, the test case for >>>> the client ID was changed: instead of using get_random_bytes to generate >>>> the client ID, the client ID is now hardcoded in the test case. >>> >>> Generally "random" usage is not incompatible with parameterized tests, we can >>> create parameterized tests that use random data. >>> The idea is to pass a function that generates the actual param (where we have a >>> pointer to function as one of the members in "params" struct). >>> >>> For example, see "random_dp_query_enc_client_id" usage here: >>> https://lore.kernel.org/dri-devel/20220117232259.180459-7-michal.winiarski@xxxxxxxxx/ Although it is possible, I don't see the benefit in this case to use the get_random_bytes instead of hardcoding. I believe it will only add more boilerplate to the tests. >>> >>> In this case, we just compare data going in with data going out (and the data >>> itself is not transformed in any way), so it doesn't really matter for coverage >>> and we can hardcode. >>> >>> -Michał >> >> FWIW, while the uses of randomness in DRM tests so far haven't >> concerned me much, I think we'll eventually want to have some way of >> ensuring the inputs to tests are deterministic. >> >> My thoughts are that (at some point) we'll add a kunit_random() >> function or similar, which will use a pseudorandom number generator >> which can be set to a deterministic seed before each test case. That >> way, there'd be a way to reproduce an error easily if it occurred. (Of >> course, there'd be a way of setting different or random seeds to >> preserve the extra coverage you'd otherwise get.) > > That's exactly what DRM tests do (well... most DRM tests, this one being the > exception, and those other tests also seem to have lost a printk with seed value > after being refactored into kunit). I will take a look at those lost printk in drm_mm_test and drm_buddy_test, as they are important to assure the reproducibility of the tests. > See the usage of DRM_RND_STATE in drm_mm_test and drm_buddy_test. > Having kunit_random() would definitely be useful and let us remove bunch of > boilerplate from the tests, but it doesn't prevent using reproducible random > data in existing tests. > >> I don't think this is something worth holding up or changing existing >> tests at the moment, but having tests behave deterministically is >> definitely desirable, so +1 to avoiding get_random_bytes() if it's not >> giving you any real benefit. > > Yeah - all I was refering to in my previous message was the wording of the > commit message. We're just removing it because it is desirable in this > particular case, not because of the fact that the test is now parameterized and > that's somehow preventing get_random_bytes() usage. I will send a v3 rewording the commit message to make it more clear. Best Regards, - Maíra Canal > > -Michał > >> We've also had a few requests in the past for being able to pass in a >> custom set of parameters from userspace, which opens up some other >> interesting possibilities, though it's not a priority at the moment. >> >> Cheers, >> -- David > >
