On Fri, Sep 30, 2022 at 02:50:43PM +0800, David Gow wrote: > On Fri, Sep 30, 2022 at 6:33 AM Michał Winiarski > <michal.winiarski@xxxxxxxxx> wrote: > > > > On Tue, Sep 27, 2022 at 07:12:06PM -0300, Maíra Canal wrote: > > > The drm_test_dp_mst_sideband_msg_req_decode repeats the same test > > > structure with different parameters. This could be better represented > > > by parameterized tests, provided by KUnit. > > > > > > In order to convert the tests to parameterized tests, the test case for > > > the client ID was changed: instead of using get_random_bytes to generate > > > the client ID, the client ID is now hardcoded in the test case. > > > > Generally "random" usage is not incompatible with parameterized tests, we can > > create parameterized tests that use random data. > > The idea is to pass a function that generates the actual param (where we have a > > pointer to function as one of the members in "params" struct). > > > > For example, see "random_dp_query_enc_client_id" usage here: > > https://lore.kernel.org/dri-devel/20220117232259.180459-7-michal.winiarski@xxxxxxxxx/ > > > > In this case, we just compare data going in with data going out (and the data > > itself is not transformed in any way), so it doesn't really matter for coverage > > and we can hardcode. > > > > -Michał > > FWIW, while the uses of randomness in DRM tests so far haven't > concerned me much, I think we'll eventually want to have some way of > ensuring the inputs to tests are deterministic. > > My thoughts are that (at some point) we'll add a kunit_random() > function or similar, which will use a pseudorandom number generator > which can be set to a deterministic seed before each test case. That > way, there'd be a way to reproduce an error easily if it occurred. (Of > course, there'd be a way of setting different or random seeds to > preserve the extra coverage you'd otherwise get.) That's exactly what DRM tests do (well... most DRM tests, this one being the exception, and those other tests also seem to have lost a printk with seed value after being refactored into kunit). See the usage of DRM_RND_STATE in drm_mm_test and drm_buddy_test. Having kunit_random() would definitely be useful and let us remove bunch of boilerplate from the tests, but it doesn't prevent using reproducible random data in existing tests. > I don't think this is something worth holding up or changing existing > tests at the moment, but having tests behave deterministically is > definitely desirable, so +1 to avoiding get_random_bytes() if it's not > giving you any real benefit. Yeah - all I was refering to in my previous message was the wording of the commit message. We're just removing it because it is desirable in this particular case, not because of the fact that the test is now parameterized and that's somehow preventing get_random_bytes() usage. -Michał > We've also had a few requests in the past for being able to pass in a > custom set of parameters from userspace, which opens up some other > interesting possibilities, though it's not a priority at the moment. > > Cheers, > -- David
