On Mon, Feb 10, 2025 at 3:20 PM Yury Norov <yury.norov@xxxxxxxxx> wrote: > > On Mon, Feb 10, 2025 at 11:35:48AM -0800, John Hubbard wrote: > > On 2/9/25 11:54 PM, Geert Uytterhoeven wrote: > > > On Sat, 8 Feb 2025 at 18:53, Yury Norov <yury.norov@xxxxxxxxx> wrote: > > > > On Fri, Feb 07, 2025 at 03:14:01PM -0500, Tamir Duberstein wrote: > > > > > On 7/27/24 12:35 AM, Shuah Khan wrote: > > ... > > > > > The crux of the argument seems to be that the config help text is taken > > > > > to describe the author's intent with the fragment "at boot". I think > > > > > > IMO, "at boot" is a misnomer, as most tests can be either builtin > > > or modular. > > > > Right. > > > > > > > > > KUNIT is disabled in defconfig, at least on x86_64. It is also disabled > > > > on my Ubuntu 24.04 machine. If I take your patches, I'll be unable to > > > > OK so I just bought a shiny new test machine, and installed one of the > > big name distros on it, hoping they've moved ahead and bought into the kunit > > story... > > > > $ grep KUNIT /boot/config-6.8.0-52-generic > > # CONFIG_KUNIT is not set > > > > ...gagghh! No such luck. One more data point, in support of Yuri's complaint. :) > > > > > > > > I think distros should start setting CONFIG_KUNIT=m. > > > > Yes they should! kunit really does have important advantages for many use > > cases, including bitmaps here, and "CONFIG_KUNIT is not set" is the main > > obstacle. > > Hi John, Geert, Tamir, > > Can you please explain in details which advantages KUNIT brings to > the test_bitmap.c, find_bit_benchmark.c and other low-level tests? I can try, but I'm not the expert, and David Gow can probably elaborate further. As I understand it the main benefit of KUnit is standardization and speed (and standardization _is_ speed). KUnit makes it very easy for me, a person who has not previously contributed to any of the bitmap code, to run those tests, and it requires zero configuration, it all just works. It's basically just `tools/testing/kunit/kunit.py run bitmap`, and I get the test results in a human-readable format. The same benefit applies on the author side: test facilities are standardized, so once you get to know the tools, all the tests start to look the same: you can jump in and contribute without having to first learn the so-called local "testing framework". The important part is that this all applies to ~all other tests written in KUnit. I can even run them *all* trivially: `tools/testing/kunit/kunit.py run`. Anecdotally I've also noticed there are bots running those KUnit tests e.g. see https://lore.kernel.org/all/20250207-blackholedev-kunit-convert-v1-1-8ef0dc1ff881@xxxxxxxxx/ where a test I converted was immediately flagged by a robot as having dubious type coercion. None of these are must-haves, they are just (to me) a nice way to make the kernel more approachable for new contributors. As for pure benchmarks like find_bit_benchmark.c and test_bitmap.c::`test_bitmap_{read,write}_perf` specifically: I believe the benefits are super limited or even negative: AFAIK KUnit is designed to generally suppress output (in the userspace reporter, not in the kernel) unless a test fails, so I wouldn't hurry to use it for these. > I'm not strongly against moving under KUNIT's hat, but I do: > - respect commitment of my contributors, so I don't want to wipe git > history for no serious reason; > - respect time of my testers, so no extra dependencies; > - respect time of reviewers. These are valid concerns. Certainly the testing case is the most compelling and folks are clearly interested in lowering those barriers. I don't have any influence in this area, but I am grateful to John for starting the conversation. As I mentioned in the previous thread: I think we could keep `test_bitmap_{read,write}_perf` in test_bitmap.c and get the best of both worlds. WDYT? > Tamir, > > If it comes to v2, can you please begin your series with an exhaustive > and clear answer to the following questions: > - What do the tests miss now? > - What do _you_ need from the tests? Describe your test scenario. > - How exactly KUNIT helps you testing bitmaps and friends better? > - Is there a way to meet your needs with a less invasive approach, > particularly without run-time dependencies? Hopefully I've answered these above. I can include some of it in a v2, but perhaps the general pitch for KUnit is better placed in documentation or slides from a conference? Cheers. Tamir
