On Thu, Jun 27, 2024 at 02:25:36PM -0400, Liam R. Howlett wrote: > * Kees Cook <kees@xxxxxxxxxx> [240627 12:58]: > > On Thu, Jun 27, 2024 at 11:39:32AM +0100, Lorenzo Stoakes wrote: > > > Establish a new userland VMA unit testing implementation under > > > tools/testing which utilises existing logic providing maple tree support in > > > userland utilising the now-shared code previously exclusive to radix tree > > > testing. > > > > > > This provides fundamental VMA operations whose API is defined in mm/vma.h, > > > while stubbing out superfluous functionality. > > > > > > This exists as a proof-of-concept, with the test implementation functional > > > and sufficient to allow userland compilation of vma.c, but containing only > > > cursory tests to demonstrate basic functionality. > > > > Interesting! Why do you want to have this in userspace instead of just > > wiring up what you have here to KUnit so testing can be performed by > > existing CI systems that are running all the KUnit tests? > > The primary reason we did the maple tree testing in userspace was for > speed of testing. We don't need to build the kernel, but a subset of > APIs. Debugging problems is also much quicker since we can instrument > and rebuild, iterate down faster. Tracing every call to the maple tree > on boot alone is massive. > > It is also difficult to verify the vma correctness without exposing APIs > we don't want exported (or, I guess, parse proc files..). On my side, I > have a module for testing the overall interface while I have more tests > on the userspace side that poke and prod on internal states, and > userspace rcu testing is possible. I expect the same issues on the vma > side. > > Adding tests can also be made very efficient with tracepoints dumping > something to add to an array, for example. > > Finally, you have ultimate control on what other functions return (or > do) - so you can fail allocations to test error paths, for example. Or > set the external function to fail after N allocations. This comes in > handy when a syzbot reports a failed allocation at line X caused a > crash. > > This has worked out phenomenally on the maple tree side. I've been able > to record boot failures and import them, syzbot tests, and fuzzer tests. > The result is a huge list of tests that allowed me to rewrite my node > replacement algorithm and have it just work, once it passed the > collected tests. > > I haven't used kunit as much as I have userspace testing, so I cannot > say if all of these points are not possible, but I didn't see a way to > test races like I do with rcu in userspace. > > Thanks, > Liam Liam's response is excellent, and obviously I agree wholeheartedly. Additionally, I'm not really experienced with kunit, but surely it's implemented as a kernel module somehow? If so, these interfaces are largely not exported so it wouldn't be functional as a unit test. And also as Liam says, it'd be very difficult to test this stuff _in_ the kernel without unwanted side-effects triggering and it'd be very difficult to isolate or mock components we don't want to play a role (for instance - rlimits that we might not be able to control). But overall (again as Liam says) the performance benefit, flexibility and ability to recreate things at a whim are huge. And the fact maple tree (which forms a HUGE part of these VMA operations) and related radix tree and other shims/stubs already exist means that it wasn't anywhere near as huge a task to implement this as it would be otherwise.
