On Tue, May 07, 2019 at 10:01:19AM +0200, Greg KH wrote: > > My understanding is that the intent of KUnit is to avoid booting a kernel on > > real hardware or in a virtual machine. That seems to be a matter of semantics > > to me because isn't invoking a UML Linux just running the Linux kernel in > > a different form of virtualization? > > > > So I do not understand why KUnit is an improvement over kselftest. > > > > It seems to me that KUnit is just another piece of infrastructure that I > > am going to have to be familiar with as a kernel developer. More overhead, > > more information to stuff into my tiny little brain. > > > > I would guess that some developers will focus on just one of the two test > > environments (and some will focus on both), splitting the development > > resources instead of pooling them on a common infrastructure. > > > > What am I missing? > > kselftest provides no in-kernel framework for testing kernel code > specifically. That should be what kunit provides, an "easy" way to > write in-kernel tests for things. > > Brendan, did I get it right? Yes, that's basically right. You don't *have* to use KUnit. It's supposed to be a simple way to run a large number of small tests that for specific small components in a system. For example, I currently use xfstests using KVM and GCE to test all of ext4. These tests require using multiple 5 GB and 20GB virtual disks, and it works by mounting ext4 file systems and exercising ext4 through the system call interfaces, using userspace tools such as fsstress, fsx, fio, etc. It requires time overhead to start the VM, create and allocate virtual disks, etc. For example, to run a single 3 seconds xfstest (generic/001), it requires full 10 seconds to run it via kvm-xfstests. KUnit is something else; it's specifically intended to allow you to create lightweight tests quickly and easily, and by reducing the effort needed to write and run unit tests, hopefully we'll have a lot more of them and thus improve kernel quality. As an example, I have a volunteer working on developing KUinit tests for ext4. We're going to start by testing the ext4 extent status tree. The source code is at fs/ext4/extent_status.c; it's approximately 1800 LOC. The Kunit tests for the extent status tree will exercise all of the corner cases for the various extent status tree functions --- e.g., ext4_es_insert_delayed_block(), ext4_es_remove_extent(), ext4_es_cache_extent(), etc. And it will do this in isolation without our needing to create a test file system or using a test block device. Next we'll test the ext4 block allocator, again in isolation. To test the block allocator we will have to write "mock functions" which simulate reading allocation bitmaps from disk. Again, this will allow the test writer to explicitly construct corner cases and validate that the block allocator works as expected without having to reverese engineer file system data structures which will force a particular code path to be executed. So this is why it's largely irrelevant to me that KUinit uses UML. In fact, it's a feature. We're not testing device drivers, or the scheduler, or anything else architecture-specific. UML is not about virtualization. What it's about in this context is allowing us to start running test code as quickly as possible. Booting KVM takes about 3-4 seconds, and this includes initializing virtio_scsi and other device drivers. If by using UML we can hold the amount of unnecessary kernel subsystem initialization down to the absolute minimum, and if it means that we can communicating to the test framework via a userspace "printf" from UML/KUnit code, as opposed to via a virtual serial port to KVM's virtual console, it all makes for lighter weight testing. Why did I go looking for a volunteer to write KUnit tests for ext4? Well, I have a plan to make some changes in restructing how ext4's write path works, in order to support things like copy-on-write, a more efficient delayed allocation system, etc. This will require making changes to the extent status tree, and by having unit tests for the extent status tree, we'll be able to detect any bugs that we might accidentally introduce in the es tree far more quickly than if we didn't have those tests available. Google has long found that having these sorts of unit tests is a real win for developer velocity for any non-trivial code module (or C++ class), even when you take into account the time it takes to create the unit tests. - Ted P.S. Many thanks to Brendan for finding such a volunteer for me; the person in question is a SRE from Switzerland who is interested in getting involved with kernel testing, and this is going to be their 20% project. :-)
