On Wed, May 08, 2019 at 05:58:49PM -0700, Frank Rowand wrote: > Hi Ted, > > On 5/7/19 10:22 AM, Theodore Ts'o wrote: > > On Tue, May 07, 2019 at 10:01:19AM +0200, Greg KH wrote: > Not very helpful to cut the text here, plus not explicitly indicating that > text was cut (yes, I know the ">>>" will be a clue for the careful reader), > losing the set up for my question. > > > >>> My understanding is that the intent of KUnit is to avoid booting a kernel on > >>> real hardware or in a virtual machine. That seems to be a matter of semantics > >>> to me because isn't invoking a UML Linux just running the Linux kernel in > >>> a different form of virtualization? > >>> > >>> So I do not understand why KUnit is an improvement over kselftest. > >>> > >>> It seems to me that KUnit is just another piece of infrastructure that I > >>> am going to have to be familiar with as a kernel developer. More overhead, > >>> more information to stuff into my tiny little brain. > >>> > >>> I would guess that some developers will focus on just one of the two test > >>> environments (and some will focus on both), splitting the development > >>> resources instead of pooling them on a common infrastructure. > >>> > >>> What am I missing? > >> > >> kselftest provides no in-kernel framework for testing kernel code > >> specifically. That should be what kunit provides, an "easy" way to > >> write in-kernel tests for things. > >> > >> Brendan, did I get it right? > > > > Yes, that's basically right. You don't *have* to use KUnit. It's > > If KUnit is added to the kernel, and a subsystem that I am submitting > code for has chosen to use KUnit instead of kselftest, then yes, I do > *have* to use KUnit if my submission needs to contain a test for the > code unless I want to convince the maintainer that somehow my case > is special and I prefer to use kselftest instead of KUnittest. > > > > supposed to be a simple way to run a large number of small tests that > > for specific small components in a system. > > kselftest also supports running a subset of tests. That subset of tests > can also be a large number of small tests. There is nothing inherent > in KUnit vs kselftest in this regard, as far as I am aware. > > > > For example, I currently use xfstests using KVM and GCE to test all of > > ext4. These tests require using multiple 5 GB and 20GB virtual disks, > > and it works by mounting ext4 file systems and exercising ext4 through > > the system call interfaces, using userspace tools such as fsstress, > > fsx, fio, etc. It requires time overhead to start the VM, create and > > allocate virtual disks, etc. For example, to run a single 3 seconds > > xfstest (generic/001), it requires full 10 seconds to run it via > > kvm-xfstests. > > > > > > KUnit is something else; it's specifically intended to allow you to > > create lightweight tests quickly and easily, and by reducing the > > effort needed to write and run unit tests, hopefully we'll have a lot > > more of them and thus improve kernel quality. > > The same is true of kselftest. You can create lightweight tests in > kselftest. > > > > As an example, I have a volunteer working on developing KUinit tests > > for ext4. We're going to start by testing the ext4 extent status > > tree. The source code is at fs/ext4/extent_status.c; it's > > approximately 1800 LOC. The Kunit tests for the extent status tree > > will exercise all of the corner cases for the various extent status > > tree functions --- e.g., ext4_es_insert_delayed_block(), > > ext4_es_remove_extent(), ext4_es_cache_extent(), etc. And it will do > > this in isolation without our needing to create a test file system or > > using a test block device. > > > > > Next we'll test the ext4 block allocator, again in isolation. To test > > the block allocator we will have to write "mock functions" which > > simulate reading allocation bitmaps from disk. Again, this will allow > > the test writer to explicitly construct corner cases and validate that > > the block allocator works as expected without having to reverese > > engineer file system data structures which will force a particular > > code path to be executed. > > This would be a difference, but mock functions do not exist in KUnit. > The KUnit test will call the real kernel function in the UML kernel. > > I think Brendan has indicated a desire to have mock functions in the > future. > > Brendan, do I understand that correctly? Oh, sorry, I missed this comment from earlier. Yes, you are correct. Function mocking is a feature I will be introducing in a follow up patchset (assuming this one gets merged of course ;-) ). Cheers! > -Frank > > > So this is why it's largely irrelevant to me that KUinit uses UML. In > > fact, it's a feature. We're not testing device drivers, or the > > scheduler, or anything else architecture-specific. UML is not about > > virtualization. What it's about in this context is allowing us to > > start running test code as quickly as possible. Booting KVM takes > > about 3-4 seconds, and this includes initializing virtio_scsi and > > other device drivers. If by using UML we can hold the amount of > > unnecessary kernel subsystem initialization down to the absolute > > minimum, and if it means that we can communicating to the test > > framework via a userspace "printf" from UML/KUnit code, as opposed to > > via a virtual serial port to KVM's virtual console, it all makes for > > lighter weight testing. > > > > Why did I go looking for a volunteer to write KUnit tests for ext4? > > Well, I have a plan to make some changes in restructing how ext4's > > write path works, in order to support things like copy-on-write, a > > more efficient delayed allocation system, etc. This will require > > making changes to the extent status tree, and by having unit tests for > > the extent status tree, we'll be able to detect any bugs that we might > > accidentally introduce in the es tree far more quickly than if we > > didn't have those tests available. Google has long found that having > > these sorts of unit tests is a real win for developer velocity for any > > non-trivial code module (or C++ class), even when you take into > > account the time it takes to create the unit tests. > > > > - Ted> > > P.S. Many thanks to Brendan for finding such a volunteer for me; the > > person in question is a SRE from Switzerland who is interested in > > getting involved with kernel testing, and this is going to be their > > 20% project. :-) > > > >
