On Tue, Jan 16, 2024 at 11:41:25PM -0500, Kent Overstreet wrote: > On Mon, Jan 15, 2024 at 09:13:01PM +0100, Greg KH wrote: > > On Mon, Jan 15, 2024 at 01:42:53PM -0500, Kent Overstreet wrote: > > > > That sounds more like a "(reproducible) tests don't exist" complaint > > > > which is a different thing again to people going off and NIHing fancy > > > > frameworks. > > > > > > No, it's a leadership/mentorship thing. > > > > > > And this is something that's always been lacking in kernel culture. > > > Witness the kind of general grousing that goes on at maintainer summits; > > > maintainers complain about being overworked and people not stepping up > > > to help with the grungy responsibilities, while simultaneously we still > > > very much have a "fuck off if you haven't proven yourself" attitude > > > towards newcomers. Understandable given the historical realities (this > > > shit is hard and the penalties of fucking up are high, so there does > > > need to be a barrier to entry), but it's left us with some real gaps. > > > > > > We don't have enough a people in the senier engineer role who lay out > > > designs and organise people to take on projects that are bigger than one > > > single person can do, or that are necessary but not "fun". > > > > > > Tests and test infrastructure fall into the necessary but not fun > > > category, so they languish. > > > > No, they fall into the "no company wants to pay someone to do the work" > > category, so it doesn't get done. > > > > It's not a "leadership" issue, what is the "leadership" supposed to do > > here, refuse to take any new changes unless someone ponys up and does > > the infrastructure and testing work first? That's not going to fly, for > > valid reasons. > > > > And as proof of this, we have had many real features, that benefit > > everyone, called out as "please, companies, pay for this to be done, you > > all want it, and so do we!" and yet, no one does it. One real example > > is the RT work, it has a real roadmap, people to do the work, a tiny > > price tag, yet almost no one sponsoring it. Yes, for that specific > > issue it's slowly getting there and better, but it is one example of how > > you view of this might not be all that correct. > > Well, what's so special about any of those features? What's special > about the RT work? The list of features and enhancements we want is > never ending. Nothing is special about RT except it is a good example of the kernel "leadership" asking for help, and companies just ignoring us by not funding the work to be done that they themselves want to see happen because their own devices rely on it. > But good tools are important beacuse they affect the rate of everyday > development; they're a multiplier on the money everone is spending on > salaries. > > In everyday development, the rate at which we can run tests and verify > the corectness of the code we're working on is more often than not _the_ > limiting factor on rate of development. It's a particularly big deal for > getting new people up to speed, and for work that crosses subsystems. Agreed, I'm not objecting here at all. > > And you will see that we now have the infrastructure in places for this. > > The great kunit testing framework, the kselftest framework, and the > > stuff tying it all together is there. All it takes is people actually > > using it to write their tests, which is slowly happening. > > > > So maybe, the "leadership" here is working, but in a nice organic way of > > "wouldn't it be nice if you cleaned that out-of-tree unit test framework > > up and get it merged" type of leadership, not mandates-from-on-high that > > just don't work. So organic you might have missed it :) > > Things are moving in the right direction; the testing track at Plumber's > was exciting to see. > > Kselftests is not there yet, though. Those tests could all be runnable > with a single command - and _most_ of what's needed is there, the kernel > config dependencies are listed out, but we're still lacking a > testrunner. 'make kselftest' is a good start, it outputs in proper format that test runners can consume. We even have 'make rusttest' now too because "rust is special" for some odd reason :) And that should be all that the kernel needs to provide as test runners all work differently for various reasons, but if you want to help standardize on something, that's what kernelci is doing, I know they can always appreciate the help as well. > I've been trying to get someone interested in hooking them up to ktest > (my ktest, not that other thing), so that we'd have one common > testrunner for running anything that can be a VM test. Similarly with > blktests, mmtests, et cetera. Hey, that "other" ktest.pl is what I have been using for stable kernel test builds for years, it does work well for what it is designed for, and I know other developers also use it. > Having one common way of running all our functional VM tests, and a > common collection of those tests would be a huge win for productivity > because _way_ too many developers are still using slow ad hoc testing > methods, and a good test runner (ktest) gets the edit/compile/test cycle > down to < 1 minute, with the same tests framework for local development > and automated testing in the big test cloud... Agreed, and that's what kernelci is working to help provide. thanks, greg k-h
