On Mon, Jan 15, 2024 at 09:13:01PM +0100, Greg KH wrote: > On Mon, Jan 15, 2024 at 01:42:53PM -0500, Kent Overstreet wrote: > > > That sounds more like a "(reproducible) tests don't exist" complaint > > > which is a different thing again to people going off and NIHing fancy > > > frameworks. > > > > No, it's a leadership/mentorship thing. > > > > And this is something that's always been lacking in kernel culture. > > Witness the kind of general grousing that goes on at maintainer summits; > > maintainers complain about being overworked and people not stepping up > > to help with the grungy responsibilities, while simultaneously we still > > very much have a "fuck off if you haven't proven yourself" attitude > > towards newcomers. Understandable given the historical realities (this > > shit is hard and the penalties of fucking up are high, so there does > > need to be a barrier to entry), but it's left us with some real gaps. > > > > We don't have enough a people in the senier engineer role who lay out > > designs and organise people to take on projects that are bigger than one > > single person can do, or that are necessary but not "fun". > > > > Tests and test infrastructure fall into the necessary but not fun > > category, so they languish. > > No, they fall into the "no company wants to pay someone to do the work" > category, so it doesn't get done. > > It's not a "leadership" issue, what is the "leadership" supposed to do > here, refuse to take any new changes unless someone ponys up and does > the infrastructure and testing work first? That's not going to fly, for > valid reasons. > > And as proof of this, we have had many real features, that benefit > everyone, called out as "please, companies, pay for this to be done, you > all want it, and so do we!" and yet, no one does it. One real example > is the RT work, it has a real roadmap, people to do the work, a tiny > price tag, yet almost no one sponsoring it. Yes, for that specific > issue it's slowly getting there and better, but it is one example of how > you view of this might not be all that correct. Well, what's so special about any of those features? What's special about the RT work? The list of features and enhancements we want is never ending. But good tools are important beacuse they affect the rate of everyday development; they're a multiplier on the money everone is spending on salaries. In everyday development, the rate at which we can run tests and verify the corectness of the code we're working on is more often than not _the_ limiting factor on rate of development. It's a particularly big deal for getting new people up to speed, and for work that crosses subsystems. > And you will see that we now have the infrastructure in places for this. > The great kunit testing framework, the kselftest framework, and the > stuff tying it all together is there. All it takes is people actually > using it to write their tests, which is slowly happening. > > So maybe, the "leadership" here is working, but in a nice organic way of > "wouldn't it be nice if you cleaned that out-of-tree unit test framework > up and get it merged" type of leadership, not mandates-from-on-high that > just don't work. So organic you might have missed it :) Things are moving in the right direction; the testing track at Plumber's was exciting to see. Kselftests is not there yet, though. Those tests could all be runnable with a single command - and _most_ of what's needed is there, the kernel config dependencies are listed out, but we're still lacking a testrunner. I've been trying to get someone interested in hooking them up to ktest (my ktest, not that other thing), so that we'd have one common testrunner for running anything that can be a VM test. Similarly with blktests, mmtests, et cetera. Having one common way of running all our functional VM tests, and a common collection of those tests would be a huge win for productivity because _way_ too many developers are still using slow ad hoc testing methods, and a good test runner (ktest) gets the edit/compile/test cycle down to < 1 minute, with the same tests framework for local development and automated testing in the big test cloud...
