On Fri, Feb 07, 2025 at 06:16:38AM -0600, Dr. Greg wrote: > > The all powerful sub-system maintainer model works well if the big > technology companies can employ omniscient individuals in these roles, > but those types are a bit hard to come by. I'll let you in a secret. The maintainers are not "all-powerfui". We are the "thin blue line" that is trying to keep the code to be maintainable and high quality. Like most leaders of volunteer organization, whether it is the Internet Engineerint Task Force (the standards body for the Internet), we actually have very little power. We can not *command* people to work on retiring technical debt, or to improve testing infrastructure, or work on some particular feature that we'd very like for our users. All we can do is stop things from being accepted (either in our subsystem, or the imprimatur of the IETF). Hopefully, we're only stopping bad things from progressing, but that *really* is all we can actually do. One of the things which gets very frustrating from the maintainer's perspective is development teams that are only interested in their pet feature, and we *know*, through very bitter experience, that 95+% of the time, once the code is accepted, the engineers which contribute the code will disappear, never to be seen again. As a result, a very common dynamic is that maintainers will exercise the one and only power which they have --- which is to refuse to accept code until it is pretty much perfect --- since once we accept the code, we instantly lose all leverge, and the contributors will be disappear, and we will be left with the responsibility of cleanig up the mess. (And once there are users, we can't even rip out the code, since that would be a user-visible regression.) Occasionally, there is an accusation that different standards that might apply for people who are in the "gold old buys club". But what isn't appreciated, is that it is precisely because people who are long-term members of the community are trusted to stick around and will support code that the have sponsored. For example, Darrick Wong, who contributed ext4 metadata checksuming support before we moved on to become the XFS maintainer, is still reviewing and making contributions to code that he contributed many years to ext4. I've been known to submit fixes to test bugs for xfs-specific tests in xfstests that I discovered when running daily spinner tests on the linux git tree. Just in the past two weeks, I've spent 15+ hours working on cleaning up a casefold "security fix" that had nasty on-disk backwards compatibility issues. Part of it was that it was ext4 (although I discovered that there was also fsck.f2fs problems that had been overlooked), but the *other* part of why I spent so much time was that I sponsored the casefolding patches, and so I felt I certain moral responsibility to make sure the code was continued to be maintained. (And note, this has nothing to do with who employs me; the 15-20 hours that I spent working on creating the fix and the test scripts was purely on my own time, on the weekend, or late at night. The time that I spend doing this isn't reflected in my performance reviews, or promotion propects --- in fact, arguably, over the past 15 years, this has probably slowed down my promotion prospects; I've only been promoted once since joining said big tech company...) > Not sure what the fix is, from a project management perspective the > technology industry has never faced a challenge like this. The fork > model, which was the classic protection in open-source, doesn't work > at this scale. Well, here's my suggestion. Teams that want to get features, especially ones that might be potentially disruptive, into the tree, need to spend time becoming part of the *community*. This means that they need to participate in part of the joint effort to keep the code maintainable and high quaity --- even if it isn't part of their company's short-term goals, or directly related to their pet feature that they are trying to get upstream. This was the trust of my "Beyond Upstream First" presentation which I gave to the Linux Foundation Member Summit last fall[1]. [1] https://docs.google.com/presentation/d/11rMc8PBeyMItV6hv31OHSZ626_6FCZxjX6ZxEAehCpQ/edit#slide=id.p Now, I'll say that the Rust developers are making strides in the right direction here. Miguel has stood for election for the Technical Adviory Board, and has been contributing in various ways beyond just Rust for Linux. Ultimately, Cristoph's concern is that Rust is going to make life harder for maintainers because of particular build breaks getting in the way of the very limited bandwidth that Maintainers have (and again, a lot of the work that we do gets done on our own personal time; it's rare that even those maintainers employed by a "big technology company" have complete freedom to work on whatever they want; and they certainly don't have minions employed to do all of the grunt work to support code maintenance work, especially if we let crappy code slip past us in the name of "time to market" concerns.) So I think we'll get past this. It might take some more effort, and more trust building --- on both sides --- but I'm fairly optismistic that it will happen. It might not happe as *fast* as people might like; as Linus pointed out, it took ten years for the Clang compiler to be considered 100% fully supported --- and this was without needing to worrying about language issues, including an upstream language community which refuses to make any kind of backwards compatibility guarantees, and which is *actively* hostile to a second Rust compiler implementation (I suspect because it might limit their ability to make arbitrary backwards-incompatble language changes). > Obviously respect and open-mindedness to new ideas appears to be the > grease that makes all of this run smoothly. Unfortunately that seems > to be about as rare a commodity as omniscience in our industry. The other thing which is super rare is people and companies who care about tech debt cleanup, code maintainability, and code quality. Instead of complaining about maintainers for who are unreasonably caring about these things, when they are desparately under-resourced to do as good of a job as they industry demands, how about meeting us half-way and *helping* us with these sort of long-term code health issues? Maybe if you engage us as part of the community, we'll be a lot more open to adding changes that might increase the code maintenance burden? Cheers, - Ted
