On Sun, 2024-03-31 at 20:27 -0700, Adam Williamson wrote: > > > What WOULD have greatly reduced the impact of this attack: > > * NOT enabling updates-testing by default for Branched releases. > > This would only have helped by coincidence - the coincidence that the > compromise was discovered so quickly. Otherwise, we were busy trying to > get this update pushed stable as fast as possible, because rwmj wanted > that. If the compromise had happened to be caught a few days later, the > update would have been pushed stable anyway. Our gating and manual > testing processes are not designed to catch security issues, and > certainly do not reliably do that job. They did not in this case. So it > seems wrong, to me, to suggest we should rely on them to do it in > future. Thus I disagree that any possible security benefit gained by > doing this would outweigh the substantial disadvantage that we wouldn't > get testing of stuff promptly any more. > Having thought this over a bit, I think I should moderate my position here and make it a bit more nuanced. It occurs to me - maybe you don't agree, but this is how it looks to me - that, ironically, you and I usually argue the exact *opposite* side of this case, no? I argue in *favor* of somewhat-arbitrary delays to packages appearing in 'stable', and you argue *against* them. :D So, I don't think I can in full conscience justify being so categorical. So let me put more emphasis on the "difference between security and quality" aspect here. Let's look at the *intent* of the existing policies, here. Practically speaking, what our policies *achieve* is that for stable releases, updates-testing acts to *guard Fedora users' systems*. That is its function there. That's why it is not enabled by default, and we ask only people who want to contribute to testing to enable it, and understand that they are "taking a bullet for quality" by doing so. The idea is that by preventing updates reaching 'ordinary' users' systems before our testers have at least had an opportunity to try the packages out, we will catch at least some quality issues and prevent ordinary users from being affected by them. The process is specifically designed to achieve this. We have automated *quality* checks on packages in updates-testing (but almost nothing in the way of automated *security* checks). We have a whole process geared towards getting testers to test out the packages and report their findings - in *quality* terms. This is, to me, a job it's reasonable to ask a fairly broad range of folks to help with. It works reasonably well. We have done a reasonable job of implementing a process that lets a person without really specific training install a test update, experience a problem in it, and communicate that they encountered that problem in such a way that we are able to prevent the package reaching a wider swathe of users. It's not *perfect*, but it does provide some demonstrable value. Similarly on the automated testing side, both Fedora CI and openQA provide useful gating of some packages. It is by no means comprehensive, but it *is* useful. We catch bugs and prevent them reaching regular users. We don't catch *all* bugs, but we can reasonably claim to be purposefully and intentionally catching a reasonable number with this system. OK, that's for stable releases. What about development releases? It's key to realize that for development releases, the process looks very similar, but just that one change - updates-testing defaulting to off - *changes its entire purpose*, and this is by intent. For development releases, the purpose of updates-testing is **NOT** to protect 'ordinary users'. It is to protect **the Fedora development process**. We don't really intend for there to *be* "ordinary users" of Fedora development releases. We make it fairly clear that, by using a development release, you are taking yourself out of the "regular user" category and putting yourself in the "tester / developer" category. We do not feel such an obligation to provide users of development releases with quality-tested code. On the contrary, to a degree, they are supposed to be *helping* with the process of quality testing for the ultimate goal of making the *final* release good for "ordinary users". So if you're a user of a development release, we do not really feel an obligation to provide the same "safety net" for you that updates- testing provides for "regular users" of stable releases. Instead, the point of updates-testing in development releases is to let us prevent breakages in *the process of building the distribution*. We expect *all* users of development releases to help with this by flagging up when they see breakage, so we can prevent broken packages appearing in the buildroot or the compose process. *That* is why we have updates-testing for development releases. So...to return to the initial question, ultimately my argument is that it doesn't make much sense to try and use the existing updates-testing process as a "security gate". Not even for production releases, really, but *especially* not for development releases. That's *not what it's designed to do*. There is very little meaningful automated *security* testing going on for updates-testing. Nor do we have any kind of process to try and get informed *manual* security checks done on packages in updates-testing. Given that, I think the benefit of trying to use our existing updates-testing process to improve security would be very small. Now, if we were to actually *intentionally design a process* to do some meaningful security checks on 'candidate' packages, that would be an entirely different story. That might be something worth actually doing, and if we were to set out to do that, the question of whether it should be kinda merged into the existing 'updates-testing' process, or should use some different kinda gating approach, would be a reasonable one. My vote might be very different in that case. But I just don't think we would get much security value from *just* enabling updates-testing on Branched, given the background above. Sorry this got much longer and less clear. :D -- Adam Williamson (he/him/his) Fedora QA Fedora Chat: @adamwill:fedora.im | Mastodon: @adamw@xxxxxxxxxxxxx https://www.happyassassin.net -- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
