On Tue, May 03, 2022 at 05:14:53PM -0700, Adam Williamson wrote: > but these bugs weren't discovered at that time. This is likely because > of your idea 2: "basic functionality" is a bit up for debate. You can > take an extremely minimalist approach to this (run the app, click a > couple of buttons, say it's OK if nothing explodes and no babies are > eaten) or a slightly more maximalist one (run the app, and actually try > and do something useful with it). In this case, before Final, when we > ran this test we mostly did the minimalist thing. At Final RC stage, we > went a bit more maximalist. Yeah, the tension: we're moving towards more polish, but the available time to fix gets smaller. And it's not like fixing the things that are at that "more maximalist" end takes less time. > > As a barometer for "embarrasing", you can imagine me trying to explain > > the issue to a tech reporter, and weigh how awkward I will feel saying > > "this is fine" vs. how I will feel explaining that we delayed the whole > > release for that same issue. > > I agree with the sentiment but I'm not sure about the phrasing. It's > extremely subjective, and I don't think subjective criteria work very > well. It also wouldn't necessarily "solve the problem": the bugs we Yesssss, sorry. That was meant to be a sentiment-based way to hopefully convey what I'm trying to say, not a suggested phrasing. > discovered this time really are pretty embarrassing, honestly. Imagine > doing a keynote showing off the sleek default apps included in GNOME, > running them, and trying to do...well...actually anything at all useful > with them. It wouldn't go very well. This is why I don't do live demos. :) > > 3. Problems found which are not regressions should not be blockers. We're > > just hurting ourselves when we make this our forcing function to get > > something fixed. > This is one of those things that sounds great until it doesn't. I can't > quite recall any specifics, but I definitely think there have been > cases recently where we've had strong support for a bug that is not a > regression to be a blocker. Sometimes a bug is just really bad but we > didn't see it before; even if you can make a wonk-y argument that > there's no point making it a blocker because if we do, it just means > the previous release stays as the "current" one for longer and it has > the same bug, in practice it's hard to hold that line when now there's > a bug report that everyone can see that says how badly this thing is > broken. I feel like I can defend that line pretty confortably. Send 'em to me. :) In seriousness, I think we look to blocking the relase because it's such a big signal. The train comes to a stop and everyone can see. But it's a big signal _because it's painful_, and pain isn't the best motivator. And while that pain does sometimes land on people who can do something, even if that _would_ help, it's indiscriminate. In the metaphor, people can't get to where they are going, the tracks are blocked, etc., etc., and none of that has anything to do with the issue to fix. (I'm tempted to keep running with the train metaphor, but reluctantly will stop.) This is the idea of Prioritized Bugs, and I think it's relatively successful. It's another way to make sure we're focusing on serious problems. Making that more visible could help. > > I don't know how to phrase this in a way that doesn't make Adam sad with > > me, but maybe something like: Desktop application blockers discovered > > during the final freeze are automatically waived unless the relevant Spin > > or Edition team decides otherwise. > You're right that this makes me sad. I don't think it's a good > approach. I think it's an attempt to solve a problem that I would maybe > look at differently, and which we're currently discussing in a ticket: > https://pagure.io/fedora-workstation/issue/304 > > For me, the big question your mail never quite arrived at is, *why* did > these bugs show up in Fedora 36 Final RCs at all? They really should > not have done. They are bugs in applications that are, supposedly, core > parts of upstream GNOME. They appear in 42.0 releases of those > applications - i.e. in releases of those applications that are, > according to upstream's versioning scheme, stable releases for public > consumption. > > Stable releases of core components of a major desktop should never > contain bugs like "deleting contacts sometimes doesn't work" or "you > can't add photos to an album in the Photos application because the > dialog where you're supposed to do it is completely broken and the list > entries multiply like rabbits who've been dosed up on viagra". > Distribution validation testing is not *for* finding bugs like this. > > The reason we're all instinctively feeling that something is Not Right > here is that something *is* Not Right, but the big thing that's Not > Right is the upstream GNOME release process. It's not (IMHO) any part > of the Fedora process - there are things we could tighten up there, but > I see those as subsidiary problems. > > It's Not Right that GNOME can ship a 42.0 release containing entirely > broken applications. That should not be happening. It's not something > we should have to design our Fedora distribution validation testing > process to fix. So, yeah, I think there's something to do this. There were a lot of last-minute-feeling kinds of changes not just to applications but also with, like the dark-mode desktop background stuff. Shouldn't it be rare that we're discovering general "this doesn't work" issues _that aren't Fedora Linux specific_ in Fedora QA? Also, I think some of the time-tension (from the start of this message) stems from an assumption that isn't working out: "this app might be rough at beta, but will more polished by the time we're doing final validation, because upstream is working on that". Maybe this is something we can bring to GUADEC to work together to improve. (Virtual at the end of July.) > The criterion and the test case were written with the unspoken, but > IMHO reasonable, assumption that in general we could trust desktops to > provide us with more-or-less-working software. They were never written > with the intent of finding this kind of bug. The scenario I was > envisioning when we wrote them was "oh, we accidentally packaged a > broken development version of app X" or "we're still including random > third-party app Y in the Workstation edition but it's not been > maintained for years and doesn't work any more, let's throw it out". I > was never envisaging having to deal with "GNOME shipped us completely > broken applications in a stable release". I don't think our goal should > be to design a release validation process that deals with that, because > *that shouldn't happen*. This all makes sense. I still think my suggestion might help even then, for the same reason I gave to Kamil: it flips the default pressure, moves it away from QA, away from the threat of stopping the whole train and motivation-through-guilt. > > 5. Okay, and... bigger: we should aim for more approaches which let us > > decouple as much as possible from the Release. (My grand hope is that we > > can release every deliverable on its own schedule, but I also understand > > the _highly aspirational_ nature of that idea. But...) What if we could > > just easily ship GNOME Photos from GNOME 41 until a fix is found in the > > updated one? > > I mean, we maybe could. I dunno if we tried that yet. There's not > necessarily anything in the current rules/policies that precludes this, > AFAIR. For small things, it might just be a matter of shipping an old RPM. Maybe with the dreaded Epoch. But in a lot of cases it probably means we are gonna need a bigger [container]. -- Matthew Miller <mattdm@xxxxxxxxxxxxxxxxx> Fedora Project Leader _______________________________________________ desktop mailing list -- desktop@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to desktop-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/desktop@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
