On Thu, 2016-11-10 at 21:44 -0500, Josh Boyer wrote: > On Thu, Nov 10, 2016 at 5:09 PM, Adam Williamson > <adamwill@xxxxxxxxxxxxxxxxx> wrote: > > On Thu, 2016-11-10 at 13:31 -0800, Chris Murphy wrote: > > > https://meetbot.fedoraproject.org/fedora-meeting-2/2016-11-10/f25-final-gono-go-meeting.2016-11-10-17.00.log.html > > > > > > > > > 17:10:26 <adamw> i can't really vote -1 on this under the current criteria unless someone tries on a newer mac and it works. but given https://www.happyassassin.net/testcase_stats/25/Installation/QA_Testcase_dualboot_with_OSX_Miscellaneous.html , i am entirely open to dropping the criterion on the basis of failure to test. > > > > > > > > > I think it's specious to drop the criterion on this basis. There are > > > plenty of other things that don't get tested and their criterion don't > > > get dropped. > > > > I am actually planning to propose precisely this. > > My sincere apologies for not being in the meeting. I could not attend > due to a conflict with a different meeting. > > I am somewhat befuddled at the decision to block the entire release > for this issue. We seem to have created a criteria under the premise > that "lots of people have Macs and want to run Linux/Fedora on them", > yet our empirical data seems to have shown a distinct lack of testing > that would bear that premise out. The premise under which the criterion was created was basically that "the Workstation technical specification included it". Here's the whole paper trail for the criterion: Workstation tech spec - https://fedoraproject.org/wiki/Workstation/Technical_Specification : "One aspect of storage configuration that will be needed is support for dual-boot setups (preserving preexisting Windows or OS X installations), since e.g. students may be required to run software on those platforms for their coursework." This led to: desktop@ discussion - https://lists.fedoraproject.org/pipermail/desktop/2014-June/009932.html : "1a. Does preserve preexisting include providing a working menu entry in the boot manager (e.g. in the GRUB menu)? 1b. Or is it sufficient to just preserve the installation data — meaning it's permissible for its bootability to be either non-obvious or broken?" Excursions and alarums, etc. etc. followed, till eventually mcatanzaro posted a... test@ list proposal - https://lists.fedoraproject.org/pipermail/test/2014-August/122496.html : "Since our Mac support is very poor and has no prospect of near term improvement -- in particular, we have concerns that running Fedora on a Mac has caused at least one Mac to overheat and die -- the consensus seems to be that dual booting with OS X should not be a requirement." There was an initial wording proposal, some more discussion, and then I posted a... final proposal - https://lists.fedoraproject.org/pipermail/test/2014-September/123012.html : "The installer must be able to install into free space alongside an existing OS X installation, install and configure a bootloader that will boot Fedora; if the boot menu presents OS X entries, they should boot OS X." which is the wording that remains. Note that it specifically does not require that OS X keeps working, but it *does* require that the Fedora installation actually boots. > I agree 100% that lots of people have Macs. I agree in part that > people want to run Linux/Fedora on them. I agree that a subset of > those that want to run Linux on a Mac also want to dual-boot OS X. > What I cannot get my head around though is how we've essentially made > a decision based on perceived marketing value of those target users at > the expense of every other platform we support. Our engineering and > testing resources are clearly not sufficient to cover this case. If > they are, then we have a fairly large communication problem > illustrated here. Or if that wasn't the reason, and it was simply > "because we have a criteria written down" that also seems somewhat > myopic. It's important to note that we may not have the full story on testing here. At the meeting I said that no-one had tested this throughout the cycle; this was based on the fact that no-one has filled out a result for the test in the wiki pages throughout the cycle. However, since the meeting, Chris Murphy has said in the bug that he *has* tested dual boot install during the Fedora 25 cycle and found that it worked. Given my analysis of the bug I find that a bit surprising, but I can see (I think) one possible scenario in which the install might possibly work (if there was an existing Fedora install on the system). So that might be what happened there. Anyway, point is, we *do* have some testing resources. However, I do mean to try and do a sweep through the criteria and test cases after the F25 release and try to highlight areas where test coverage is not sufficient (where 'sufficient' means we have at least two people and/or a bot regularly testing the thing), so we can re-consider these areas. It's also worth noting that there's a possible scenario where we could completely nerf the OS X install here. It's a fairly *unlikely* scenario - the user would have to manually disable journalling on the macOS partition - but it is *possible*. For instance, they may have disabled it so they could write to the partition in Linux, for some reason, and then forgotten to re-enable it. So this bug, at least in that case, can be even worse than just 'Fedora doesn't boot'. It can be 'the Fedora installer ate my OS X kernel'. I don't really *like* the thing where we suddenly decide at the end of the release cycle that we don't like a criterion that *just happens* to be standing in the way of release after all, and magically say it's not a criterion any more. It's crappy process. But we have done it before and probably will again, and I did float that possibility during the meeting. To be honest, I thought more people would want to go that way. But in fact it turned out that just about everyone at the meeting did not want to do that. > Please don't misunderstand me. I want this to work and I think it is > valuable. I also appreciate everyone pitching in at the last minute > and I'm sure it will get fixed. However, I think we really need to > take a strong look at what our Project can sustain, the value of the > distribution as a whole across all Editions and platforms, and the > resulting impacts of every possible slip. The conversation I read in > the IRC logs does not seem to reflect that, and despite people wishing > for "not hero-ing" that seems like exactly what happened and will > continue to happen, extending even into the release itself over a > major holiday. 'Hero-ing' has a specific if informal meaning in the context of the release process: it means fudging the process so that we don't declare the release go or no-go on Thursday (as we're supposed to) but try to do a super-fast patch, build, test cycle and then declare a go on Friday. We decided not to do that but rather to have a full week slip, hence we are not 'hero-ing'. I didn't really *have* to debug and fix the whole thing live during the meeting (though it was kinda important to try and analyze the precise impact of the bug at least), that was more just me not being able to stop something once I've started it. > We are technical people and bugs bother us and we want to fix them. > Yet, we need to make judgement calls on blockers based on reality and > overall benefit/cost of blocking the release, In a sense this is true, but in another sense, we do need to at least try to honor the process we set up to decide what's a release blocking bug in the first place. The "last minute" criticism applies just as strongly to "criticizing the criteria" as it does to "finding blockers": if it's bad that we find blockers at the last minute, it's also bad that very few people seem to bother proposing revisions to the release criteria at any point other than "right at the end of the cycle" and for any reason other than "because we want to SHIPIT RIGHT NOW". The criteria are not secret, the whole blocker process is *extensively* documented. You can go read the criteria at *any time* in the cycle and propose changes if you think they're wrong. Now, for instance, would be a perfect time to propose changes to the criteria for the F26 cycle. As opposed to "during the F26 Go/No-Go meeting". > not because we have a > known root-cause at the last minute and can probably fix it if we > slip. FWIW, I did not really get the sense at all that people were voting for it as a blocker purely on the grounds that "we can get a fix in at the last minute". A one week slip might feel short to you, I dunno, but I think most release process people think it's a really long time and hate it (hence the pressure to 'hero' sometimes). Having a *whole week* to fix a blocker and respin and test, frankly, feels like quite a lot of time, in a Fedora context. I think people just simply thought, the bug is very clearly a complete violation of the criterion, and for whatever reason decided they valued the criterion and didn't want to throw it out. Thus, blocker. > That way lies madness and we'll continue to have further > arguments about playing catch-up in the schedule with a shorter cycle > next release, etc. How can we ensure that we balance that with a > broader focus across the Project so that we don't continue to have > problems of our own making? Eh, I dunno. This really isn't terribly out of the ordinary. We've had far, far worse cycles than this. Assuming we ship next week I think that gives us a 2-week total slip for the F25 cycle, which would be one of our *best* results in terms of staying on schedule. We've had far messier ones before. A last-minute Final blocker discovery leading to a one-week slip for one or two bugs is really not that unusual at all. In an ideal world we'd like to avoid it, and we *have* been doing a gradually better job of this over time, I believe, but it's hard to get it perfect. We'll keep trying. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
