On Fri, Nov 25, 2016 at 7:26 AM, Bastien Nocera <bnocera@xxxxxxxxxx> wrote: > > But if the installer is (completely) broken, it might as well be dropped. > Alas, it's not completely broken. Unwieldy, perhaps. It's kinda hard to argue that the installer needs to be this complicated, that Fedora (mainly QA) has to put in so much effort into the installer each cycle testing for bugs in new features and also regressions. > >> 2. The Fedora QA group has 1 mac mini which is very old and is only >> used for total install and not dual boot. It would not have found this >> issue. > > The testing should be switched to be a dual-boot test, as it's what > Mac users are more likely to be using (and also a necessity for firmware > upgrades). The firmware angle is a very good point. Ostensibly the firmware could be downloaded, extracted and placed on the FAT32 volume (that Fedora doesn't use) in the proper location, and NVRAM pointed to that firmware file, and the firmware will consume it and thereby update the firmware, without needing macOS. But the path of least resistance is having a minimal macOS installed, even if it's not otherwise used. > >> The Fedora QA group also has no one using Mac hardware day to >> day. > > This isn't a problem. There are people using Macs day-to-day, and they report > bugs. The problem here, and I can't emphasise this enough, this problem is > a systemic problem with the installer QA, specifically. Automated testing is problematic because it's not really possible to virtualize Macs. The various guides for using kvm to virtualize macOS actually use BIOS firmware, not UEFI, and one of several hacked up bootloaders (e.g. Chameleon) is use to fake out XNU into booting on non-Apple hardware. On these systems, there is no EFI System partition of either the FAT32 or HFS+ variety, so it wouldn't test the installer behavior we need to test. I don't know what work would be needed to get Beaker to help do baremetal installations of Fedora on Mac hardware, and if that is sufficiently not fakey that it'd be a valid test. So for now it's a manual test. And as the installer regressions can come at any time, it's a lot of manual testing, and in this particular case it's not enough to just test if the install media boots, and the installer launches, an installation writing to the drive must be done. > Once the machine is installed, it's usually fairly straight forward to > update packages, downgrade them, and fix hardware specific problems as long > as the device can be booted, and a sufficient amount of hardware is working. > > The installer not working, especially when it's a last minute problem, > it becomes a blocker. Do we need a different schedule for installer > development? I'm pretty confident this bug was in Rawhide before F25 branch. I'm also pretty confident, looking at the Mac I use for testing, that the small HFS+ volume Anaconda creates and uses as an EFI system partition only on Macs, was already present for each of the tests I did, and that's why I never ran into the problem. It looks virtually certain I did not in fact have completely free space for the installations, everything but the three Apple creates partitions (FAT32 ESP, macOS main volume, macOS recovery volume), and the Fedora HFS+ ESP were deleted, but the presence of that one Fedora HFS+ ESP was seemingly enough for me to not hit the bug. So it can't be proven that this is a case of insufficient testing; it's a case of testing that was imprecise or was just not thorough. Possibly there should be a single Mac test case that asks for a very explicit setup, with instructions on how to get to that state using a path of least resistance, to just test the most minimal "get from A to B" path, instead of 2-3 or more ways to get there. So at least we have the most common and simple path tested, and flagged earlier on in the release cycle that it hasn't been tested. > It's better, but I don't think that the problem lies in the QA team, > although some things could be done better there. > > The main problem to me seems to be that the installer sees too little > testing, or too little testing when big changes occur, or not a wide > enough breadth of testing scenarios, at the development stage. I had no idea macefi stuff was touched. If I'd known this, I'd probably have been more skeptical and vigilant how I was testing, instead of becoming more trusting of the installer (and more complacent of its, in my opinion, inevitable betrayal) - but that's rather circular logic on my part. -- Chris Murphy _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
