On Wed, Oct 10, 2012 at 01:18:37PM -0700, Adam Williamson wrote: > On Wed, 2012-10-10 at 12:17 -0400, David Cantrell wrote: > > > You're not understanding what I was pointing out. The blocker criteria > > between alpha and beta should be more open than the blocker criteria between > > beta and rc. The idea is that we start accepting fewer bugs as blockers as > > we get closer to RC. Every problem encountered can be evaluated along the > > lines of: > > > > 1) Who is impacted? > > 2) Is there a workaround? > > 3) Is the workaround documented? > > 4) Is the problem in the standard install path? > > > > And so on. I'm not saying we should compromise on release quality or > > anything like that, but just start to ask more and more questions when > > proposed blockers show up late. Is it really a blocker or not. > > > Again, that's not what I'm saying we should do. But if there's a traceback > > that appears when someone is doing a LUKS installation to USB attached > > storage that also has a pre-existing NTFS volume for "music"....that doesn't > > sound like a really suport important use case late in the cycle. Maybe we > > should consider documenting that as a limitation with or without a > > workaround for that release and slating it for the next release. > > I have a better understanding of where you're coming from now, and > particularly that you're basing this on a RHEL model (I didn't know this > was how RHEL did blocker tracking). That really shouldn't come as a surprise. RHEL drives our work. > But I still disagree. :) > > I don't think my disagreement is necessarily a problem for you, > though...let me see if I can put it a different way: > > Take the example you just gave. My position isn't 'we should take that > bug as a blocker both four weeks out and one week out'. My position is > 'we shouldn't take it as a blocker four weeks out _or_ one week out'. > > I'd phrase the problem more as 'there is a tendency in blocker review to > accept things too casually early on'. We should only accept things as > blockers that we really believe we should hold the release up for - even > if it's six weeks until release and we know the bug is going to be fixed > in two days, we should not accept it as a blocker unless it passes the > 'if today was go/no-go and this was the last bug, would we block the > release' smell test. > > Rather than throw almost everything on the blocker list when it's early > and get more rigorous later on, I'd rather we just be rigorous the whole > time. The blocker list isn't a work list (using the blocker list as a > 'todo list' / reminder list is a tendency I've noticed in both RHEL and > Fedora, and something I think we should fight), it is a list of bugs > that must be fixed for the release to go out. No more, no less. If we > wouldn't really block the release for a bug, it doesn't belong on the > list at any point in time. I think we've been making a good effort to > improve this in recent releases; it's not just an aspiration. That's fine, but how is that not a to do list? > So if there are concrete cases where you think bugs have been accepted > as blockers that you wouldn't be comfortable taking as blockers the day > before release, that's _always a problem_, and please highlight those > bugs to us. I don't think it's 'not a problem' if we accepted them as > blockers a long time before release and they got fixed before any chance > of causing a slip, it's still a problem. We should stick firmly to our > tight, minimal definition of what constitutes a blocker, regardless of > the point in the release schedule. The big difference here and what we see on the RHEL side is that RHEL allows for blocker bugs to be re-evaluated and booted if a later workaround or some other acceptable solution besides a bug fix is found. It becomes a schedule thing, which I am routinely told is very important for Fedora. If something marked as a blocker early on is still not fixed through several test releases, does it really matter anymore for the release? If not, why not boot it off the blocker list? > > We do want to make a good release and keep the users happy, but we need to > > also remember that developers are people too and not just meat grinders for > > bug reports. > > Sure, hence the need to keep the criteria realistic. > > > 1) Adjustments to the blocker acceptance criteria that I explained. > > 2) Acknowledgement that concurrent releases are always in play for the > > installer team. Generally two RHEL releases and one Fedora release. > > RHEL gets priority. > > I would prefer to phrase 2) the way I already did: Fedora should ensure > its release requirements are realistic in the context of the resources > available. If the 'resources available' are to an extent defined by > other commitments on the part of the anaconda team, then so be it, but > I'd rather consider it in that context, rather than haul RHEL > specifically into a Fedora context. Call it want you want, but we all know it's RHEL. > > What I'm asking for is blocker acceptance criteria changes that are more > > realistic given remaining time and other commitments. Or dump the > > time/schedule requirement and just say we'll call it done when it's done. > > The schedule / blocker process balance is a bit of a delicate one, yeah. > In practice we start getting a lot of pressure if the blocker process > results in delays of several weeks. I think I'd like to look at this > way, though: any time that following the release validation process > properly results in a long slip, _that means something is wrong_. It > usually means either that the release criteria are excessive, or it > means that something went wrong in the feature process. I believe the > feature process as it stands is not adequate in ensuring proposed > features are really viable on the Fedora release schedule, and FESCo has > also not always been assertive enough in dropping or delaying features > that are clearly not meeting the schedule; I think sometimes problems > are due to that, rather than to over-ambitious release requirements. I'll agree with that. I've had that complaint for a long time, ever since we merged Core and Extras. As we have both already stated and agreed on, the installer is unique. Our development process is so very much tied to the Fedora release schedule. There is no easy way for us to develop something large like a new installer and not have it tied to a release. This became harder for us when we stopped doing rawhide nightly composes many years ago (and I have no idea if those are happening again these days). People don't go and download a new installer to try it out. You download a release of something and try installing it. Our only way to get that to users is to follow the Fedora release schedule, unless we want to start making our own distribution. I know what you're getting at and it's that newui was a huge feature and that's made our lives hard. Unfortunately, the Fedora release schedule is such that work like this is always painful for us. > > > To give a competing example, in anaconda 18.13 there is a bug in the new > > > partitioning process - I call it 'guided partitioning', the dialog which > > > attempts to help you free up space on a full disk, by deleting or > > > shrinking partitions - which causes it to crash when trying to delete > > > partitions. But if you go into the 'custom partitioning' interface you > > > can successfully delete partitions. So by your principle, we would not > > > take that bug as a release blocker. I don't think that would be a good > > > decision: we should not release an installer which crashes when you try > > > to follow the path you're guided to, for freeing up space to install the > > > operating system. 'Don't do what the installer recommends you to do, > > > instead go into this advanced process that's supposed to be for experts' > > > is a workaround, and hence satisfies your requirement for not-a-blocker, > > > but I really don't think it's a good story to tell people in the case of > > > such a critical bit of functionality. > > > > Crashes through the standard install path would of course be a blocker. Why > > would you think I would want otherwise? You're reading my proposal way too > > literally. > > I'm afraid I have a tendency to read things literally, because processes > and policies are written down, and if the person who wrote them is eaten > by a raptor the only way you can reliably read them is 'literally' :) It > makes drafting things a pain in the ass, but I think it makes what you > wind up with a stronger process/policy. Your proposal didn't have escape > clauses or exceptions, it simply read "Installer blockers should only be > granted when there is no other way to accomplish the same task during > installation." That's a very clear and unequivocal statement. We could > not adopt it into the blocker evaluation process as it's written. > > As I said in my initial mail, I think in practice we mostly already do > what you really want here, but we do need to document it more clearly. > As the documentation stands, it really doesn't cover the question of > workarounds much at all. > > (in reference to my points about development planning) > > > This is nothing new to us, we've been through it many times before. > > But it doesn't mean we enjoy it. Unless we want to put anaconda in a > > permanent maintenance mode, it means we will always be playing this > > game. > > I don't think this is entirely correct. I don't want to get into > specifics because I'm not plugged into the development planning process > enough to make accurate criticisms, and I don't want to teach anyone's > grandma to suck eggs, but to talk generally: > > Of course any project which involves major development will be more > susceptible to bugs than any project which is solely in maintenance > mode. But the _magnitude_ of the effect is certainly something that can > be addressed by good planning. I don't want to harp on this too much as > I'm sure it's something you're aware of, but we can certainly aim to be > realistic and rigorous in our planning and timing of major new features. > For instance, trying to ensure major developments are feature complete > at Alpha time, so we're not still writing stuff during the Beta cycle. > FESCo requiring much more detailed planning for feature submissions and > being more decisive about rejecting/delaying features that are clearly > lagging at Alpha stage. Stuff like that. See above. > > I didn't read this paragraph. This is a lot of text, but I've skimmed it. > > What I think we should consider is whether Fedora should have a separate > > entity or group that accepts blockers. Right now, that responsibility is > > largely falling on your team. > > This isn't really the intent. The blocker meeting SOP - > https://fedoraproject.org/wiki/QA:SOP_Blocker_Bug_Meeting - reads: > > "Blocker Bug Meetings are not owned by any one team in Fedora. They are > a collaborative effort between Release Engineering, Quality Assurance, > Development, and Project Management." > > and the intent is that, ideally, some or all of those groups should be > represented at blocker review meetings. Lately, it seems that this > hasn't been happening as much as it used to. We used to have fairly > solid attendance from releng, FPL / FPM, and interested development > parties. Recently it seems like these groups just aren't terribly > interested in showing up and the meetings tend to be all or mostly QA > people. And we go ahead and do the evaluations, because hell, someone > has to. But as the procedure clearly states, it's not a QA-owned > process, it's meant to be collaborative, and the onus is really on other > interested parties to _show up and take part_. We do usually ping > #anaconda when blocker review starts up and ask if anyone wants to take > part; often no-one does. I think it's seen as taking away precious > development time. It would be really nice if we starting getting more > developers and Robyn and Jaroslav showing up for more blocker reviews. > Admittedly, the meetings do get very long sometimes, but we've found it > pretty hard to resolve that (reviewing 30 blockers is going to take a > while, however you slice it). I do think it helps to have multiple > perspectives, and that QA folks often tend to be more 'liberal' about > accepting blockers than other groups; it would definitely be useful to > have other groups present to act as a check on this. > > It might be an idea to start CCing the blocker meeting announcements out > a bit more widely, to help with this. Or heck, we could also have some > non-QA person run the meetings for a while, give it to FPL or FPM or > someone. That would liven things up. The SOP can say whatever it wants, but what actually happens in practice matters. People don't go to the meetings because they take too long. I'm going to make a comparison to RHEL process again, but I think it's useful here. Meetings have a hard time limit. There is only so much work a person can do in a day and it's ridiculous to keep going through bug status over and over and over just because it's there. When RHEL meetings get in to "bugzilla bingo", they cover what can be covered in the allotted time. What isn't covered isn't forgotten, just brought up at the next meeting. And from my own point of view, I've never seen your invites in #anaconda simply because I do not look at IRC unless addressed by my nick. I can't. I don't have time to sit and watch the text scroll. The length of this reply that you sent....I get hundreds of those per day that I'm expected to read and respond to. Plus all the meetings I'm scheduled for on a daily basis. This is not me whining, just trying to explain how I make the most effective use of the resources I have. Direct invites to these meetings would help me work them in to my schedule and perhaps even attend. -- David Cantrell <dcantrell@xxxxxxxxxx> Manager, Installer Engineering Team Red Hat, Inc. | Westford, MA | EST5EDT _______________________________________________ Anaconda-devel-list mailing list Anaconda-devel-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/anaconda-devel-list
