On Tue, 2012-02-21 at 00:11 +0000, "Jóhann B. Guðmundsson" wrote: > On 02/20/2012 11:42 PM, Adam Williamson wrote: > > I wish I had better background on it, but I'm pretty sure we did discuss > > having an explicit criterion before going with the 'vague' paragraph > > instead. All that mail (by me) says is that I didn't think a specific > > criterion 'worked well', which is pretty useless looking back - sigh. > > > > One problem I do remember is that, in practice, we don't really want to > > block the release if a single extremely obscure keyboard layout turns > > out to be broken at the last minute; that's one factor in favor of the > > vague hand-wavey judgment call paragraph. We want to block only on > > reasonably popularly-used layouts, your Frenches and Germans and > > whatevers, but it's a bit hard to write a criterion that properly > > restricts the list without being too restrictive. > > > > Still, I'm not really against the criterion, just trying to remember the > > thought process from before. > > If I remember correctly the vague part was put there as an escape clause > to go to if enough number of community members voiced up for their > layout not working to block the release. ( which basically is what you > say there with "popular" ) Yes, indeed, I kinda 'assumed' that knowledge in my post but should have made it explicit: the function of that paragraph is to give us an explicit thing to point to to justify making judgment calls about bugs which break the criteria _only in specific configurations_. The question now is whether it's appropriate to always simply handle keymap breakage by that mechanism - consider it a conditional breakage of the encrypted partition criterion and judge based on how serious the breakage is and how many keymaps it affects - or whether we should actually add a specific criterion for keymaps. > The fact is that we cant implement any kind of formal test procedure to > test all available layouts as in we cant really cover any other layout > then our current community members use ( I for one cant use Chinese, or > Japanese layouts as an example ). It is actually possible to test a keyboard map you don't really know how to use, though a bit painful. The only things you need to type in a normal install are passwords. All you need to do to test a keymap is to look at a diagram of the keymap - GNOME can help with that - and identify keys which are different from en-US, then use those keys to make up your passwords. Then you can tell whether the keymap is properly used because if it isn't, the password entry will fail. You can select any keyboard layout you like, even one which doesn't match your actual physical keyboard at all, and you'll still usually be able to complete an install well enough to test that the correct layout is used at all points. > Due to the fact the we cant cover them all, implementing such criteria > in any shape or form would force us to start chosing which layouts to > test which is not fair to others + community members on different > layout are already reporting any issues that they find which begs the > question what benefits would it bring implementing such criteria. Well, we do have a precedent for having criteria which are not fully testable. One of the final criteria is "The installer must be able to create and install to any workable partition layout using any file system offered in a default installer configuration, LVM, software, hardware or BIOS RAID, or combination of the above" - that's impossible to test fully. We've always simply interpreted this that we will test as many layouts as possible, and any bug roughly of the form 'I tried to create <some valid partition layout> and anaconda crashed' that comes to our attention will be accepted as a blocker. I'm not sure 'fairness' in this way is really worth applying to QA. After all, it's a truism that we can _never_ test everything. It doesn't really mean we're being 'unfair' to the things we don't have time to test. The way I see it, testing one layout is better than testing none, testing two is better than testing one, testing three is better than testing two...it's never really 'unfair' to the ones we don't manage to test. Say in Scenario A we test only English, and in Scenario B we test English, French, German and Japanese. We don't test Austrian in either case. It's not like Austrian's any *worse* off in Scenario B than it was in Scenario A... > I for one aggree with Adam that broken keyboard layout should not block > the release ( as I was uhum when this decition was made in the first > place If I can recall correctly me and Dennis vote hard -1 on the > relevant go/no go meeting but to no prevail since we got outnumbered by > +1 by community members with broken keyboard layout which lead to the > vague thingy ) however if such criteria such be implemented I ack the > proposed proposal... Thanks for your thoughts! -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
