Actually this represents currently accepted engineering practice in the usability field. According to Nielsen you can identify 80% of issues with small scale studies of 5 people. The point here is not to create a statistically representative sample, it is to identity the chief issues that may be of concern. You should always do a pre-survey before you do the main run or else you will fail to ask the right questions. Now I do have a big problem with the practice of publishing 'scientific studies' on the basis of small sample sizes. You can get a paper published in the security usability field on the basis of three samples of 9 persons each. That is actually one of the larger studies. But that is a different matter and in any case they have rather more methodological issues to deal with than sample size. > -----Original Message----- > From: ietf-bounces@xxxxxxxx [mailto:ietf-bounces@xxxxxxxx] On > Behalf Of Eric Gray > Sent: Thursday, May 08, 2008 8:09 AM > To: Mark Nottingham > Cc: ietf@xxxxxxxx > Subject: RE: IETF Website Redesign Effort > > Mark, > > Unfortunately, the way I read this, it is effectively > not the latter. The impression I am left with is that 10 > volunteers are being asked for feedback on changes. Assuming > this is an iterative process, their collective feedback could > have almost any arbitrary degree of impact on the final > result - depending on how well developed the current plan is > and how responsive the change makers expect to be to feedback. > > Hence the value of Brian's observation that there > should be some effort to determine if the general outline of > planned changes is something that most people agree to. > > Also, as has been politely hinted at already, the > sample selection method is likely to "poison" the sample. > Why the first 10 starting from a specific message at a > specific time convenient to only a narrow set of time zones? > Why only 10? Why publish the names of the > victim^H^H^H^H^Holunteers? None of these seem to be the sort > of factors one would like to see in getting a reasonable > cross section of the (probably) tens of thousands of people > who use the IETF website on a regular basis (where is that > visitor counter, or the web-site at the moment, anyway?). > > -- > Eric Gray > Principal Engineer > Ericsson > > > -----Original Message----- > > From: ietf-bounces@xxxxxxxx [mailto:ietf-bounces@xxxxxxxx] On > > Behalf Of Mark Nottingham > > Sent: Wednesday, May 07, 2008 9:53 PM > > To: Russ Housley > > Cc: ietf@xxxxxxxx > > Subject: Re: IETF Website Redesign Effort > > > > The message on-announce was worded as if these people would be > > *performing* the redesign, but I read your message below as saying > > that they'll serve a a pool of testers. > > > > I sincerely hope the truth is closer to the latter... > > > > > > On 08/05/2008, at 6:11 AM, Russ Housley wrote: > > > > > Stephane: > > > > > >>> We are inviting the first ten (10) interested members > of the IETF > > >>> community who respond to this email to become a part of > > the website > > >>> redesign team. If you are interested in assisting with > > this effort, > > >>> please respond to this email as soon as possible. > > >> > > >> It is no longer "rough consensus and running code", it is > > now "quick > > >> click and no SMTP latency". > > > > > > This approach was taken at my suggestion. > > > > > > These early reviewers are so that the Secretariat can get some > > > feedback about the new design without have a large number > of people > > > pile on. Think of it as a product test group. Once > these folks are > > > comfortable that the new design is an improvement, there > will be an > > > opportunity for broader comment. > > > > > > Russ > > > IETF Chair > > > > > > > > > _______________________________________________ > > > IETF mailing list > > > IETF@xxxxxxxx > > > https://www.ietf.org/mailman/listinfo/ietf > > > > > > -- > > Mark Nottingham http://www.mnot.net/ > > > > _______________________________________________ > > IETF mailing list > > IETF@xxxxxxxx > > https://www.ietf.org/mailman/listinfo/ietf > > > _______________________________________________ > IETF mailing list > IETF@xxxxxxxx > https://www.ietf.org/mailman/listinfo/ietf > _______________________________________________ IETF mailing list IETF@xxxxxxxx https://www.ietf.org/mailman/listinfo/ietf