On Wed, Mar 10, 2010 at 11:08 AM, Stephen John Smoogen <smooge@xxxxxxxxx> wrote: > On Tue, Mar 9, 2010 at 10:48 PM, Jon Masters <jonathan@xxxxxxxxxxxxxx> wrote: >> On Tue, 2010-03-09 at 17:30 -0500, Chris Ball wrote: >> >>> Now, there's a reasonable argument that says that Fedora users without >>> FAS accounts didn't vote for FESCo, so it's still legitimate to ask >>> *those* users what they think. The impossibility of reaching such a >>> group of users without incorporating selection bias would turn me off >>> from trying to do that -- it would be nice if we could find out how >>> the entire Fedora-using world feels about updates, but that's not >>> actually plausible. Even just the set of Fedora users who visit >>> http://fedoraproject.org/ is significantly selection-biased already, >>> in my opinion. >> >> You could argue that only certain really interested parties will look at >> fedoraproject.org, but that's why I was suggesting the start page or >> firstboot (bad idea, I know), or something people will see when they >> first open a browser or use a system. Not just the die hards. >> >> My personal opinion is that user feedback in general is a good thing, >> even if you ultimately choose to disregard it. And I think putting a >> survey up for 6 months (e.g. all of F13) would give plenty of time to do >> reasonably useful statistical analysis of opinions. Shove in a few other >> more generic questions if there are any others - e.g. "what do you use >> Fedora for anyway?". Wouldn't that be good to know :) >> >> Jon. >> >> P.S. The government is elected, doesn't mean they don't still hold a >> census every decade to find out who their users are. >> > > Ok just basic statistics here (I will defer to Jeff Spaleta or Diana > or other gurus)... but your analogy and survey need improvement to > have statistical validity. In these sort of surveys you either need to > survey everyone OR survey a random sample that are not self-selected. > > Valid census data is one that has close to 100% coverage within some > statistical deltas. It is probably the most valid data because of > that, however it can be gammed if the people taking don't take it > seriously. [The statistical deltas are the things that everyone argues > about saying you can't trust the data... but well we are all comrade > scientists here (thats supposed to be a funny)] > > Surveys are valid within some margin of error and 'confidence level' > as long as the data can be shown not be self-selecting, the questions > properly phrased, and a study of various groups being polled is known > (reference to census material to know that if you randomly call X > people in Y region you will get Z% of sub-group.) You have to work out > how many people you tried to survey, how many completed the survey, > how much confidence you have in the population etc. > > For a group as small as Fedora (less than 50,000 registered users who > you could survey) you would need to survey that actually is much > larger than standard. An initial survey would probably want to have a > confidence level of 95% and a confidence interval of +/- 5%. If after > the survey your percentages are within that error bar (47% for/48% > against..) you need to resurvey with a larger number. So for a first > step, we would need to get a survey list made, have the questions > reworded/etc to meet various survey tests, and then randomly pick a > population and survey them (I am over simplifying here... there are > various steps required I am not sure of). > > That looks to be about 400 people need to randomly selected and > complete the survey (for +/- 5%). to get down to 1% you would need to > get 6500 people. I don't think that the near-impossibility of having a statistically sound sample that would hold up in a court of law means that we shouldn't at least get a feel for who users are by doing a survey. Certainly, an somewhat unstatistically-sound sampling is much better than all of us guessing, is it not? :) > > > > -- > Stephen J Smoogen. > > Ah, but a man's reach should exceed his grasp. Or what's a heaven for? > -- Robert Browning > -- > devel mailing list > devel@xxxxxxxxxxxxxxxxxxxxxxx > https://admin.fedoraproject.org/mailman/listinfo/devel > -- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel
