Re: Survey design and interpretation (was: Re: Final reminder: Please fill out our survey on IETF authoring tools)

[IETF Executive Director <exec-director@xxxxxxxx>]

On 10/11/2020, at 4:08 AM, Ned Freed <ned.freed@xxxxxxxxxxx> wrote:

Hi Ned

On 5/11/2020, at 10:10 AM, Ned Freed <ned.freed@xxxxxxxxxxx> wrote:

(It was at the point of trying to determine if I had crossed the 50 threshhold
that I decided to stop googling and start guessing.)

And perhaps more to the point, why ask the question at all? Does authoring more
stuff mean you're better judge of the tools to use? Or worse?

We can see from Datatracker how many submissions there have been over the same period and bucket those in the same way as the survey.  Then we can compare the data from the survey to see what coverage we have.  i.e. if there are 500+ who submitted >20 times and we only captured data from 5 of them then that’s a concern.

Apparently you missed my main point, which was that the pepople who have
learned over time to operate the process in a fashion that lets them  produce
lots of documents may not be the best source of information about how to
improve the process for those who have not managed to do this.

And I think you’ve missed my point.  The fact that we are asking how many documents some has produced does not mean that we weight those higher.  I was aiming to detect skewing rather than correct for it.  

If you want to give higher weight to the answers from those that have produced a few documents then we need to know a) how many documents a respondent has produced; and b) if the responses overall are already weighted more towards one end of the spectrum of the other. 

Both use cases require that question about the number of documents.

Jay

-- 
Jay Daley

IETF Executive Director
jay@xxxxxxxx