On 1/28/21 4:46 AM, Eliot Lear wrote:
On 28 Jan 2021, at 10:18, Keith Moore <moore@xxxxxxxxxxxxxxxxxxxx> wrote:
If you give people a set of demographics questions, the questions you pick bias the responses you get. I suspect it's more useful to ask everyone what they think are the barriers to their contributing or seeking leadership roles. The answers probably won't cluster to fit into tidy graphs and might not confirm people's preconceptions, but they'll be more informative.
Let’s take some examples:
One person says “there are no barriers. I am comfortable participating as is.” Another person says, “The meetings are never convenient for me to attend.” This tells us nothing useful, even in aggregate. But if we ask an additional question, “Where do you reside?” And the people answering the former are primarily in the US and Europe, and the people answering the latter are in the Southern Hemisphere, then we know we have a problem with not serving the needs of people in the Southern Hemisphere.
If we ask the question, “Do you feel welcomed at the IETF?” and some people answer “yes” and some people answer “no”, again, this tells us nothing. But if the people who say “yes” are primarily white men from Europe and North America and the people who say “no” are everyone else, we clearly have a broad problem. But perhaps it’s more nuanced. Maybe it’s a language problem. Maybe the mix in demographics doesn’t indicate clean groupings.
All of this is possible, but all require establishing some demographics. By the way, establishing a set of groupings now doesn’t mean we can’t learn about other groupings either in the process or later.
Of course it's necessary to carefully construct the questions to
be asked if you want useful results. So don't ask, for example,
"Is it convenient for you to attend meetings?" without also
asking why. Try to avoid inferring the reason why. After all
these questions are as much about perception as reality, because a
perceived barrier to participation discourages participation.
The problem I consistently see with surveys of any kind is that they are constructed in such a way as to confirm the preconceptions of the party asking the questions, or to make that party look better than it might deserve. If the questions asked are primarily or even significantly about "demographics" like geography, race, and gender, the survey will be sensitive to those variables. But it will be insensitive to things which may be even more significant effects on diversity of IETF participation.
Keith
