--On Monday, April 29, 2013 09:55 +0100 Stewart Bryant <stbryant@xxxxxxxxx> wrote: >> The question that people are asking is why the diversity of >> the IETF leadership doesn't reflect the diversity of _the >> IETF_. >> > The evidence seems to be that human's are terrible at > "guessing" > statistics, and the only statistics that are reliable as those > objectively gathered and subjected to rigorous statistical > analysis. I mostly agree with this, but it means that attempts at statistical measurement of populations we can't really characterize are irrelevant. In particular, as soon as one talks about the "diversity of _the IETF_", one is talking about the participant population. There is no evidence at all, and some evidences to the contrary, that the attendee population is a good surrogate (approximation to a random sample, if you prefer) for the participant population. Making that assumption by polling or measuring the attendee function and assuming it is representative of the IETF may introduce far more biases than most of what we are talking about. > You can often see this in human assessments of risk. It is > also in the nature of statistics that you get long runs of > outliers, and > that only when you take a long view to you see the averages you > would expect. Again Humans are terrible with this, assuming > for example that a coin that comes up heads 10 times in a row > the assumption is that this is bias, and not a normal > statistical > variation that you would expect in an infinite number of > throws. On the other hand, as a loyal empirical Bayesian, I suggest that, if I observe a run of 10 heads and, as a result, bet on the next toss being heads, I am somewhat more likely to carry home my winnings at the end of the day that you are if you continue to bet on a 50-50 chance no matter how long the run gets... _even_ if the rules are normal statistical variation. Now, after an infinite number of coin tosses occur, you may be proven correct, but part of the reason for that Bayesian judgment (or a judgment based on moving average properties of the time series) is that few of us are going to be able to wait for that infinite number of tosses. > It would be useful to the discussion if we could see data on > diversity > that was the output of a rigorous statistical analysis. i.e. > one that > included a confidence analysis and not a simple average in a > few spot years. I agree. But I also suggest that humans are pretty good at binary comparisons and some longitudinal relationships that do not involve population samples. For example, with no effort to compare the population statistics of the IESG with the population statistics of the IETF (the precise comparison that is most susceptible to the statistical problems both of us are concerned about), it is easy to look at IESG membership longitudinally and observe that, between the early 1990s and 2010, there were always at least one, and often two or three, women on the IESG. Since then, zero. Now, based on around 17 years of moving average, I feel somewhat justified statistically in believing that something odd is happening. I would feel much more justified if we went a couple years more with no change in our procedures and how we think about things and the "zero women" trend continued, but that illustrates the other problems with this sort of analysis and an attempt to base it on population statistics, especially the population statistics of experimental design. First, our having these discussions have, I believe, already increased sensitivities to the issues and maybe even how the community thinks about it. If we end up with a woman or three on the IESG a year from now, it will basically be impossible to know whether that was -- simply a return to normal behavior after a period of deviation that could be attributed to statistical variation or -- whether it was because this discussion was effectively a consciousness-raising exercise that changed how decisions are made. The second issue is that, as in a clinical trial in which it becomes obvious (with all of those subjective human judgments as well as strict statistical ones) that one of the treatment groups is doing much better than others, it may be socially and morally unacceptable to continue the experiment in order to get cleaner statistical results. --On Monday, April 29, 2013 06:14 +0000 Christian Huitema <huitema@xxxxxxxxxxxxx> wrote: > Certainly useful, but it is easy to inject one's own bias into > such processes, and to overlook other factors. I may be > biased, but I have the impression that the largest source of > bias in IESG selection is the need to secure funding for the > job, which effectively self-select people working for large > companies making networking products. Or at least large companies and mostly those with a significant stake in the Internet. I agree with this impression. In principle, we could separate gender (or other) bias from this by comparing those who are willing to be candidates with those who are selected. I don't think that works in practice unless we require people to get binding company approval before agreeing to be candidates. We have never done that. Ignoring the other bad effects it might have, it is a rational strategy for many people to agree to be candidates after only a minimal discussion with their organizations and then to have the serious discussion only after the Nomcom's "please confirm that you are still really available" note arrives. > Gender may be the least > of the problems there; there are other dimensions of > diversity, e.g. academic vs. industry, network equipment > versus internet service providers, software versus hardware, > etc. Only a fraction of these segments can afford to have > someone working full-time on the IESG. Now, having to work > full time is a bit much for a volunteer position, and we may > want to consider ways to remedy that. Even within a company, there may be biases about who gets the approval. On the gender issue, we might be seeing symptoms of something that is actually positive for the women engineers although bad for the IETF. A series of analyses[1] suggest that, in standards bodies that are mostly working on mature technologies, few companies consider it a good investment to send design-capable people with good management skills to standards meetings, much less to have them commit to standards management activities. When the technologies are less mature, it makes more sense for organizations to commit design and implementation talent for standards development and even for standards management. Marshall Rose memorably suggested that those trends and others led to SDOs that contain a lot of "Goers" rather than a lot of designer-implementer "Doers". If one were to hypothesize (I have no idea whether this is true) that many of the women with design engineer capabilities who attend IETF early in their careers have those talents recognized by their companies and end up in positions that make it unattractive to send them to the IETF --much to contribute all of their time for a minimum of two to four years-- then we just don't see them on the IETF. Even if an exactly equal number of incoming men have the same talents and the same recognition and promotion pattern, the smaller number of incoming women would predict an IESG that was largely or entirely men. Another possible corollary to Marshall's analysis and the earlier studies is that the leadership of any SDO will gradually converge on entirely Go-ers or "professional standardizers", simply because they are less expensive --in terms of loss of skilled time for an extended period as well as more direct costs-- than design or development talent. That particular issue isn't related to most of what we've been discussing as "diversity" but it may be another very important and related issue. I don't think the IETF is there yet but I've wondered if we are on the downhill slope. best, john [1] These analyses go back long before Rose contributed the much more colorful interpretation and vocabulary to the discussion that I've invoked above.